[DNM] Upgrade TiDB dependency and refactor code to reflect upstream package and API changes. #1930

tenfyzhong · 2025-09-04T14:32:55Z

What problem does this PR solve?

Issue Number: ref #1755

What is changed and how it works?

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Questions

Will it cause performance regression or break compatibility?

Do you need to update user documentation, design documentation or monitoring documentation?

Release note

None

- Update `github.com/pingcap/tidb` to the latest version to incorporate recent fixes and features. - Adapt to `tidb` API changes, including: - Moving `infosync` package to `serverinfo`. - Relocating `ddl.JobTableID` and `ddl.HistoryTableID` to `metadef`. - Shifting `session.SetSchemaLease` to `vardef.SetSchemaLease`. - Update various other Go module dependencies to their latest compatible versions. Signed-off-by: tenfyzhong <[email protected]>

- Run `go mod tidy` to remove entries for unused or outdated dependencies. Signed-off-by: tenfyzhong <[email protected]>

ti-chi-bot · 2025-09-04T14:33:01Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign wlwilliamx for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

gemini-code-assist

Summary of Changes

Hello @tenfyzhong, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on a significant dependency upgrade, primarily for TiDB, and the subsequent refactoring required to accommodate changes in its upstream packages and APIs. The goal is to maintain compatibility and leverage improvements from the updated libraries, ensuring the project remains current with its core components.

Highlights

TiDB Dependency Upgrade: Upgraded the TiDB dependency to a newer version, which necessitated several code refactorings to align with upstream API changes.
API Refactoring: Adapted the codebase to breaking API changes in TiDB, specifically migrating from infosync to serverinfo for topology information, from ddl to metadef for DDL job IDs, and updating session.SetSchemaLease to vardef.SetSchemaLease.
Extensive Dependency Updates: Performed a broad update of various Go module dependencies, including etcd, golang.org/x packages, tikv/client-go, and gorm, ensuring the project uses more recent and stable versions of its external libraries.
Temporary Tiflow Replace Directive: Introduced a temporary replace directive in go.mod for github.com/pingcap/tiflow pointing to a personal fork, with a TODO to remove it before merging to master, indicating ongoing development or testing against a specific tiflow version.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request primarily involves upgrading TiDB dependencies and refactoring the codebase to align with upstream package and API changes. The modifications, such as updating import paths from infosync to serverinfo and ddl to metadef, are consistent and appear correct. The main point of concern is a temporary replace directive in go.mod that points to a personal fork, which must be removed before merging to the master branch.

go.mod

The Rowkey is changed after update tidb Signed-off-by: tenfyzhong <[email protected]>

- The `ObjPrefix` was incorrectly prepended with `subDir`, leading to a redundant and incorrect path prefix when combined with the `SubDir` option in `WalkDir`. - This ensures that `WalkDir` correctly searches for schema files within the specified `subDir` by using a relative prefix. Signed-off-by: tenfyzhong <[email protected]>

tenfyzhong · 2025-09-05T06:32:52Z

/retest-required

tenfyzhong · 2025-09-07T14:47:11Z

/retest-required

- Replaced `&>>` with `>> ... 2>&1` for log redirection. - This ensures that both standard output and standard error are explicitly captured and appended to the log file. - Improves clarity and ensures consistent logging behavior in the test utility script. Signed-off-by: tenfyzhong <[email protected]>

tenfyzhong · 2025-09-08T02:26:16Z

/retest-required

- Ensures compatibility with TiDB's updated IsSystemSchema function. - TiDB now expects lowercase schema names for system schema identification. Signed-off-by: tenfyzhong <[email protected]>

- Introduce a temporary `replace` directive for `github.com/pingcap/tiflow` to point to a specific fork. - This change is for local development and testing purposes only. - The `replace` directive must be removed before merging to the main branch. Signed-off-by: tenfyzhong <[email protected]>

tenfyzhong · 2025-09-08T09:54:44Z

/retest-required

tenfyzhong · 2025-09-08T15:07:36Z

/gemini review

gemini-code-assist

Code Review

This pull request upgrades the TiDB dependency and refactors the codebase to reflect upstream changes. The majority of the changes are straightforward and align with the goal of the PR. However, I've identified two critical issues that need to be addressed before merging. One is a temporary replace directive in go.mod that points to a personal fork, which must be removed. The other is a potential bug in pkg/sink/cloudstorage/path.go where an incorrect object prefix is used, which could break schema file discovery in cloud storage sinks.

gemini-code-assist · 2025-09-08T15:09:08Z

go.mod

+// DO-NOT-MERGE tenfyzhong 2025-09-02 10:28:03 remove this replace before merge to master
+replace github.com/pingcap/tiflow => github.com/tenfyzhong/tiflow v0.0.0-20250902015837-94c9f6372559


This replace directive points to a personal fork of tiflow. While this is acceptable for development, it must be removed before this pull request is merged. Merging with this directive would introduce a dependency on a personal repository, which is a security and maintenance risk. The "DO-NOT-MERGE" comment indicates awareness, but this is a critical check for merging.

pkg/sink/cloudstorage/path.go

- The `github.com/pingcap/tiflow` replace directive was a temporary measure for a specific development branch and is no longer needed. - This ensures the project uses the upstream `tiflow` module as intended. Signed-off-by: tenfyzhong <[email protected]>

ti-chi-bot · 2025-09-17T06:24:12Z

@tenfyzhong: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-check	`a42f4cc`	link	true	`/test pull-check`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

tenfyzhong added 2 commits September 4, 2025 22:26

chore(deps): Tidy go.sum file

58b1075

- Run `go mod tidy` to remove entries for unused or outdated dependencies. Signed-off-by: tenfyzhong <[email protected]>

ti-chi-bot bot added the release-note-none Denotes a PR that doesn't merit a release note. label Sep 4, 2025

ti-chi-bot bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Sep 4, 2025

gemini-code-assist bot reviewed Sep 4, 2025

View reviewed changes

go.mod Outdated Show resolved Hide resolved

test(canal): update RowKey assertion in canal_json_test

2fd11f2

The Rowkey is changed after update tidb Signed-off-by: tenfyzhong <[email protected]>

tenfyzhong changed the title ~~Upgrade TiDB dependency and refactor code to reflect upstream package and API changes.~~ [DNM] Upgrade TiDB dependency and refactor code to reflect upstream package and API changes. Sep 4, 2025

tenfyzhong added 3 commits September 8, 2025 11:39

fix(filter): Convert schema name to lowercase in IsSysSchema

92978a5

- Ensures compatibility with TiDB's updated IsSystemSchema function. - TiDB now expects lowercase schema names for system schema identification. Signed-off-by: tenfyzhong <[email protected]>

Merge branch 'master' into chroe-keyspace-dependencies

0a617b8

gemini-code-assist bot reviewed Sep 8, 2025

View reviewed changes

tenfyzhong added 2 commits September 15, 2025 13:05

Merge branch 'master' into chroe-keyspace-dependencies

90c51ff

tenfyzhong closed this Sep 17, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[DNM] Upgrade TiDB dependency and refactor code to reflect upstream package and API changes. #1930

[DNM] Upgrade TiDB dependency and refactor code to reflect upstream package and API changes. #1930

Uh oh!

tenfyzhong commented Sep 4, 2025

Uh oh!

ti-chi-bot bot commented Sep 4, 2025

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

tenfyzhong commented Sep 5, 2025

Uh oh!

tenfyzhong commented Sep 7, 2025

Uh oh!

tenfyzhong commented Sep 8, 2025

Uh oh!

tenfyzhong commented Sep 8, 2025

Uh oh!

tenfyzhong commented Sep 8, 2025

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Sep 8, 2025

Uh oh!

Uh oh!

ti-chi-bot bot commented Sep 17, 2025

Uh oh!

Uh oh!

		// DO-NOT-MERGE tenfyzhong 2025-09-02 10:28:03 remove this replace before merge to master
		replace github.com/pingcap/tiflow => github.com/tenfyzhong/tiflow v0.0.0-20250902015837-94c9f6372559

[DNM] Upgrade TiDB dependency and refactor code to reflect upstream package and API changes. #1930

[DNM] Upgrade TiDB dependency and refactor code to reflect upstream package and API changes. #1930

Uh oh!

Conversation

tenfyzhong commented Sep 4, 2025

What problem does this PR solve?

What is changed and how it works?

Check List

Tests

Questions

Will it cause performance regression or break compatibility?

Do you need to update user documentation, design documentation or monitoring documentation?

Release note

Uh oh!

ti-chi-bot bot commented Sep 4, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

tenfyzhong commented Sep 5, 2025

Uh oh!

tenfyzhong commented Sep 7, 2025

Uh oh!

tenfyzhong commented Sep 8, 2025

Uh oh!

tenfyzhong commented Sep 8, 2025

Uh oh!

tenfyzhong commented Sep 8, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Sep 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ti-chi-bot bot commented Sep 17, 2025

Uh oh!

Uh oh!