codec: fix a panic in Debezium when configuring column-selector #12210

wk989898 · 2025-07-03T09:26:19Z

What problem does this PR solve?

Issue Number: close #12208

What is changed and how it works?

When configuring the column-selector, some columns that are not selected must be ignored. Because these columns are nil and cause a panic when encoding.

This pr also fixes other integration tests. When running two SQL and checking that the same table exists, the second SQL may not replicate to the downstream.

Check List

Tests

Unit test
Integration test

Questions

Will it cause performance regression or break compatibility?

Do you need to update user documentation, design documentation or monitoring documentation?

Release note

Fix a panic when configuring the column-selector with the Debezium protocol.

gemini-code-assist

Summary of Changes

Hello @wk989898, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves a panic in the Debezium codec that occurs when using column-selector and encountering nil columns. The fix involves adding nil checks in the encoding logic to safely handle these cases. Integration tests have been added to ensure the fix works as expected and to prevent regressions.

Highlights

Bug Fix: Addresses a panic in the Debezium codec when column-selector is configured and some columns are nil.
Codec Modification: The code now includes a nil check for columns in writeDebeziumFieldValues and EncodeValue to prevent panics.
Integration Test: Integration tests were added to verify the fix, including a changefeed configuration with column selectors.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request effectively resolves a panic in the Debezium codec when using column-selector. The fix is straightforward, adding necessary nil checks for filtered columns. The changes are well-supported by a new integration test case that validates the fix. The overall implementation is solid.

tests/integration_tests/debezium_basic/run.sh

codecov · 2025-07-03T09:39:11Z

Codecov Report

Attention: Patch coverage is 0% with 4 lines in your changes missing coverage. Please review.

Project coverage is 55.0649%. Comparing base (9ffd20b) to head (75dc07d).
Report is 1 commits behind head on master.

Additional details and impacted files

Components	Coverage Δ
cdc	`59.3252% <0.0000%> (+0.0163%)`	⬆️
dm	`50.0738% <ø> (-0.0269%)`	⬇️
engine	`53.2336% <ø> (-0.0113%)`	⬇️

Flag	Coverage Δ
unit	`55.0649% <0.0000%> (-0.0054%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master     #12210        +/-   ##
================================================
- Coverage   55.0702%   55.0649%   -0.0054%     
================================================
  Files          1030       1030                
  Lines        143484     143487         +3     
================================================
- Hits          79017      79011         -6     
- Misses        58658      58668        +10     
+ Partials       5809       5808         -1

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

wk989898 · 2025-07-03T11:43:27Z

/retest

3AceShowHand · 2025-07-04T03:19:44Z

pkg/sink/codec/debezium/codec.go

 	colInfos := tableInfo.GetColInfosForRowChangedEvent()
 	writer.WriteObjectField(fieldName, func() {
 		for i, col := range cols {
+			if col == nil {


you can add one unit test to cover this

ti-chi-bot · 2025-07-04T03:31:57Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: 3AceShowHand, hongyunyan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [3AceShowHand,hongyunyan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2025-07-04T03:31:59Z

[LGTM Timeline notifier]

Timeline:

2025-07-04 03:19:06.381476604 +0000 UTC m=+1624199.104655622: ☑️ agreed by 3AceShowHand.
2025-07-04 03:31:59.121936752 +0000 UTC m=+1624971.845115719: ☑️ agreed by hongyunyan.

wk989898 · 2025-07-04T04:22:35Z

/retest

wk989898 · 2025-07-04T07:20:03Z

/retest-required

wk989898 · 2025-07-04T07:52:22Z

/retest-required

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot · 2025-07-04T08:20:34Z

In response to a cherrypick label: new pull request created to branch release-8.1: #12212.
But this PR has conflicts, please resolve them!

ti-chi-bot · 2025-07-04T08:21:12Z

In response to a cherrypick label: new pull request created to branch release-8.5: #12213.
But this PR has conflicts, please resolve them!

Signed-off-by: ti-chi-bot <[email protected]>

…) (#12213) close #12208

init

be37247

ti-chi-bot bot added do-not-merge/needs-triage-completed release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed do-not-merge/needs-triage-completed labels Jul 3, 2025

gemini-code-assist bot reviewed Jul 3, 2025

View reviewed changes

tests/integration_tests/debezium_basic/run.sh Show resolved Hide resolved

fix test

fe33914

ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jul 3, 2025

fix encoding

75dc07d

wk989898 added needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. labels Jul 4, 2025

3AceShowHand approved these changes Jul 4, 2025

View reviewed changes

ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Jul 4, 2025

3AceShowHand reviewed Jul 4, 2025

View reviewed changes

hongyunyan approved these changes Jul 4, 2025

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jul 4, 2025

ti-chi-bot bot merged commit 9bc740d into pingcap:master Jul 4, 2025
28 checks passed

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Jul 4, 2025

This is an automated cherry-pick of pingcap#12210

c64b12e

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot mentioned this pull request Jul 4, 2025

codec: fix a panic in Debezium when configuring column-selector (#12210) #12212

Open

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Jul 4, 2025

This is an automated cherry-pick of pingcap#12210

f9ca609

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot mentioned this pull request Jul 4, 2025

codec: fix a panic in Debezium when configuring column-selector (#12210) #12213

Merged

ti-chi-bot bot pushed a commit that referenced this pull request Jul 14, 2025

codec: fix a panic in Debezium when configuring column-selector (#12210…

2d8137b

…) (#12213) close #12208

wk989898 deleted the debezium-value branch September 10, 2025 03:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

codec: fix a panic in Debezium when configuring column-selector #12210

codec: fix a panic in Debezium when configuring column-selector #12210

wk989898 commented Jul 3, 2025 •

edited

Loading

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

codecov bot commented Jul 3, 2025 •

edited

Loading

Uh oh!

wk989898 commented Jul 3, 2025

Uh oh!

3AceShowHand Jul 4, 2025

Uh oh!

ti-chi-bot bot commented Jul 4, 2025

Uh oh!

ti-chi-bot bot commented Jul 4, 2025

Uh oh!

wk989898 commented Jul 4, 2025

Uh oh!

wk989898 commented Jul 4, 2025

Uh oh!

wk989898 commented Jul 4, 2025

Uh oh!

Uh oh!

ti-chi-bot commented Jul 4, 2025

Uh oh!

ti-chi-bot commented Jul 4, 2025

Uh oh!

Uh oh!

codec: fix a panic in Debezium when configuring column-selector #12210

codec: fix a panic in Debezium when configuring column-selector #12210

Conversation

wk989898 commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

What is changed and how it works?

Check List

Tests

Questions

Will it cause performance regression or break compatibility?

Do you need to update user documentation, design documentation or monitoring documentation?

Release note

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

codecov bot commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

wk989898 commented Jul 3, 2025

Uh oh!

3AceShowHand Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

ti-chi-bot bot commented Jul 4, 2025

Uh oh!

ti-chi-bot bot commented Jul 4, 2025

[LGTM Timeline notifier]

Uh oh!

wk989898 commented Jul 4, 2025

Uh oh!

wk989898 commented Jul 4, 2025

Uh oh!

wk989898 commented Jul 4, 2025

Uh oh!

Uh oh!

ti-chi-bot commented Jul 4, 2025

Uh oh!

ti-chi-bot commented Jul 4, 2025

Uh oh!

Uh oh!

wk989898 commented Jul 3, 2025 •

edited

Loading

codecov bot commented Jul 3, 2025 •

edited

Loading