Skip to content

In [[filter.event-filters]], with "rename" DDLs the matcher is filtered based on the new name rather than old name #11946

@kennytm

Description

@kennytm

What did you do?

  1. Create 3 tables:

    use test;
    create table a1 (pk int primary key, val int);
    create table a2 like a1;
    create table b3 like a1;
  2. Create a changefeed with configuration:

    [filter]
    rules = ['test.a*', 'test.b*']
    [[filter.event-filters]]
    matcher = ['test.a*']
    ignore-event = ['all ddl']
    cdc cli changefeed create --sink-uri='file:///tmp/test-cdc-storage?protocol=canal-json' --config=cfg.toml -c xxx
  3. Perform the following renames:

    rename table a1 to b1;
    rename table a2 to c2;
    rename table b3 to a3;

What did you expect to see?

As explained in https://docs.pingcap.com/tidb/stable/ticdc-ddl#ddl-replication-considerations-for-renaming-tables, since filter.rules is looking at the old name, we expect filter.event-filters[].matcher is also looking at the old name too. That means the downstream should ignore the first two renames and only replicate the last.

-- rename table a1 to b1;  -- ignored
-- rename table a2 to c2;  -- ignored
rename table b3 to a3;  -- replicated

What did you see instead?

The filter.event-filters[].matcher rule actually looks at the new name, so instead the first two DDLs are replicated while the last one is ignored.

rename table a1 to b1;  -- incorrectly replicated
rename table a2 to c2;  -- incorrectly replicated
-- rename table b3 to a3;  -- incorrectly ignored

Versions of the cluster

Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

v8.5.0

Upstream TiKV version (execute tikv-server --version):

v8.5.0

TiCDC version (execute cdc version):

v8.5.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    affects-6.5This bug affects the 6.5.x(LTS) versions.affects-7.1This bug affects the 7.1.x(LTS) versions.affects-7.5This bug affects the 7.5.x(LTS) versions.affects-8.1This bug affects the 8.1.x(LTS) versions.affects-8.5This bug affects the 8.5.x(LTS) versions.area/ticdcIssues or PRs related to TiCDC.found/gsseverity/moderatetype/bugThe issue is confirmed as a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions