Skip to content

DM uses lightning "replace" deduplication mode and failed #52886

@lance6716

Description

@lance6716

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

https://do.pingcap.net/jenkins/blue/organizations/jenkins/pingcap%2Ftiflow%2Frelease-7.5%2Fpull_dm_integration_test/detail/pull_dm_integration_test/249/pipeline/

github.com/pingcap/tidb v1.1.0-beta.0.20240410073751-e540f0fd00b5

2. What did you expect to see? (Required)

3. What did you see instead (Required)

...
[2024/04/25 16:36:05.603 +08:00] [DEBUG] [duplicate.go:1159] [getLatestValue] [task=test] [unit=lightning-load] [table=`lightning_mode`.`dup`] [key=74800000000000007E5F728000000000000001] [value="gAABAAAAAgEAYQ=="] []
[2024/04/25 16:36:05.603 +08:00] [DEBUG] [tables.go:1003] [addRecord] [key=74800000000000007e5f728000000000000001]
[2024/04/25 16:36:05.603 +08:00] [DEBUG] [errormanager.go:640] ["got group raw_key, raw_value from table"] [task=test] [unit=lightning-load] [raw_key=74800000000000007E5F728000000000000001] [raw_value="gAABAAAAAgIAYTI="]
[2024/04/25 16:36:05.603 +08:00] [DEBUG] [tables.go:1003] [addRecord] [key=74800000000000007e5f728000000000000001]
[2024/04/25 16:36:05.603 +08:00] [DEBUG] [errormanager.go:702] ["got encoded KV"] [task=test] [unit=lightning-load] [key=74800000000000007E5F728000000000000001] [value=CAQCBGEy]
[2024/04/25 16:36:05.604 +08:00] [DEBUG] [duplicate.go:1159] [getLatestValue] [task=test] [unit=lightning-load] [table=`lightning_mode`.`dup`] [key=74800000000000007E5F728000000000000001] [value="gAABAAAAAgEAYQ=="] []
[2024/04/25 16:36:05.604 +08:00] [DEBUG] [errormanager.go:640] ["got group raw_key, raw_value from table"] [task=test] [unit=lightning-load] [raw_key=74800000000000007E5F728000000000000002] [raw_value="gAABAAAAAgEAYg=="]
[2024/04/25 16:36:05.605 +08:00] [DEBUG] [duplicate.go:1159] [getLatestValue] [task=test] [unit=lightning-load] [table=`lightning_mode`.`dup`] [key=74800000000000007E5F728000000000000002] [value="gAABAAAAAgEAYg=="] []
[2024/04/25 16:36:05.605 +08:00] [DEBUG] [tables.go:1003] [addRecord] [key=74800000000000007e5f728000000000000002]
[2024/04/25 16:36:05.605 +08:00] [DEBUG] [errormanager.go:640] ["got group raw_key, raw_value from table"] [task=test] [unit=lightning-load] [raw_key=74800000000000007E5F728000000000000002] [raw_value="gAABAAAAAgIAYjI="]
[2024/04/25 16:36:05.605 +08:00] [DEBUG] [tables.go:1003] [addRecord] [key=74800000000000007e5f728000000000000002]
[2024/04/25 16:36:05.605 +08:00] [DEBUG] [errormanager.go:702] ["got encoded KV"] [task=test] [unit=lightning-load] [key=74800000000000007E5F728000000000000002] [value=CAQCBGIy]
[2024/04/25 16:36:05.606 +08:00] [DEBUG] [duplicate.go:1159] [getLatestValue] [task=test] [unit=lightning-load] [table=`lightning_mode`.`dup`] [key=74800000000000007E5F728000000000000002] [value="gAABAAAAAgEAYg=="] []
[2024/04/25 16:36:05.607 +08:00] [DEBUG] [errormanager.go:640] ["got group raw_key, raw_value from table"] [task=test] [unit=lightning-load] [raw_key=74800000000000007E5F728000000000000003] [raw_value="gAABAAAAAgEAYw=="]
[2024/04/25 16:36:05.607 +08:00] [DEBUG] [duplicate.go:1159] [getLatestValue] [task=test] [unit=lightning-load] [table=`lightning_mode`.`dup`] [key=74800000000000007E5F728000000000000003] [value="gAABAAAAAgEAYw=="] []
[2024/04/25 16:36:05.607 +08:00] [DEBUG] [tables.go:1003] [addRecord] [key=74800000000000007e5f728000000000000003]
[2024/04/25 16:36:05.607 +08:00] [DEBUG] [errormanager.go:640] ["got group raw_key, raw_value from table"] [task=test] [unit=lightning-load] [raw_key=74800000000000007E5F728000000000000003] [raw_value="gAABAAAAAgIAYzI="]
[2024/04/25 16:36:05.608 +08:00] [DEBUG] [tables.go:1003] [addRecord] [key=74800000000000007e5f728000000000000003]
[2024/04/25 16:36:05.608 +08:00] [DEBUG] [errormanager.go:702] ["got encoded KV"] [task=test] [unit=lightning-load] [key=74800000000000007E5F728000000000000003] [value=CAQCBGMy]
[2024/04/25 16:36:05.608 +08:00] [DEBUG] [duplicate.go:1159] [getLatestValue] [task=test] [unit=lightning-load] [table=`lightning_mode`.`dup`] [key=74800000000000007E5F728000000000000003] [value="gAABAAAAAgEAYw=="] []
[2024/04/25 16:36:05.609 +08:00] [WARN] [util.go:201] ["delete additionally inserted rows for conflict detection 'replace' mode failed with no retry"] [task=test] [unit=lightning-load] [retryCnt=0] [error="Error 1054 (42S22): Unknown column '' in 'where clause'"]
[2024/04/25 16:36:05.610 +08:00] [ERROR] [duplicate.go:1109] ["[resolve-dupe] resolve duplicate rows failed"] [task=test] [unit=lightning-load] [table=`lightning_mode`.`dup`] [takeTime=17.378303ms] [error="delete additionally inserted rows for conflict detection 'replace' mode failed: Error 1054 (42S22): Unknown column '' in 'where clause'"]

4. What is your TiDB version? (Required)

Metadata

Metadata

Assignees

No one assigned

    Labels

    affects-7.5This bug affects the 7.5.x(LTS) versions.affects-8.1This bug affects the 8.1.x(LTS) versions.component/lightningThis issue is related to Lightning of TiDB.severity/majortype/bugThe issue is confirmed as a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions