Skip to content

CDC print lots of "etcd client outCh blocking too long, the etcdWorker may be stuck" and checkpoint lag metric abnormal #11884

@fubinzh

Description

@fubinzh

What did you do?

  1. create cdc changefeed
  2. run workload (use lighting tidb backend to import 10+ tables )

What did you expect to see?

CDC should be normal.

What did you see instead?

CDC print lots of WARN :
{"container":"ticdc","namespace":"cdc-testbed-tps-7653796-1-407","pod":"upstream-ticdc-0","log":"[client.go:272] ["etcd client outCh blocking too long, the etcdWorker may be stuck"] [duration=5m32.997091037s] [role=owner]","level":"WARN"}

CDC checkpoint lag abnormal:
Image

Versions of the cluster

cdc version: master, git-hash: 4538839

Metadata

Metadata

Assignees

Labels

affects-6.5This bug affects the 6.5.x(LTS) versions.affects-7.1This bug affects the 7.1.x(LTS) versions.affects-7.5This bug affects the 7.5.x(LTS) versions.affects-8.1This bug affects the 8.1.x(LTS) versions.affects-8.5This bug affects the 8.5.x(LTS) versions.area/ticdcIssues or PRs related to TiCDC.severity/majortype/bugThe issue is confirmed as a bug.type/regression

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions