Skip to content

Too many MemberList request after PD lost leader for a while #46770

@HuSharp

Description

@HuSharp

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

use. pd failpoint binary, and ingest the failpoint.

  1. enable fail point, like gRPC is throttling, cannot read from etcd.
    curl -X PUT -d 'return(10)' http://tc-pd-1.tc-pd-peer.csn-simulator-big-cluster-vd62g.svc:2379/pd/api/v1/fail/github.com/tikv/pd/pkg/etcdutil/SlowEtcdKVGet

  2. simulate pd lost leader
    curl -X PUT -d 'return("2346857576170797299")' http://tc-pd-1.tc-pd-peer.csn-simulator-big-cluster-vd62g.svc:2379/pd/api/v1/fail/github.com/tikv/pd/server/exitCampaignLeader

2. What did you expect to see? (Required)

we can add backoff mechanism to sleep when an error is encountered.

3. What did you see instead (Required)

Grpc request GetMember keeps high:
image

4. What is your TiDB version? (Required)

master

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions