lightning failed at CreateOrUpdateRegionLabelRule when one PD node is unavailable #46688
Description
Bug Report
Please answer these questions before submitting your issue. Thanks!
1. Minimal reproduce step (Required)
randomly inject some PD error to one node
2. What did you expect to see? (Required)
lightning import success
3. What did you see instead (Required)
Post "http://tc-a-pd:2379/pd/api/v1/config/region-label/rule": dial tcp 10.67.250.186:2379: connect: connection refused
github.com/pingcap/errors.AddStack
/go/pkg/mod/github.com/pingcap/[email protected]/errors.go:174
github.com/pingcap/errors.Trace
/go/pkg/mod/github.com/pingcap/[email protected]/juju_adaptor.go:15
github.com/pingcap/tidb/br/pkg/pdutil.pdRequestWithCode
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/pdutil/pd.go:202
github.com/pingcap/tidb/br/pkg/pdutil.pdRequest
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/pdutil/pd.go:153
github.com/pingcap/tidb/br/pkg/pdutil.(*PdController).CreateOrUpdateRegionLabelRule
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/pdutil/pd.go:948
github.com/pingcap/tidb/br/pkg/pdutil.(*PdController).pauseSchedulerByKeyRangeWithTTL
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/pdutil/pd.go:1010
github.com/pingcap/tidb/br/pkg/pdutil.(*PdController).PauseSchedulersByKeyRange
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/pdutil/pd.go:990
github.com/pingcap/tidb/br/pkg/lightning/backend/local.(*local).ImportEngine
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/backend/local/local.go:1645
github.com/pingcap/tidb/br/pkg/lightning/backend.(*ClosedEngine).Import
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/backend/backend.go:477
github.com/pingcap/tidb/br/pkg/lightning/restore.(*TableRestore).importKV
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/restore/table_restore.go:1039
github.com/pingcap/tidb/br/pkg/lightning/restore.(*TableRestore).importEngine
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/restore/table_restore.go:705
github.com/pingcap/tidb/br/pkg/lightning/restore.(*TableRestore).restoreEngines.func3
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/restore/table_restore.go:333
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1594