storage: backport the stale read enhancement and bug fix to release 6.5

[cfzjywxk]

Enhancement

The tidb repo

• store/copr: fix missting Request.TxnScope

  • 6.5.3 store/copr: fix missting Request.TxnScope (#43367) #43553
  • 7.1.0 store/copr: fix missting Request.TxnScope (#43367) #43374
  • master store/copr: fix missting Request.TxnScope #43367

• planner: recalculate as-of ts of staleread when plan is cached

  • 6.5.3 planner: recalculate as-of ts of staleread when plan is cached (#43204) #43550
  • 7.1.0 planner: recalculate as-of ts of staleread when plan is cached (#43204) #43257
  • master planner: recalculate as-of ts of staleread when plan is cached #43204

• grafana: add traffic metrics for staleread traffic

  • 6.5.3 grafana: add traffic metrics for staleread traffic (#43324) #44209
  • 7.1.1 *: update tikv/client-go and fix stale read metrics in grafana #45340
  • master grafana: add traffic metrics for staleread traffic #43324

• store/copr: disable stale read when meeting lock for cop

  • 6.5.3 store/copr: disable stale read when meeting lock for cop (#44240) #44251
  • 7.1.1 store/copr: disable stale read when meeting lock for cop (#44240) #44253
  • master store/copr: disable stale read when meeting lock for cop #44240

• sessiontxn/staleread: more accurate stale ts

  • 6.5.3 sessiontxn/staleread: more accurate stale ts (#44272) #44281
  • 7.1 sessiontxn/staleread: more accurate stale ts (#44272) #45760
  • master sessiontxn/staleread: more accurate stale ts #44272

The client-go repo

• avoid dataIsNotReady error while retrying stale read on the leader

  • 6.5.3 avoid dataIsNotReady error while retrying stale read on the leader(#765) tikv/client-go#789
  • 7.1.0 same as master
  • master avoid dataIsNotReady error while retrying stale read on the leader tikv/client-go#765

• Remove stale-read flag after resolving lock

  • 6.5.3 Backport the stale read optimizations to 6.5 tikv/client-go#818
  • 7.1 same as master
  • master Remove stale-read flag after resolving lock tikv/client-go#792

• add metrics for stale-read traffic

  • 6.5.3 Backport the stale read optimizations to 6.5 tikv/client-go#818
  • 7.1.0 same as master
  • master add metrics for stale-read traffic tikv/client-go#776

• batch_client: auto re-connect idle connection when wait connection ready

  • 6.5.3 batch_client: auto re-connect idle connection when wait connection ready tikv/client-go#833
  • 7.1 Add gRPC connection monitor and fix related bug tikv/client-go#665
  • master Add gRPC connection monitor and fix related bug tikv/client-go#665

• reload region cache when store is resolved from invalid status

  • 6.5.4 reload region cache when store is resolved from invalid status (#843) tikv/client-go#846
  • 7.1 Cherry pick some patches to 7.1 tikv/client-go#940
  • master reload region cache when store is resolved from invalid status tikv/client-go#843

• *: refine non-global stale-read request retry logic

  • 6.5.4 *: refine non-global stale-read request retry logic tikv/client-go#851
  • 7.1.1 *: refine non-global stale-read request retry logic (#851) tikv/client-go#880
  • master *: refine non-global stale-read request retry logic tikv/client-go#863

• *: fix stale read ops metric

  • 6.5.4 *: fix stale read ops metric tikv/client-go#878
  • 7.1.1 *: fix stale read ops metric (#878) tikv/client-go#886
  • master *: fix stale read ops metric (#878) tikv/client-go#889

• handle mismatch peer id region error

  • 6.5.4 handle mismatch peer id region error tikv/client-go#888
  • 7.1.1 handle mismatch peer id region error tikv/client-go#885 & Match error message correctly tikv/client-go#887
  • master handle mismatch peer id region error tikv/client-go#935

• fix unexpected slow query during GC running after stop 1 tikv-server

  • 6.5.4 fix unexpected slow query during GC running after stop 1 tikv-server tikv/client-go#899
  • 7.1 fix unexpected slow query during GC running after stop 1 tikv-server … tikv/client-go#930
  • master fix unexpected slow query during GC running after stop 1 tikv-server (#899) tikv/client-go#909

• Resume the max retry check for stale read processing

  • 6.5.4 Resume the max retry check for stale read processing tikv/client-go#903
  • 7.1 Cherry pick some patches to 7.1 tikv/client-go#940
  • master Resume max retry time check for stale read retry with leader option(#903) tikv/client-go#911

• fallback to follower when leader is busy

  • 6.5.4 fallback to follower when leader is busy tikv/client-go#916
  • 7.1 Cherry pick some patches to 7.1 tikv/client-go#940
  • master fallback to follower when leader is busy (#916) tikv/client-go#923

• add more log for diagnose

  • 6.5.4 add more log for diagnose tikv/client-go#914
  • 7.1 add more log for diagnose (#915) tikv/client-go#931
  • master add more log for diagnose tikv/client-go#915

• add region cache state test & fix some issues of replica selector

  • 6.5.4 add region cache state test & fix some issues of replica selector tikv/client-go#910
  • 7.1 Cherry pick some patches to 7.1 tikv/client-go#940
  • master add region cache state test & fix some issues of replica selector (#910)  tikv/client-go#942

• use context logger as much as possible

  • 6.5 TODO
  • 7.1 use context logger as much as possible (#908) tikv/client-go#932
  • master use context logger as much as possible tikv/client-go#908

The tikv repo

• add gzip compression for check_leader requests
  • 6.5.3 add gzip compression for check_leader requests (#14721) tikv/tikv#14824
  • 7.1 add gzip compression for check_leader requests (#14721) tikv/tikv#14825
  • master add gzip compression for check_leader requests tikv/tikv#14721