Skip to content

add-index/import-into: data/region skew after ingest using global-sort #60077

@D3Hunter

Description

@D3Hunter

Bug Report

What did you do?

import a large table or add-index for a large table, say 200T, we might see data/region skew
the reason is we are using ScatterRegions API to scatter, but PD doesn't return the regions that failed to scatter, see tikv/pd#9088, right now the scatter client will take all of them as success, see

resp, err := c.client.ScatterRegions(ctx, regionsID, opt.WithSkipStoreLimit())
if err != nil {
return err
}
if pbErr := resp.GetHeader().GetError(); pbErr.GetType() != pdpb.ErrorType_OK {
return errors.Annotatef(berrors.ErrPDInvalidResponse,
"pd returns error during batch scattering: %s", pbErr)
}

from PD log we can see

[2025/03/12 12:28:21.045 +00:00] [WARN] [region_scatterer.go:280] ["region not replicated during scatter"] [region-id=6706317]
[2025/03/12 12:28:21.045 +00:00] [WARN] [region_scatterer.go:280] ["region not replicated during scatter"] [region-id=6705667]
[2025/03/12 12:28:21.045 +00:00] [WARN] [region_scatterer.go:280] ["region not replicated during scatter"] [region-id=6706327]
[2025/03/12 12:28:21.045 +00:00] [WARN] [region_scatterer.go:280] ["region not replicated during scatter"] [region-id=6706052]

What did you expect to see?

region/data skew

What did you see instead?

no skew

What version of PD are you using (pd-server -V)?

8.1.2

Metadata

Metadata

Assignees

Labels

affects-8.1This bug affects the 8.1.x(LTS) versions.affects-8.5This bug affects the 8.5.x(LTS) versions.component/ddlThis issue is related to DDL of TiDB.severity/majortype/bugThe issue is confirmed as a bug.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions