Skip to content

Global sort add index should use fixed memory size #52328

@tangenta

Description

@tangenta

Enhancement

Currently, we use available memory to determine writer memory size when read-index stage starts.

memTotal, err := memory.MemTotal()
if err != nil {
return 0, err
}
memUsed, err := memory.MemUsed()
if err != nil {
return 0, err
}
memAvailable := memTotal - memUsed

This value can be different under different workloads. As a result, we may encounter the error "merge sort multi parts exceed 10000", because we cannot make sure the following formula if memsize is unstable:

partsize * 10000 >= 4000 * memsize

Metadata

Metadata

Assignees

No one assigned

    Labels

    affects-8.1This bug affects the 8.1.x(LTS) versions.component/ddlThis issue is related to DDL of TiDB.severity/moderatetype/enhancementThe issue or PR belongs to an enhancement.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions