Skip to content

external engine oom on loadIngestData #47643

@D3Hunter

Description

@D3Hunter

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

import 32g data using global sort on a 4c8g node

backend worker concurrency is 32, range data loader concurrency is min(int(MergeSortOverlapThreshold) / len(e.dataFiles), 8),write&ingest worker concurrency is fixed as 32, external engine might load too many data into memory

image

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiDB version? (Required)

master

Metadata

Metadata

Assignees

No one assigned

    Labels

    affects-7.5This bug affects the 7.5.x(LTS) versions.component/ddlThis issue is related to DDL of TiDB.feature/developingthe related feature is in developmentseverity/majortype/bugThe issue is confirmed as a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions