Skip to content

DXF: support modify task concurrency and business params at runtime #57497

@D3Hunter

Description

@D3Hunter

Feature Request

Is your feature request related to a problem? Please describe:

Currently, after a task is submitted to DXF, we cannot change the max resource it can be used at runtime. But we need this feature in some cases, such as when submitting alter table xxx add index, current value of tidb_ddl_reorg_worker_cnt of which default is 4, is used as the task concurrency, later users might want to accelerate it by increasing the concurrency, which is a supported feature when adding index in standalone mode. And import-into might need it too.

Also we need to modify some business specific param at runtime, such as batch-size and max-write-speed for add-index.

Describe the feature you'd like:

support modify task concurrency at runtime, we want to make it easier to extend as we might need to modify other task config at runtime later. and support modify business specific param

Describe alternatives you've considered:

this is new feature
Teachability, Documentation, Adoption, Migration Strategy:

DXF is the underlying framework behind upper business, including add-index and import-into, so to support change concurrency or business specific param for upper business, DXF need to support it first, then upper layer can integrate the ability and expose some API to user

Metadata

Metadata

Assignees

No one assigned

    Labels

    type/feature-requestCategorizes issue or PR as related to a new feature.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions