-
Notifications
You must be signed in to change notification settings - Fork 26
Open
Labels
type/enhancementThe issue or PR belongs to an enhancement.The issue or PR belongs to an enhancement.
Description
Core Requirements
Optimize the puller task scheduling mechanism to prevent changefeeds (CFs) with a large volume of incremental scan tasks from starving (i.e., blocking or delaying) other incremental scan tasks generated by region changes. The optimization must meet two key goals:
- Ensure delay stability for all changefeeds (no significant lag fluctuations across CFs);
- Avoid compromising the efficiency of existing incremental scan operations.
Core Priority Design Principles
Priority rules shall be formulated based on the intrinsic characteristics of tasks, with the following specifications:
- Highest Priority Scenario:
RegionTasks triggered by the need for region reconnection (e.g., due to region migration, node failure, or other region-level changes) shall be assigned the highest priority. - General Priority Rule:
For all other RegionTasks (i.e., those not related to reconnection), priority shall increase with task waiting time. This design aims to approximate a "first-in-first-out (FIFO)" scheduling effect, thereby preventing long-term accumulation of pending tasks.
Metadata
Metadata
Assignees
Labels
type/enhancementThe issue or PR belongs to an enhancement.The issue or PR belongs to an enhancement.