-
Notifications
You must be signed in to change notification settings - Fork 6k
Labels
affects-8.5This bug affects the 8.5.x(LTS) versions.This bug affects the 8.5.x(LTS) versions.component/brThis issue is related to BR of TiDB.This issue is related to BR of TiDB.type/feature-requestCategorizes issue or PR as related to a new feature.Categorizes issue or PR as related to a new feature.
Description
Feature Request
Right now only snapshot backup/restore supports filter, users are asking for PiTR to support it. We only need to add filter for log restore and not log backup.
Need to consider table rename corner cases
- table renamed into the filter during log replay should be kept after restore
- table renamed out of filter range during log replay should be dropped after restore
- Should naturally works with insertGCRow, repairIndex and TiFlash
- Partitions exchanged across the filter boundary should not be supported and will error out
This issue now becomes a covering issue for multiple other functionalities including:
- Online filtered restore: need to refresh schema change by tables and not full load otherwise can freeze DDL op for the entire cluster.
- Parallel restore: to allow multiple restore tasks with disjoint filters to proceed at the same time
- Log backup compatibility: allow log backup turned on while doing PiTR
- Better scheduler control: only set on tables that are going to restore instead of all.
Tasks:
- br: separate nested logic of table rewrite and building id map #58112
- brpb: add filtered_out flag for pitr map kvproto#1294
- br: add table filter for log restore #57394
- br: error out if pitr with table filter has exchange partition in/out filter range #59683
- br: add log restore ts into snapshot checkpoint metadata #59678
- br: add tests to cover sequence id, auto increment id and auto random id, revert meta kv capture logic #59109
- br: use region label rule to remove schedulers in db/table restore #59359
- br: PiTR table filter online support #59281 (depends on ddl: Implement TableMode feature #59009 and ddl: support refresh meta ddl #60837)
- ddl support to drop tables in restore mode domain: must load repair table #61451
- br: compatibility of log backup and log restore #61238
- br: add restoreid to idmap table #61278
- br: enable parallel restore #58724
- br: add a checkpoint metadata validation for restore #61108
- br: better control pd scheduler pause during log restore with filter #61819
- br: add tool to clean up registry and checkpoints #61977
- br: prune failed tasks for restore tidb-operator#6288
Bugfixes
- br: fix pitr meta kv fail to parse in during table filter code refactor #60023
- br: fix pitr system table inconsistency issue #60202
- br: fix table filter nil pointer issue #61225
- br: sort meta kv by ts when updating table history #61366
- br: add ddl test for pitr online filter #61631
- br: skip incomplete metakv #61734
- br: resume task that only differs in RestoredTS #61610
- br: support update gc ratio threshold for concurrent log restore #61941
- set
gc.ratio-threshold
to-1
when performing pitr restore tidb-operator#6267 - br: allow append migration to work with restore #62433
Metadata
Metadata
Assignees
Labels
affects-8.5This bug affects the 8.5.x(LTS) versions.This bug affects the 8.5.x(LTS) versions.component/brThis issue is related to BR of TiDB.This issue is related to BR of TiDB.type/feature-requestCategorizes issue or PR as related to a new feature.Categorizes issue or PR as related to a new feature.