*: Fix for TRUNCATE PARTITION and Global Index #57724

mjonss · 2024-11-26T11:22:10Z

What problem does this PR solve?

Issue Number: close #57721

Problem Summary:
There were issues with TRUNCATE PARTITION and Global indexes, no rollback and blocked writes during some stages.

What changed and how does it work?

Cherry pick of #55831 and #56029

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No need to test
- I checked and no code files have been changed.

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

No more partition wide write block during "delete only" and "delete reorganization" for TRUNCATE PARTITION

close pingcap#55819

ref pingcap#55819

tiprow · 2024-11-26T11:22:27Z

Hi @mjonss. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

codecov · 2024-11-26T11:54:11Z

Codecov Report

Attention: Patch coverage is 74.31907% with 66 lines in your changes missing coverage. Please review.

Please upload report for BASE (release-8.5@aca349f). Learn more about missing BASE report.

Additional details and impacted files

@@               Coverage Diff                @@
##             release-8.5     #57724   +/-   ##
================================================
  Coverage               ?   56.8880%           
================================================
  Files                  ?       1770           
  Lines                  ?     627522           
  Branches               ?          0           
================================================
  Hits                   ?     356985           
  Misses                 ?     246426           
  Partials               ?      24111

Flag	Coverage Δ
integration	`37.0007% <40.0778%> (?)`
unit	`72.5540% <74.3190%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`52.9478% <0.0000%> (?)`
parser	`∅ <0.0000%> (?)`
br	`52.3289% <0.0000%> (?)`

mjonss · 2024-11-26T12:39:54Z

/retest

tiprow · 2024-11-26T12:40:20Z

@mjonss: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

purelind · 2024-11-26T13:35:50Z

/test all

tiprow · 2024-11-26T13:36:16Z

@purelind: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test all

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

ti-chi-bot · 2024-11-27T01:57:43Z

[LGTM Timeline notifier]

Timeline:

2024-11-27 01:35:54.128699513 +0000 UTC m=+600341.748354029: ☑️ agreed by Defined2014.
2024-11-27 01:57:42.556727526 +0000 UTC m=+601650.176382040: ☑️ agreed by tangenta.

ti-chi-bot · 2024-11-27T03:29:02Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Defined2014, tangenta, winoros

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [Defined2014,tangenta,winoros]
~~pkg/ddl/OWNERS~~ [tangenta]
~~pkg/meta/OWNERS~~ [tangenta]
~~pkg/planner/OWNERS~~ [winoros]
~~pkg/table/OWNERS~~ [tangenta]
~~pkg/tablecodec/OWNERS~~ [tangenta]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

* skip TestIndexJoin31494 * fix * global sort: add boundaries to split keys when generating plan (pingcap#58323) (pingcap#58356) * statistics: get right max table id when to init stats (pingcap#58280) (pingcap#58298) * executor: Fix the parse problematic slow log panic issue due to empty … * statstics: trigger evict by the timer (pingcap#58027) (pingcap#58268) * br: make table existence check unified on different br client (pingcap#58211) (pingcap#58262) * log backup: use global checkpoint ts as source of truth (pingcap#58135) (pingcap#58265) * executor: skip execution when build query for VIEW in I_S (pingcap#58203) (pingcap#58236) * statistics: copy stats when to update it for avoiding data race (pingcap#5810… * domain,infoschema: make infoschema activity block GC safepoint advanci… * planner: handle panic when loading bindings at startup (pingcap#58017) (pingcap#58035) * statistics: right deal with error for reading stats from storage (pingcap#58… * statistics: lite init used wrong value to build table stats ver (pingcap#5802… * lightning, ddl: set TS to engineMeta after ResetEngineSkipAllocTS (pingcap#5… * *: avoid unlock of unlocked mutex panic on TableDeltaMap (pingcap#57799) (pingcap#57997) * ddl: handle context done after sending DDL jobs (pingcap#57945) (pingcap#57989) * *: activate txn for query on infoschema tables (pingcap#57937) (pingcap#57951) * lightning: add PK to internal tables (pingcap#57480) (pingcap#57932) * statistics: correct behavior of non-lite InitStats and stats sync load… * statistics: avoid stats meta full load after table analysis (pingcap#57756) (pingcap#57911) * dumpling: use I_S to get table list for TiDB and add database to WHERE… * br: fix insert gc failed due to slow schema reload (pingcap#57742) (pingcap#57907) * statistics: do not record historical stats meta if the table is locked… * metrics: remove the filled colors (pingcap#57838) (pingcap#57866) * planner: use TableInfo.DBID to locate schema (pingcap#57785) (pingcap#57870) * *: support cancel query like 'select * from information_schema.tables'… * session: make `TxnInfo()` return even if process info is empty (pingcap#57044) (pingcap#57161) * ddl: Fixed partitioning a non-partitioned table with placement rules (… * *: Reorg partition fix delete ranges and handling non-clustered tables… * executor: fix query infoschema.tables table_schema/table_name with fil… * ddl: check context done in isReorgRunnable function (pingcap#57813) (pingcap#57820) * ddl: fix ExistsTableRow and add tests for skip reorg checks (pingcap#57778) (pingcap#57801) * *: Fix for TRUNCATE PARTITION and Global Index (pingcap#57724) * br: prompt k8s.io/api version (pingcap#57791) (pingcap#57802) * statistics: fix some problem related to stats async load (pingcap#57723) (pingcap#57775) * expression: fix wrong calculation order of `radians` (pingcap#57672) (pingcap#57688) * statistics: rightly deal with timout when to send sync load (pingcap#57712) (pingcap#57751) * ddl: `tidb_scatter_region` variable supports setting value in both upp… * planner: fix that vector index output empty result when pk is non-int … * ddl: dynamically adjusting the max write speed of reorganization job (… * executor: fix hang in hash agg when exceeding memory limit leads to pa… * statistics: use infoschema api to get table info (pingcap#57574) (pingcap#57614) * planner: Use realtimeRowCount when all topN collected (pingcap#56848) (pingcap#57689) * statistics: handle deleted tables correctly in the PQ (pingcap#57649) (pingcap#57674) * backup: reset timeout on store level (pingcap#55526) (pingcap#57667) * planner/core: fix a wrong privilege check for CTE & UPDATE statement (…

mjonss added 2 commits November 26, 2024 12:16

*: Truncate partition with global index improvement (pingcap#55831)

fc1312f

close pingcap#55819

*: Truncate partition failpoint tests (pingcap#56029)

291a2e9

ref pingcap#55819

ti-chi-bot bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/cherry-pick-not-approved sig/planner SIG: Planner size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Nov 26, 2024

mjonss requested a review from Defined2014 November 26, 2024 11:22

mjonss requested review from winoros and tangenta November 26, 2024 23:19

Defined2014 approved these changes Nov 27, 2024

View reviewed changes

ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Nov 27, 2024

tangenta approved these changes Nov 27, 2024

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Nov 27, 2024

winoros approved these changes Nov 27, 2024

View reviewed changes

ti-chi-bot bot added approved cherry-pick-approved Cherry pick PR approved by release team. and removed do-not-merge/cherry-pick-not-approved labels Nov 27, 2024

ti-chi-bot bot merged commit f07d030 into pingcap:release-8.5 Nov 29, 2024
19 checks passed

mjonss deleted the truncate-partition-fix-57721 branch November 29, 2024 00:50

mjonss mentioned this pull request Nov 29, 2024

non-existing rollback of TRUNCATE PARTITION, causing hang due to errors. #57721

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

*: Fix for TRUNCATE PARTITION and Global Index #57724

*: Fix for TRUNCATE PARTITION and Global Index #57724

Uh oh!

mjonss commented Nov 26, 2024 •

edited

Loading

Uh oh!

tiprow bot commented Nov 26, 2024

Uh oh!

codecov bot commented Nov 26, 2024 •

edited

Loading

Uh oh!

mjonss commented Nov 26, 2024

Uh oh!

tiprow bot commented Nov 26, 2024

Uh oh!

purelind commented Nov 26, 2024

Uh oh!

tiprow bot commented Nov 26, 2024

Uh oh!

ti-chi-bot bot commented Nov 27, 2024

Uh oh!

ti-chi-bot bot commented Nov 27, 2024

Uh oh!

Uh oh!

Uh oh!

*: Fix for TRUNCATE PARTITION and Global Index #57724

*: Fix for TRUNCATE PARTITION and Global Index #57724

Uh oh!

Conversation

mjonss commented Nov 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

What changed and how does it work?

Check List

Release note

Uh oh!

tiprow bot commented Nov 26, 2024

Uh oh!

codecov bot commented Nov 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mjonss commented Nov 26, 2024

Uh oh!

tiprow bot commented Nov 26, 2024

Uh oh!

purelind commented Nov 26, 2024

Uh oh!

tiprow bot commented Nov 26, 2024

Uh oh!

ti-chi-bot bot commented Nov 27, 2024

[LGTM Timeline notifier]

Uh oh!

ti-chi-bot bot commented Nov 27, 2024

Uh oh!

Uh oh!

Uh oh!

mjonss commented Nov 26, 2024 •

edited

Loading

codecov bot commented Nov 26, 2024 •

edited

Loading