Skip to content

Conversation

0xPoe
Copy link
Member

@0xPoe 0xPoe commented Apr 13, 2025

What problem does this PR solve?

Issue Number: close #60567

Problem Summary:

What changed and how does it work?

Based on the flaky test results, sometimes the column status is missing, and other times it’s uninitialized.

This status is generated from here:

if recordForTbl.ColAndIdxStatus != nil && recordForTbl.ColAndIdxStatus.(*statistics.ColAndIdxExistenceMap).HasAnalyzed(id, isIndex) {

The root cause is that sometimes the DDL notifier handles the CREATE TABLE DDL event, which triggers the initialization of statistics histograms for all columns, resulting in the column status being uninitialized. However, if the DDL notifier hasn’t processed the event yet, the column status remains missing.

Since DDL events are handled asynchronously and this is an integration test, there’s no reliable way to stabilize the test by forcing the DDL events to be processed in time.

In this PR, I use the ANALYZE command to explicitly create histogram metadata for all columns, ensuring consistent behavior for the test.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 13, 2025
Copy link

codecov bot commented Apr 13, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.7607%. Comparing base (df15b5d) to head (a9a8a1b).
Report is 30 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #60514        +/-   ##
================================================
+ Coverage   73.1384%   73.7607%   +0.6222%     
================================================
  Files          1714       1714                
  Lines        474343     481762      +7419     
================================================
+ Hits         346927     355351      +8424     
+ Misses       106077     105077      -1000     
+ Partials      21339      21334         -5     
Flag Coverage Δ
integration 42.7927% <ø> (?)
unit 72.4907% <ø> (+0.1224%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 52.6553% <ø> (ø)
parser ∅ <ø> (∅)
br 47.0862% <ø> (-0.1523%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ti-chi-bot ti-chi-bot bot added the sig/planner SIG: Planner label Apr 14, 2025
@0xPoe
Copy link
Member Author

0xPoe commented Apr 14, 2025

/retest

@ti-chi-bot ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 14, 2025
@ti-chi-bot ti-chi-bot bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 14, 2025
@0xPoe
Copy link
Member Author

0xPoe commented Apr 14, 2025

/test all

1 similar comment
@0xPoe
Copy link
Member Author

0xPoe commented Apr 14, 2025

/test all

@0xPoe
Copy link
Member Author

0xPoe commented Apr 15, 2025

/test all

Signed-off-by: Rustin170506 <[email protected]>
@0xPoe 0xPoe changed the title test: add sleep statements to integration tests for timing control test: analyze the tables to stabilize the flaky status of the columns Apr 15, 2025
@0xPoe 0xPoe requested a review from Copilot April 15, 2025 05:35
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.

Files not reviewed (2)
  • tests/integrationtest/r/explain_complex_stats.result: Language not supported
  • tests/integrationtest/t/explain_complex_stats.test: Language not supported

@0xPoe
Copy link
Member Author

0xPoe commented Apr 15, 2025

/test all

@ti-chi-bot ti-chi-bot bot added the approved label Apr 15, 2025
@ti-chi-bot ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Apr 15, 2025
@0xPoe 0xPoe moved this to ⏳ Await responses in 🎒My Work Apr 15, 2025
@0xPoe
Copy link
Member Author

0xPoe commented Apr 15, 2025

/retest

@0xPoe 0xPoe added the needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. label Apr 15, 2025
Copy link

ti-chi-bot bot commented Apr 15, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: time-and-fate, winoros

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [time-and-fate,winoros]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Apr 15, 2025
Copy link

ti-chi-bot bot commented Apr 15, 2025

[LGTM Timeline notifier]

Timeline:

  • 2025-04-15 06:06:18.957369666 +0000 UTC m=+2754872.641605757: ☑️ agreed by time-and-fate.
  • 2025-04-15 09:44:59.711697285 +0000 UTC m=+2767993.395933376: ☑️ agreed by winoros.

@ti-chi-bot ti-chi-bot bot merged commit 3146532 into pingcap:master Apr 15, 2025
24 checks passed
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-8.5: #60584.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. release-note-none Denotes a PR that doesn't merit a release note. sig/planner SIG: Planner size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

flaky test: explain_complex_stats
4 participants