statistics: handle deleted tables correctly in the PQ #57649

0xPoe · 2024-11-23T08:38:16Z

What problem does this PR solve?

Issue Number: close #57648

Problem Summary:

What changed and how does it work?

Correctly skip the deleted table. Sorry for overlooking this.
Make sure we don't write to nil runningJobs and mustRetryJobs because the worker might access them again after the queue is closed (we don't stop the worker at the same time because we don't want to block the etcd election process). Accessing it is ok because we will reinitialize the queue next time. So we will take care of the failed jobs next time.

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No need to test
- I checked and no code files have been changed.

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

Signed-off-by: Rustin170506 <[email protected]>

0xPoe

🔢 Self-check (PR reviewed by myself and ready for feedback.)

0xPoe · 2024-11-23T08:52:08Z

pkg/statistics/handle/autoanalyze/priorityqueue/analysis_job_factory.go

@@ -57,7 +57,7 @@ func (f *AnalysisJobFactory) CreateNonPartitionedTableAnalysisJob(
 	tblInfo *model.TableInfo,
 	tblStats *statistics.Table,
 ) AnalysisJob {
-	if !tblStats.IsEligibleForAnalysis() {
+	if tblStats == nil || !tblStats.IsEligibleForAnalysis() {


We didn't encounter any issues here, just double-checking.

0xPoe · 2024-11-23T08:52:13Z

pkg/statistics/handle/autoanalyze/priorityqueue/analysis_job_factory.go

@@ -92,7 +92,7 @@ func (f *AnalysisJobFactory) CreateStaticPartitionAnalysisJob(
 	partitionID int64,
 	partitionStats *statistics.Table,
 ) AnalysisJob {
-	if !partitionStats.IsEligibleForAnalysis() {
+	if partitionStats == nil || !partitionStats.IsEligibleForAnalysis() {


0xPoe · 2024-11-23T08:52:18Z

pkg/statistics/handle/autoanalyze/priorityqueue/analysis_job_factory.go

@@ -128,7 +128,7 @@ func (f *AnalysisJobFactory) CreateDynamicPartitionedTableAnalysisJob(
 	globalTblStats *statistics.Table,
 	partitionStats map[PartitionIDAndName]*statistics.Table,
 ) AnalysisJob {
-	if !globalTblStats.IsEligibleForAnalysis() {
+	if globalTblStats == nil || !globalTblStats.IsEligibleForAnalysis() {


pkg/statistics/handle/autoanalyze/priorityqueue/queue.go

0xPoe · 2024-11-23T08:53:13Z

pkg/statistics/handle/autoanalyze/priorityqueue/queue.go

 				err := pq.syncFields.inner.delete(job)
 				if err != nil {
 					statslogutil.StatsLogger().Error("Failed to delete job from priority queue",
 						zap.Error(err),
 						zap.String("job", job.String()),
 					)
 				}
+				continue


Fix the [category=stats] [recover="runtime error: invalid memory address or nil pointer dereference"]

0xPoe · 2024-11-23T08:53:24Z

pkg/statistics/handle/autoanalyze/priorityqueue/queue_test.go

@@ -608,3 +609,47 @@ func TestPQCanBeClosedAndReInitialized(t *testing.T) {
 	// Check if the priority queue is initialized.
 	require.True(t, pq.IsInitialized())
 }
+
+func TestPQHandlesTableDeletionGracefully(t *testing.T) {


Test case for: [category=stats] [recover="runtime error: invalid memory address or nil pointer dereference"]

Signed-off-by: Rustin170506 <[email protected]>

codecov · 2024-11-23T09:07:49Z

Codecov Report

Attention: Patch coverage is 66.66667% with 7 lines in your changes missing coverage. Please review.

Project coverage is 73.7750%. Comparing base (6e22b8c) to head (fda8e18).
Report is 6 commits behind head on master.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #57649        +/-   ##
================================================
+ Coverage   72.7907%   73.7750%   +0.9843%     
================================================
  Files          1676       1707        +31     
  Lines        463750     472989      +9239     
================================================
+ Hits         337567     348948     +11381     
+ Misses       105323     102632      -2691     
- Partials      20860      21409       +549

Flag	Coverage Δ
integration	`46.2980% <9.5238%> (?)`
unit	`72.3658% <66.6666%> (+0.1772%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`52.7673% <ø> (ø)`
parser	`∅ <ø> (∅)`
br	`45.6200% <ø> (+0.1882%)`	⬆️

---- 🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests
JS Bundle Analysis - Avoid shipping oversized bundles

0xPoe

🔢 Self-check (PR reviewed by myself and ready for feedback.)

0xPoe · 2024-11-23T09:08:10Z

pkg/statistics/handle/autoanalyze/priorityqueue/queue.go

+		// We allow running jobs to complete normally rather than stopping them, so this nil check
+		// is expected when jobs finish after the switch. Failed jobs will be handled by the next
+		// initialization, so we can safely ignore them here.
+		if pq.syncFields.runningJobs == nil || pq.syncFields.mustRetryJobs == nil {


For the ["Auto analyze job execution failed"] [category=stats] [recover="assignment to entry in nil map"].

0xPoe · 2024-11-23T09:21:03Z

/retest

elsa0520

LGTM

ti-chi-bot · 2024-11-25T07:28:39Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: elsa0520, qw4990

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~pkg/statistics/OWNERS~~ [elsa0520,qw4990]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2024-11-25T07:28:42Z

[LGTM Timeline notifier]

Timeline:

2024-11-25 06:12:03.012541066 +0000 UTC m=+444110.632195595: ☑️ agreed by qw4990.
2024-11-25 07:28:41.441354769 +0000 UTC m=+448709.061009284: ☑️ agreed by elsa0520.

0xPoe · 2024-11-25T08:14:17Z

/retest

ti-chi-bot · 2024-11-25T08:36:26Z

In response to a cherrypick label: new pull request created to branch release-8.5: #57674.

close #57648

* skip TestIndexJoin31494 * fix * global sort: add boundaries to split keys when generating plan (pingcap#58323) (pingcap#58356) * statistics: get right max table id when to init stats (pingcap#58280) (pingcap#58298) * executor: Fix the parse problematic slow log panic issue due to empty … * statstics: trigger evict by the timer (pingcap#58027) (pingcap#58268) * br: make table existence check unified on different br client (pingcap#58211) (pingcap#58262) * log backup: use global checkpoint ts as source of truth (pingcap#58135) (pingcap#58265) * executor: skip execution when build query for VIEW in I_S (pingcap#58203) (pingcap#58236) * statistics: copy stats when to update it for avoiding data race (pingcap#5810… * domain,infoschema: make infoschema activity block GC safepoint advanci… * planner: handle panic when loading bindings at startup (pingcap#58017) (pingcap#58035) * statistics: right deal with error for reading stats from storage (pingcap#58… * statistics: lite init used wrong value to build table stats ver (pingcap#5802… * lightning, ddl: set TS to engineMeta after ResetEngineSkipAllocTS (pingcap#5… * *: avoid unlock of unlocked mutex panic on TableDeltaMap (pingcap#57799) (pingcap#57997) * ddl: handle context done after sending DDL jobs (pingcap#57945) (pingcap#57989) * *: activate txn for query on infoschema tables (pingcap#57937) (pingcap#57951) * lightning: add PK to internal tables (pingcap#57480) (pingcap#57932) * statistics: correct behavior of non-lite InitStats and stats sync load… * statistics: avoid stats meta full load after table analysis (pingcap#57756) (pingcap#57911) * dumpling: use I_S to get table list for TiDB and add database to WHERE… * br: fix insert gc failed due to slow schema reload (pingcap#57742) (pingcap#57907) * statistics: do not record historical stats meta if the table is locked… * metrics: remove the filled colors (pingcap#57838) (pingcap#57866) * planner: use TableInfo.DBID to locate schema (pingcap#57785) (pingcap#57870) * *: support cancel query like 'select * from information_schema.tables'… * session: make `TxnInfo()` return even if process info is empty (pingcap#57044) (pingcap#57161) * ddl: Fixed partitioning a non-partitioned table with placement rules (… * *: Reorg partition fix delete ranges and handling non-clustered tables… * executor: fix query infoschema.tables table_schema/table_name with fil… * ddl: check context done in isReorgRunnable function (pingcap#57813) (pingcap#57820) * ddl: fix ExistsTableRow and add tests for skip reorg checks (pingcap#57778) (pingcap#57801) * *: Fix for TRUNCATE PARTITION and Global Index (pingcap#57724) * br: prompt k8s.io/api version (pingcap#57791) (pingcap#57802) * statistics: fix some problem related to stats async load (pingcap#57723) (pingcap#57775) * expression: fix wrong calculation order of `radians` (pingcap#57672) (pingcap#57688) * statistics: rightly deal with timout when to send sync load (pingcap#57712) (pingcap#57751) * ddl: `tidb_scatter_region` variable supports setting value in both upp… * planner: fix that vector index output empty result when pk is non-int … * ddl: dynamically adjusting the max write speed of reorganization job (… * executor: fix hang in hash agg when exceeding memory limit leads to pa… * statistics: use infoschema api to get table info (pingcap#57574) (pingcap#57614) * planner: Use realtimeRowCount when all topN collected (pingcap#56848) (pingcap#57689) * statistics: handle deleted tables correctly in the PQ (pingcap#57649) (pingcap#57674) * backup: reset timeout on store level (pingcap#55526) (pingcap#57667) * planner/core: fix a wrong privilege check for CTE & UPDATE statement (…

statistics: handle deleted tables correctly in the PQ

009fbf2

Signed-off-by: Rustin170506 <[email protected]>

ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. sig/planner SIG: Planner size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Nov 23, 2024

0xPoe added 2 commits November 23, 2024 16:46

fix: do not block etcd

1a7bc2f

Signed-off-by: Rustin170506 <[email protected]>

refactor: better code

51eef0d

Signed-off-by: Rustin170506 <[email protected]>

0xPoe commented Nov 23, 2024

View reviewed changes

0xPoe added the needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. label Nov 23, 2024

refactor: better way to fix the issue

fda8e18

Signed-off-by: Rustin170506 <[email protected]>

0xPoe commented Nov 23, 2024

View reviewed changes

0xPoe requested review from qw4990 and elsa0520 November 25, 2024 02:13

qw4990 approved these changes Nov 25, 2024

View reviewed changes

ti-chi-bot bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Nov 25, 2024

elsa0520 approved these changes Nov 25, 2024

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Nov 25, 2024

ti-chi-bot bot merged commit 702c4f2 into pingcap:master Nov 25, 2024
25 checks passed

ti-chi-bot mentioned this pull request Nov 25, 2024

statistics: handle deleted tables correctly in the PQ (#57649) #57674

Merged

13 tasks

ti-chi-bot bot pushed a commit that referenced this pull request Nov 25, 2024

statistics: handle deleted tables correctly in the PQ (#57649) (#57674)

ed39273

close #57648

statistics: handle deleted tables correctly in the PQ #57649

statistics: handle deleted tables correctly in the PQ #57649

Uh oh!

Conversation

0xPoe commented Nov 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

What changed and how does it work?

Check List

Release note

Uh oh!

0xPoe left a comment

Choose a reason for hiding this comment

Uh oh!

0xPoe Nov 23, 2024

Choose a reason for hiding this comment

Uh oh!

0xPoe Nov 23, 2024

Choose a reason for hiding this comment

Uh oh!

0xPoe Nov 23, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

0xPoe Nov 23, 2024

Choose a reason for hiding this comment

Uh oh!

0xPoe Nov 23, 2024

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Nov 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

0xPoe left a comment

Choose a reason for hiding this comment

Uh oh!

0xPoe Nov 23, 2024

Choose a reason for hiding this comment

Uh oh!

0xPoe commented Nov 23, 2024

Uh oh!

elsa0520 left a comment

Choose a reason for hiding this comment

Uh oh!

ti-chi-bot bot commented Nov 25, 2024

Uh oh!

ti-chi-bot bot commented Nov 25, 2024

[LGTM Timeline notifier]

Uh oh!

0xPoe commented Nov 25, 2024

Uh oh!

Uh oh!

ti-chi-bot commented Nov 25, 2024

Uh oh!

Uh oh!

0xPoe commented Nov 23, 2024 •

edited

Loading

codecov bot commented Nov 23, 2024 •

edited

Loading