telemetry: add telemetry related code back #61753

Defined2014 · 2025-06-16T09:36:23Z

This reverts commit 3d1708f.

What problem does this PR solve?

Issue Number: close #61766

Problem Summary: Add telemetry related code back, it will print log to files. The code removed PR is #51202

The output log looks like

[2025/06/23 11:06:18.546 +08:00] [INFO] [telemetry.go:67] [category=telemetry] ["telemetry data"="{\"reportTimestamp\":1750647892,\"featureUsage\":{\"txn\":{\"asyncCommitUsed\":false,\"onePCUsed\":false,\"txnCommitCounter\":{\"twoPC\":0,\"asyncCommit\":0,\"onePC\":0},\"mutationCheckerUsed\":true,\"assertionLevel\":\"FAST\",\"rcCheckTS\":false,\"rcWriteCheckTS\":false,\"fairLocking\":true,\"SavepointCounter\":0,\"lazyUniqueCheckSetCounter\":0,\"FairLockingUsageCounter\":{\"txn_fair_locking_used\":0,\"txn_fair_locking_effective\":0}},\"newClusterIndex\":{\"numClusteredTables\":0,\"numTotalTables\":2},\"temporaryTable\":false,\"cte\":{\"nonRecursiveCTEUsed\":0,\"recursiveUsed\":0,\"nonCTEUsed\":15},\"accountLock\":{\"lockUser\":0,\"unlockUser\":0,\"createOrAlterUser\":0},\"cachedTable\":false,\"autoCapture\":false,\"placementPolicy\":{\"numPlacementPolicies\":0,\"numDBWithPolicies\":0,\"numTableWithPolicies\":0,\"numPartitionWithExplicitPolicies\":0},\"nonTransactional\":{\"delete\":0,\"update\":0,\"insert\":0},\"globalKill\":true,\"multiSchemaChange\":{\"multi_schema_change_used\":0},\"exchangePartition\":{\"exchange_partition_cnt\":0},\"tablePartition\":{\"table_partition_cnt\":0,\"table_partition_list_cnt\":0,\"table_partition_range_cnt\":0,\"table_partition_hash_cnt\":0,\"table_partition_range_columns_cnt\":0,\"table_partition_range_columns_gt_1_cnt\":0,\"table_partition_range_columns_gt_2_cnt\":0,\"table_partition_range_columns_gt_3_cnt\":0,\"table_partition_list_columns_cnt\":0,\"table_partition_max_partitions_cnt\":0,\"table_partition_create_interval_partitions_cnt\":0,\"table_partition_add_interval_partitions_cnt\":0,\"table_partition_drop_interval_partitions_cnt\":0,\"table_TablePartitionComactCnt\":0,\"table_reorganize_partition_cnt\":0},\"logBackup\":false,\"enablePaging\":false,\"enableCostModelVer2\":true,\"DDLUsageCounter\":{\"add_index_ingest_used\":0,\"metadata_lock_used\":true,\"flashback_cluster_used\":0,\"dist_reorg_used\":0},\"enableGlobalMemoryControl\":true,\"autoIDNoCache\":false,\"indexMergeUsageCounter\":{\"index_merge_used\":0},\"resourceControl\":{\"resourceControlEnabled\":true,\"numResourceGroups\":1},\"ttlUsage\":{\"ttl_job_enabled\":true,\"ttl_table_count\":0,\"ttl_job_enabled_tables\":0,\"ttl_hist_date\":\"2025-06-22\",\"table_hist_with_delete_rows\":[{\"less_than\":10000,\"count\":0},{\"less_than\":100000,\"count\":0},{\"less_than\":1000000,\"count\":0},{\"less_than\":10000000,\"count\":0},{\"less_than_max\":true,\"count\":0}],\"table_hist_with_delay_time\":[{\"less_than\":1,\"count\":0},{\"less_than\":6,\"count\":0},{\"less_than\":24,\"count\":0},{\"less_than\":72,\"count\":0},{\"less_than_max\":true,\"count\":0}]},\"storeBatchCopr\":{\"batch_size\":4,\"query\":0,\"tasks\":0,\"batched\":0,\"batched_fallback\":0}},\"windowedStats\":[{\"beginAt\":\"2025-06-23T11:03:22.935124584+08:00\",\"executeCount\":7,\"tiFlashUsage\":{\"pushDown\":0,\"exchangePushDown\":0,\"tableScan\":0,\"tableScanWithFastScan\":0},\"coprCacheUsage\":{\"gte0\":4,\"gte1\":0,\"gte10\":0,\"gte20\":0,\"gte40\":0,\"gte80\":0,\"gte100\":0},\"builtinFunctionsUsage\":{\"EQInt\":2,\"PlusInt\":3}},{\"beginAt\":\"2025-06-23T11:04:22.93512986+08:00\",\"executeCount\":0,\"tiFlashUsage\":{\"pushDown\":0,\"exchangePushDown\":0,\"tableScan\":0,\"tableScanWithFastScan\":0},\"coprCacheUsage\":{\"gte0\":0,\"gte1\":0,\"gte10\":0,\"gte20\":0,\"gte40\":0,\"gte80\":0,\"gte100\":0},\"builtinFunctionsUsage\":{}}]}"]

What changed and how does it work?

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No need to test
- I checked and no code files have been changed.

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Affects user behaviors
Contains syntax changes
Contains variable changes
telemetry: update telemetry related doc files docs-cn#20556
Contains experimental features
Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

This reverts commit 3d1708f.

tiprow · 2025-06-16T09:36:42Z

Hi @Defined2014. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

codecov · 2025-06-16T10:09:13Z

Codecov Report

Attention: Patch coverage is 69.66551% with 263 lines in your changes missing coverage. Please review.

Project coverage is 74.8907%. Comparing base (9f1396b) to head (0186d6b).
Report is 1 commits behind head on master.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #61753        +/-   ##
================================================
+ Coverage   72.9375%   74.8907%   +1.9532%     
================================================
  Files          1742       1794        +52     
  Lines        483190     492182      +8992     
================================================
+ Hits         352427     368599     +16172     
+ Misses       109165     100473      -8692     
- Partials      21598      23110      +1512

Flag	Coverage Δ
integration	`48.7920% <29.2964%> (?)`
unit	`72.1847% <69.6655%> (-0.0110%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`52.7804% <ø> (ø)`
parser	`∅ <ø> (∅)`
br	`63.0718% <ø> (+16.7318%)`	⬆️

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Defined2014 · 2025-06-16T12:39:14Z

/retest

tiprow · 2025-06-16T12:39:37Z

@Defined2014: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

pkg/domain/domain.go

Defined2014 · 2025-07-03T09:48:27Z

/retest

Defined2014 · 2025-07-03T11:02:01Z

/retest

Defined2014 · 2025-07-03T11:31:36Z

/retest

Defined2014 · 2025-07-03T12:45:26Z

/retest

Defined2014 · 2025-07-03T13:30:45Z

/retest

Defined2014 · 2025-07-03T13:52:00Z

/retest

Defined2014 · 2025-07-03T14:15:41Z

/retest

Defined2014 · 2025-07-03T14:45:35Z

/retest

hawkingrei · 2025-07-03T16:38:42Z

/retest

Defined2014 · 2025-07-04T01:53:05Z

/retest

Defined2014 · 2025-07-04T02:56:09Z

/retest

Defined2014 · 2025-07-04T04:24:54Z

/retest

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot · 2025-07-04T05:20:43Z

In response to a cherrypick label: new pull request created to branch release-8.1: #62208.
But this PR has conflicts, please resolve them!

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot · 2025-07-04T05:21:27Z

In response to a cherrypick label: new pull request created to branch release-8.5: #62209.
But this PR has conflicts, please resolve them!

Signed-off-by: ti-chi-bot <[email protected]>

* origin/master: (79 commits) backend/local: add rate limiter for split region and ingest data (pingcap#61555) fix(runaway): resolve the dead channel in UpdateNewAndDoneWatch (pingcap#61795) *: collect subtask summary for `IMPORT INTO` tasks (pingcap#61600) tests: update vector cluster test result (pingcap#62174) planner: change casetest instance-plan-cache test timeout. (pingcap#62195) planner: remove data race for instance plan cache (pingcap#62217) *: upgrade pd client to fix keyspace issue (pingcap#62189) resource_control: support collecting cross AZ traffic in ru consumption (pingcap#62003) test: wait for statistics owner to close on exit (pingcap#62198) planner: remove the selection above the table dual (pingcap#62193) telemetry: add telemetry related code back (pingcap#61753) planner: move physical limit related logic into physical op. (pingcap#62187) planner: remove useless code (pingcap#61968) retry: enhance retry for ingestor on nextgen (pingcap#62191) importinto: encode and ingest step use task keyspace in nextgen (pingcap#62138) planner: lift the can-push-to-cop check of aggregation/proj (pingcap#62068) ddl: cannot panic with failpoint panic in the DDLNotifier (pingcap#62151) planner: skip empty expression in the applyPredicateSimplification (pingcap#62178) codec: should also return the value even with error because the error may be ignored in the future (pingcap#62175) parser: also reduce literal list with charset (pingcap#61484) ...

close #61766

Revert "telemetry: remove the codes of Telemetry (pingcap#51202)"

4335455

This reverts commit 3d1708f.

Defined2014 added the skip-issue-check Indicates that a PR no need to check linked issue. label Jun 16, 2025

ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. sig/planner SIG: Planner size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jun 16, 2025

Defined2014 added 2 commits June 16, 2025 17:48

fix hint

d0bf7ff

fix panic

c87159f

Defined2014 added 2 commits June 16, 2025 19:26

fix test

20993ca

fix test

6a052e0

Defined2014 removed the skip-issue-check Indicates that a PR no need to check linked issue. label Jun 17, 2025

Defined2014 added 6 commits June 17, 2025 16:08

update

517bef6

update

f71a48c

remove ClusterIndexUsage in telemetry

d9f13b9

fix test

e9e68d3

Merge branch 'master' into telemetry

59d7565

update

3dcb4be

hawkingrei approved these changes Jun 23, 2025

View reviewed changes

ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Jun 23, 2025

Defined2014 requested review from bb7133, XuHuaiyu and tangenta June 23, 2025 07:17

Defined2014 mentioned this pull request Jun 25, 2025

telemetry: update telemetry related doc files pingcap/docs-cn#20556

Merged

15 tasks

tangenta reviewed Jun 25, 2025

View reviewed changes

pkg/domain/domain.go Show resolved Hide resolved

tangenta approved these changes Jun 25, 2025

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jun 25, 2025

Merge branch 'master' into telemetry

fd21876

Merge branch 'master' into telemetry

0186d6b

ti-chi-bot bot merged commit 15fe8f1 into pingcap:master Jul 4, 2025
27 checks passed

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Jul 4, 2025

This is an automated cherry-pick of pingcap#61753

8d035bd

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot mentioned this pull request Jul 4, 2025

telemetry: add telemetry related code back (#61753) #62208

Closed

13 tasks

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Jul 4, 2025

This is an automated cherry-pick of pingcap#61753

19e6c22

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot mentioned this pull request Jul 4, 2025

telemetry: add telemetry related code back (#61753) #62209

Merged

13 tasks

Defined2014 added a commit to ti-chi-bot/tidb that referenced this pull request Jul 4, 2025

This is an automated cherry-pick of pingcap#61753

27441a3

Signed-off-by: ti-chi-bot <[email protected]>

Defined2014 added a commit to ti-chi-bot/tidb that referenced this pull request Jul 4, 2025

This is an automated cherry-pick of pingcap#61753

49e53b3

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot bot pushed a commit that referenced this pull request Jul 9, 2025

telemetry: add telemetry related code back (#61753) (#62209)

4caea03

close #61766

Oreoxmt mentioned this pull request Jul 15, 2025

v8.5.3: update telemetry related doc files pingcap/docs#21404

Merged

15 tasks

This was referenced Aug 14, 2025

telemetry: update telemetry related doc files (#20556) pingcap/docs-cn#20780

Merged

v8.5.3: update telemetry related doc files (#21404) pingcap/docs#21603

Merged

telemetry: add telemetry related code back #61753

telemetry: add telemetry related code back #61753

Uh oh!

Conversation

Defined2014 commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

What changed and how does it work?

Check List

Release note

Uh oh!

tiprow bot commented Jun 16, 2025

Uh oh!

codecov bot commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Defined2014 commented Jun 16, 2025

Uh oh!

tiprow bot commented Jun 16, 2025

Uh oh!

Uh oh!

Defined2014 commented Jul 3, 2025

Uh oh!

Defined2014 commented Jul 3, 2025

Uh oh!

Defined2014 commented Jul 3, 2025

Uh oh!

Defined2014 commented Jul 3, 2025

Uh oh!

Defined2014 commented Jul 3, 2025

Uh oh!

Defined2014 commented Jul 3, 2025

Uh oh!

Defined2014 commented Jul 3, 2025

Uh oh!

Defined2014 commented Jul 3, 2025

Uh oh!

hawkingrei commented Jul 3, 2025

Uh oh!

Defined2014 commented Jul 4, 2025

Uh oh!

Defined2014 commented Jul 4, 2025

Uh oh!

Defined2014 commented Jul 4, 2025

Uh oh!

Uh oh!

ti-chi-bot commented Jul 4, 2025

Uh oh!

ti-chi-bot commented Jul 4, 2025

Uh oh!

Uh oh!

Defined2014 commented Jun 16, 2025 •

edited

Loading

codecov bot commented Jun 16, 2025 •

edited

Loading