expression: fix the length of casting from INT/REAL/DECIMAL/.... to string | tidb-test=pr/2554

[ti-chi-bot]

This is an automated cherry-pick of #61476

What problem does this PR solve?

Issue Number: close #61350

Problem Summary:

The length returned by TiDB is not compatible with MySQL. Sometimes it's too small so that the client may fail to allocate big enough buffer and cause panic/security issue. This PR modifies the logic of calculating the length, and make it compatible with MySQL for most of the cases.

What changed and how does it work?

1. Change the logic of calculating length for casting many types to string.

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No need to test

  • I checked and no code files have been changed.

Run the SQLs provided in #61350 and check the results are the same.

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

Fix the issue that the length returned by `cast` function is not compatible with MySQL.

[ti-chi-bot]

@YangKeao This PR has conflicts, I have hold it.
Please resolve them or ask others to resolve them, then comment /unhold to remove the hold label.

[codecov]

Codecov Report

Attention: Patch coverage is 89.47368% with 18 lines in your changes missing coverage. Please review.

Please upload report for BASE (release-8.5@f1df099). Learn more about missing BASE report.

Additional details and impacted files

@@               Coverage Diff                @@
##             release-8.5     #62330   +/-   ##
================================================
  Coverage               ?   57.1980%           
================================================
  Files                  ?       1777           
  Lines                  ?     631495           
  Branches               ?          0           
================================================
  Hits                   ?     361203           
  Misses                 ?     246179           
  Partials               ?      24113           

Flag	Coverage Δ
integration	37.2054% <69.0058%> (?)
unit	72.8490% <88.3040%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	52.9278% <0.0000%> (?)
parser	∅ <0.0000%> (?)
br	52.7291% <0.0000%> (?)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[YangKeao]

/unhold

[windtalker]

lgtm

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2025-07-10 03:21:26.651549495 +0000 UTC m=+2142739.374728475: ☑️ agreed by AilinKid.
• 2025-07-10 03:30:41.224874549 +0000 UTC m=+2143293.948053530: ☑️ agreed by windtalker.

[YangKeao]

/retest

[YangKeao]

/retest

[YangKeao]

/hold

[YangKeao]

There is a bug reported by TiFlash test:

TiDB [email protected]:test> create table test.t(a datetime, b date, c timestamp(3))
TiDB [email protected]:test> alter table test.t set tiflash replica 1
TiDB [email protected]:test> insert into test.t values('2021/04/13 00:34:00.123456', '2021/04/13', '2021/04/13 00:34:00.123456')
-- tidb-server throw error, 3c2dc46853f5a2edb89da78c1a678da9725ed7b8
TiDB [email protected]:test> select count(*), cast(a as char), cast(b as char), cast(c as char) from test.t group by cast(a as char), cast(b as char), cast(c as char);
(1055, "Expression #2 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'test.t.a' which is not functionally dependent on columns in GROUP BY clause; this
is incompatible with sql_mode=only_full_group_by")
TiDB [email protected]:test> select tidb_version() \G;
***************************[ 1. row ]***************************
tidb_version() | Release Version: v9.0.0-beta.2.pre-84-g3c2dc46853
Edition: Community
Git Commit Hash: 3c2dc46853f5a2edb89da78c1a678da9725ed7b8
Git Branch: master
UTC Build Time: 2025-07-10 07:12:11
GoVersion: go1.23.10
Race Enabled: false
Check Table Before Drop: false
Store: tikv
Kernel Type: Classic
-- tidb-server is ok, fe8d82fedd8f9a638d1e0d88472a320dd4f2b392 is OK
TiDB [email protected]:test> select count(*), cast(a as char), cast(b as char), cast(c as char) from test.t group by cast(a as char), cast(b as char), cast(c as char);
+----------+---------------------+-----------------+-------------------------+
| count(*) | cast(a as char)     | cast(b as char) | cast(c as char)         |
+----------+---------------------+-----------------+-------------------------+
| 1        | 2021-04-13 00:34:00 | 2021-04-13      | 2021-04-13 00:34:00.123 |
+----------+---------------------+-----------------+-------------------------+
TiDB [email protected]:test> select tidb_version() \G;
***************************[ 1. row ]***************************
tidb_version() | Release Version: v9.0.0-beta.2.pre-83-gfe8d82fedd
Edition: Community
Git Commit Hash: fe8d82fedd8f9a638d1e0d88472a320dd4f2b392
Git Branch: master
UTC Build Time: 2025-07-10 07:16:14
GoVersion: go1.23.10
Race Enabled: false
Check Table Before Drop: false
Store: tikv
Kernel Type: Classic
1 row in set
Time: 0.001s

[YangKeao]

There is a bug reported by TiFlash test:

TiDB [email protected]:test> create table test.t(a datetime, b date, c timestamp(3))
TiDB [email protected]:test> alter table test.t set tiflash replica 1
TiDB [email protected]:test> insert into test.t values('2021/04/13 00:34:00.123456', '2021/04/13', '2021/04/13 00:34:00.123456')
-- tidb-server throw error, 3c2dc46853f5a2edb89da78c1a678da9725ed7b8
TiDB [email protected]:test> select count(*), cast(a as char), cast(b as char), cast(c as char) from test.t group by cast(a as char), cast(b as char), cast(c as char);
(1055, "Expression #2 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'test.t.a' which is not functionally dependent on columns in GROUP BY clause; this
is incompatible with sql_mode=only_full_group_by")
TiDB [email protected]:test> select tidb_version() \G;
***************************[ 1. row ]***************************
tidb_version() | Release Version: v9.0.0-beta.2.pre-84-g3c2dc46853
Edition: Community
Git Commit Hash: 3c2dc46853f5a2edb89da78c1a678da9725ed7b8
Git Branch: master
UTC Build Time: 2025-07-10 07:12:11
GoVersion: go1.23.10
Race Enabled: false
Check Table Before Drop: false
Store: tikv
Kernel Type: Classic
-- tidb-server is ok, fe8d82fedd8f9a638d1e0d88472a320dd4f2b392 is OK
TiDB [email protected]:test> select count(*), cast(a as char), cast(b as char), cast(c as char) from test.t group by cast(a as char), cast(b as char), cast(c as char);
+----------+---------------------+-----------------+-------------------------+
| count(*) | cast(a as char)     | cast(b as char) | cast(c as char)         |
+----------+---------------------+-----------------+-------------------------+
| 1        | 2021-04-13 00:34:00 | 2021-04-13      | 2021-04-13 00:34:00.123 |
+----------+---------------------+-----------------+-------------------------+
TiDB [email protected]:test> select tidb_version() \G;
***************************[ 1. row ]***************************
tidb_version() | Release Version: v9.0.0-beta.2.pre-83-gfe8d82fedd
Edition: Community
Git Commit Hash: fe8d82fedd8f9a638d1e0d88472a320dd4f2b392
Git Branch: master
UTC Build Time: 2025-07-10 07:16:14
GoVersion: go1.23.10
Race Enabled: false
Check Table Before Drop: false
Store: tikv
Kernel Type: Classic
1 row in set
Time: 0.001s

It's not related to TiFlash.

[YangKeao]

Ref #62354

[YangKeao]

I'll also pick #62354 to this PR.

[YangKeao]

/retest

[YangKeao]

#62354 has been merged.

/unhold

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: AilinKid, tangenta, windtalker

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [AilinKid,tangenta,windtalker]
• pkg/expression/OWNERS [windtalker]
• pkg/planner/OWNERS [AilinKid]
• pkg/table/OWNERS [tangenta]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[YangKeao]

/retest