can't insert data using gbk charset

[wjhuang2016]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

CREATE TABLE `tl339c67ba` (
  `col_41` datetime NOT NULL DEFAULT '2011-05-21 00:00:00',
  `col_42` char(92) NOT NULL,
  `col_43` float DEFAULT '6891.0747',
  `col_44` char(5) DEFAULT 'b_',
  `col_45` smallint(6) DEFAULT '-7116',
  `col_46` json NOT NULL,
  `col_47` date NOT NULL,
  `col_48` time NOT NULL,
  `col_49` json DEFAULT NULL,
  `col_50` mediumint(8) unsigned NOT NULL,
  KEY `idx_17` ((cast(`col_46` as unsigned array)),`col_44`,`col_45`),
  PRIMARY KEY (`col_50`,`col_42`) /*T![clustered_index] NONCLUSTERED */
) ENGINE=InnoDB DEFAULT CHARSET=gbk COLLATE=gbk_chinese_ci;

INSERT INTO `tl339c67ba` VALUES ('2024-02-13 00:00:00','l',NULL,'P5',14565,'800370530','1998-09-03','06:25:19','[-137365.35472449657, 98740.22614562341, -96230.65667019188, 81088.77765708079, 126813.96345066803, 228411.26303739505, -31360.293560806753, -40673.38888043809, 421555.88300412416, 4097.420882023372, -43759.19093454673, -618381.5308285163, 57324.58237695448, -11254.431120740595, -11831.811494338499, 632228.8397806884, -147234.135661395, -10233.283960475097, -60623.850914205344, 20194.112452002195, 31881.62808860628, 20531.172403172404, -48548.85255468495, 104726.24161292956, -180867.75742741392, 160201.2879594866]',16155661);

2. What did you expect to see? (Required)

No error

3. What did you see instead (Required)

mysql> CREATE TABLE `tl339c67ba` (
    ->   `col_41` datetime NOT NULL DEFAULT '2011-05-21 00:00:00',
    ->   `col_42` char(92) NOT NULL,
    ->   `col_43` float DEFAULT '6891.0747',
    ->   `col_44` char(5) DEFAULT 'b_',
    ->   `col_45` smallint(6) DEFAULT '-7116',
    ->   `col_46` json NOT NULL,
    ->   `col_47` date NOT NULL,
    ->   `col_48` time NOT NULL,
    ->   `col_49` json DEFAULT NULL,
    ->   `col_50` mediumint(8) unsigned NOT NULL,
    ->   KEY `idx_17` ((cast(`col_46` as unsigned array)),`col_44`,`col_45`),
    ->   PRIMARY KEY (`col_50`,`col_42`) /*T![clustered_index] NONCLUSTERED */
    -> ) ENGINE=InnoDB DEFAULT CHARSET=gbk COLLATE=gbk_chinese_ci;
Query OK, 0 rows affected (0.09 sec)

mysql>
mysql> INSERT INTO `tl339c67ba` VALUES ('2024-02-13 00:00:00','l',NULL,'P5',14565,'800370530','1998-09-03','06:25:19','[-137365.35472449657, 98740.22614562341, -96230.65667019188, 81088.77765708079, 126813.96345066803, 228411.26303739505, -31360.293560806753, -40673.38888043809, 421555.88300412416, 4097.420882023372, -43759.19093454673, -618381.5308285163, 57324.58237695448, -11254.431120740595, -11831.811494338499, 632228.8397806884, -147234.135661395, -10233.283960475097, -60623.850914205344, 20194.112452002195, 31881.62808860628, 20531.172403172404, -48548.85255468495, 104726.24161292956, -180867.75742741392, 160201.2879594866]',16155661);
ERROR 1105 (HY000): Invalid JSON bytes

4. What is your TiDB version? (Required)

nightly-7d33d0d226ea69ef49fb0a6786f8972e112adf4b

[YangKeao]

A compound index contains multi-valued index and any other column which needs restore data will cause this problem. Usually a multi-valued index doesn't need to write restore data, so it works fine. However, when the index contains a column which needs restore data, and the table is using row_id (so it's not CommonHandle), it'll have to write a restored data for the multi-valued index (see genIndexValueVersion0).

Using other utf8 collation will also have this problem:

create table t (j json not null, str varchar(5),  KEY `idx` ((cast(`j` as unsigned array)),`str`));
insert into t values ('1', '');