Lightning: Performance Regression on 6.2.0 Compared with 5.3.3 on Parquet Data Source with Strings

[dsdashun]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

Import a moderate data set into different versions of Lightning: 1) 6.2.0; 2) 5.3.3
The reproduce conditions:

• The data format of the source is parquet
• The data set to import contains some string columns
• The table schema should set the encoding to 'binary'

2. What did you expect to see? (Required)

• It takes longer time to finish the import on 6.2.0 than in 5.3.3
• From the log, we can see the encode KV time is much longer in 6.2.0, the import KV time is roughly the same as in 5.3.3

3. What did you see instead (Required)

The import time on newer version of Lightning should be roughly the same as the older version

4. What is your TiDB version? (Required)

6.2.0

[dsdashun]

/component lightning

[dsdashun]

I've captured the profile of 6.2.0. The bottleneck lies in converting each string datum value from data source into a normalized string datum value. This logic was first introduced in 5.4.0. Here is the PR . Originally, the string value doesn't do any normalizing processing. After this PR, each string datum will check the charset and collation to normalize the value.

By default, when creating a string column in a table without setting collations and charsets, the collation is empty, and the charset is 'binary'. In this case, when the normalizing logic tries to check the collation, it cannot find the collation. So the logic generates an error object with a stack trace and returned back. When the upper level caller catches the error, it logs a warning message, ignores the error, and falls back to return the raw string as the datum value.

So compared to 5.3.3, each string value normalization will generate an extra error object with a stack trace, and record an extra warning log. Surprisingly, the cumulative time for generating errors with stack traces is huge. That causes the bottleneck.