Skip to content

lightning failed to log encoding error #44321

@lance6716

Description

@lance6716

func (e *BaseKVEncoder) LogKVConvertFailed(row []types.Datum, j int, colInfo *model.ColumnInfo, err error) error {
var original types.Datum
if 0 <= j && j < len(row) {
original = row[j]
row = row[j : j+1]
}
e.logger.Error("kv convert failed",
zap.Array("original", RowArrayMarshaller(row)),

When the source file has ill-formated row, the row is very large so it can't be logged. We can't locate the bad row

2023-06-01 10:55:27.887067902 +0800 CST m=+35.508861459 write error: write length 20297970322 exceeds maximum file size 536870912

image

Metadata

Metadata

Assignees

Labels

affects-6.5This bug affects the 6.5.x(LTS) versions.component/lightningThis issue is related to Lightning of TiDB.severity/moderatetype/bugThe issue is confirmed as a bug.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions