TiDB panic when new collation is disabled and two expressions use different collation

[djshow832]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

1. Create a cluster with TiDB version before v4.0.0.
2. Upgrade the cluster to master.
3. Create 2 tables with the same charset but different collations:

create table t1(code varchar(32)) CHARSET=utf8 COLLATE=utf8_general_ci;
create table t2(code varchar(32)) CHARSET=utf8 COLLATE=utf8_bin;

4. Execute a query:

select * from t1 join t2 on t1.code=t2.code and t1.code in ('1') and t2.code in ('1');

2. What did you expect to see? (Required)

Empty query result

3. What did you see instead (Required)

TiDB reports error:

ERROR 1105 (HY000): runtime error: invalid memory address or nil pointer dereference

The stack:

github.com/pingcap/tidb/expression.NewFunctionInternal\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/expression/scalar_function.go:277\ngithub.com/pingcap/tidb/expression.ColumnSubstituteImpl\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/expression/util.go:515\ngithub.com/pingcap/tidb/expression.ColumnSubstitute\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/expression/util.go:414\ngithub.com/pingcap/tidb/expression.(*propConstSolver).propagateConstantEQ\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/expression/constant_propagation.go:214\ngithub.com/pingcap/tidb/expression.(*propConstSolver).solve\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/expression/constant_propagation.go:352\ngithub.com/pingcap/tidb/expression.(*propConstSolver).PropagateConstant\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/expression/constant_propagation.go:711\ngithub.com/pingcap/tidb/expression.PropagateConstant\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/expression/constant_propagation.go:361\ngithub.com/pingcap/tidb/planner/core.(*DataSource).PredicatePushDown\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/planner/core/rule_predicate_push_down.go:135

4. What is your TiDB version? (Required)

master

[djshow832]

This bug is introduced from v6.6.0 by #38826:

1. https://github.com/pingcap/tidb/blob/v6.5.0/expression/util.go#L418 is removed.
2. Then the coercibility of column code is not derived to '1', so the coercibility of two '1' constants are both 0.
3. When comparing the two constants, the charset is the same, the collations are different, but the coercibilities are both 0, which leads to an error Illegal mix of collations (utf8_general_ci,EXPLICIT) and (utf8_bin,EXPLICIT) for operation '='".
4. After the error, the execution doesn't stop. The function returns a nil expression and then panics when visiting nil.