-
Notifications
You must be signed in to change notification settings - Fork 6k
Description
Enhancement
In the original transaction model design, there was an implicit assumption that any modification to a secondary index key would always be accompanied by a corresponding modification to the row key/row value. This assumption holds true for user transactions. However, with the introduction of fast add index, a new scenario has emerged in internal DDL transactions: during one of the phases of the add index merge state, the DDL process needs to perform internal transactions that write only secondary index keys based on results from a temporary index.
By default, user pessimistic transactions do not perform write conflict checks on secondary index keys during the prewrite phase. As a result, when such DDL internal transactions and user DML transactions run concurrently and conflict, the write conflict cannot be properly detected and ordered. This may lead to inconsistencies between data and index after the add index operation completes—posing a correctness issue in certain scenarios.
Possible solutions include:
-
Redesign the pessimistic transaction model so that all secondary index keys undergo write conflict checks during the prewrite phase. However, this would heavily impact the existing pessimistic transaction model and potentially introduce performance regressions.
-
Introduce write conflict checks on secondary index keys only during the
reorg-merge
phase of DDL add index, for user pessimistic transactions involving the corresponding table. While this approach couples internal DDL state with transaction behavior, it minimizes performance impact and contains the change scope.