A new rule engine for Optimizer #43360
Description
Feature Request
Is your feature request related to a problem? Please describe:
Current optimizer rewrite rules are executed sequentially once and some specific rules (column prunning) are done multiple times hard coded.
- Applying rules multiple times is not possible if a rule triggers another rule (for example: eliminating projections triggers join elimination). Missing such cases are lost optimization opportunities.
Describe the feature you'd like:
The New rule engine for pre-processing phase changes logical plan to another logical plan.
It also should satisfy following requirements:
- Applying rules multiple times by rule engine instead of hard code manually.
- Clean fast path for fast queries
- Easier testing for enabling/disabling rules
- Adding new rules is easier with this new architecture
Design
Design
- Code factoring:
- Functional components (FD, derived statistics and hint) out of logical plan building. The Rule Engine: refactor of Functional components #43361
- Make sure each optimization rewrite has no side effects. If any, remove the side and apply it as a functional rewrite.
- Rule engine design is based on the following abstraction
- List of API's for rule1, rule2, ... RuleN. (ordered rule set as set of interfaces)
- Order of rules "RuleOrder" = {RuleX, RuleY, ....}
- 2 dimensional array of rule interactions where M(i,j) = true means that If Rulei is applied then we should also do Rulej
The final product is depicted in the diagram below where we do pre-processing (FD, derived stats and hint annotations) of rewrites first and then we execute the rewrite engine.
