optimize execution of general DDL

[D3Hunter]

Enhancement

we have optimize create table using fast-create, but there are scenarios that we might create a lot of database, or run many DDLs like add/modify column in the maintenance window.
so we want to optimize execution of general DDL, and in a unified way

• ddl: sync schema version using watch, notify sessions on owner node by job id #53217
• ddl: check table/database existance using infoschema only #53882
• query since min job id to mitigate select very slow on an empty table from delete from xx #52905, below 2 PRs are reverted in ddl: revert 'query from min job id' #54180
  • ddl: query since min job id to mitigate slowness due to un-compacted deleted kv #54122
  • ddl: query since min job id for tidb_mdl_info #54142
• move job dependency calculation into memory, to avoid the slower SQL
  • ddl: only use runningJobs to check job dependency #54003
  • ddl: support referring objects in runningJobs #54110
• ddl: enlarge general ddl worker pool #54041
• ddl: manage job lifecycle in worker #53747
  • ddl: clear MDL related tables for create view #53944
  • ddl: fix flaky test TestCancel & fix panic when we failed to get job in delivery2Worker #53887
• [low priority] notify the session which is not on DDL owner node when owner finished handling by RPC or other way
• [hold] add index on type for tidb_ddl_job to make some query faster
• other refactors
  • ddl: decouple job scheduler from 'ddl' and make it run/exit as owner changes #53548
  • ddl: handle cluster state retrieval failure #53816
  • ddl: fill correct involved schemainfo for 'flashback table' and 'flashback schema' #54141