Tracking Issue For Maintaining Priority Queue In Memory

[0xPoe]

Enhancement

Currently, we use a priority queue at TiDB to automatically update statistics. However, we currently maintain this queue by rebuilding it every time. This causes the following problems:

1. When there are many tables, scanning all tables' schema information can have a big impact on the meta information cache. For example, it will load all the tables at once, which is a very slow process and affects other modules' use of the schema cache.
2. Because we scan it too frequently, we'll spend a lot of CPU on building this queue, since we'll be scanning all the stats cache and the schema cache. See more: High CPU usage for a cluster with large # of tables #49972
   To solve the above issues, we can try to keep the priority queue in memory instead of rebuilding it every time.