Skip to content

Reduce the infoschema v2 btree memory usage #58321

@tiancaiamao

Description

@tiancaiamao

Enhancement

Currently the infoschema v2 is a two level data struct:

Btree:   table name -> table id
Cache:   table id -> table

The size of cache is under control by @@tidb_schema_cache_size

The btree is not, it store the information of the table name to id index.

I tested the master branch, currently creating 100K tables the btree takes 40.69MB
creating 200K tables the btree takes 80.95MB
...
1M tables, the btree itself could take 400MB
After restart (multiple versions excluded), 1M tables still takes 283MB

Image

So if we have 3M tables above, the memory usage by the btree might goes up to more than 1GB
Too many small objects also empose a huge burden on Go runtime GC.

Metadata

Metadata

Assignees

No one assigned

    Labels

    type/enhancementThe issue or PR belongs to an enhancement.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions