Skip to content

fatal error: concurrent map read and map write from Config.Labels #45561

@gengliqi

Description

@gengliqi

Bug Report

fatal error: concurrent map read and map write

 goroutine 513 [running]:
 github.com/pingcap/tidb/config.(*Config).GetTiKVConfig(...)
 	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/config/config.go:312
 github.com/pingcap/tidb/sessionctx/variable.glob..func188({0xc04ef57a40?, 0xc001c01800?}, 0xc04e8fdcae?, {0xc04e8fdcae?, 0x2?})
 	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/sessionctx/variable/sysvar.go:1026 +0x33e
 github.com/pingcap/tidb/domain.(*Domain).rebuildSysVarCache(0xc000cefe00, {0x5025338, 0xc0008e2c80})
 	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/domain/sysvar_cache.go:147 +0x71e
 github.com/pingcap/tidb/domain.(*Domain).LoadSysVarCacheLoop.func1()
 	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/domain/domain.go:1415 +0x1ff
 created by github.com/pingcap/tidb/domain.(*Domain).LoadSysVarCacheLoop
 	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/domain/domain.go:1376 +0x172

tidb/server/http_handler.go

Lines 2221 to 2231 in b4183e1

if len(labels) > 0 {
cfg := *config.GetGlobalConfig()
if cfg.Labels == nil {
cfg.Labels = make(map[string]string, len(labels))
}
for k, v := range labels {
cfg.Labels[k] = v
}
config.StoreGlobalConfig(&cfg)
logutil.BgLogger().Info("update server labels", zap.Any("labels", cfg.Labels))
}

The code above wants to update Labels in Config. It copies the global config first and then changes the Labels in the copy.
However, the Labels is a map[string]string so the copy is shallow, which causes this data race.

Metadata

Metadata

Assignees

No one assigned

    Labels

    affects-6.5This bug affects the 6.5.x(LTS) versions.affects-7.1This bug affects the 7.1.x(LTS) versions.component/pdreport/customerCustomers have encountered this bug.severity/majortype/bugThe issue is confirmed as a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions