A dead LDAP upstream can block the authentication and `show global variables`.

[YangKeao]

Enhancement

Some functions about LDAP don't have timeout mechanism (e.g. StartTLS), therefore, the RLock of it cannot be released until the upstream returns.

I'd like to address the following enhancement:

1. Don't include any network / IO in the RLock. An RLock in golang can block the Lock, and a pending write lock will blocks all other RLock, which will make the things much worse.
2. Add timeout mechanism for LDAP functions.
3. Add metrics for them.

[aki263]

@YangKeao
We should also add caching creds for upto a minutes so we dont DDoS LDAP sever.

[YangKeao]

@aki263 Good suggestion.

TiDB has a mechanism to retry to establish connection with the LDAP server (for 10 times), when it fails. I'll add some backoff to avoid retrying too frequently. I think it'll help a lot to limit the request frequency.

Cache on authentication needs more consideration/discussion, because the latency of cache invalidation is not always expected (especially when it's related to security) 🤔 .