SADDEX is broken #3915
Description
Describe the bug
SADDEX fails to expire members occasionally. Leaving them not having an expiration.
To Reproduce
Here's a test run in python with the redis-py package:
from asyncio import sleep, Runner, gather
from random import random
from time import time_ns
from redis.asyncio import Redis
TIMEOUT = 10
KEY = 'concurrentRequest:user:1'
async def simulate_request(redis):
member = time_ns()
await redis.execute_command('SADDEX', KEY, TIMEOUT, member)
await sleep(random()) # simulate handling the request
if random() < 0.05: # 5% chance to fail, members should be expired automatically
raise Exception
await redis.srem(KEY, member)
async def main():
# simulate 5000 requests
async with Redis(decode_responses=True) as redis:
requests = [simulate_request(redis) for _ in range(5000)]
await gather(*requests, return_exceptions=True)
# wait for the remaining failed requests to expire
await sleep(TIMEOUT + 1)
# actively expire by reading the key
await redis.smembers(KEY)
# assert the set is empty after running the test
cardinality = await redis.scard(KEY)
print(cardinality)
assert cardinality == 0
if __name__ == "__main__":
with Runner() as runner:
runner.run(main())Expected behavior
The test runs without error, i.e. the set should be empty.
Environment (please complete the following information):
- OS: Debian 11 Bullseye
- Dragonfly Version: v1.23.2-4959bef8d17e6132b4227ea7ca413faf1b1dc037
Additional context
I have a service in which I need to keep track of client's requests concurrency. I use a set to store the request starting-time as members, then it's SREM when the response is returned, therefore keeping the cardinality as the number of on-flight requests. I use SADDEX (add members with expiration to a set) instead of the normal SADD so that if any error occurs, the member will get removed automatically after a certain timeout. I'm aware that SADDEX is dragonfly-only and experimental, it's the one of the reason I'm using dragonfly over redis.