Description
While looking through the async rate limiter implementation, I noticed that EnhancedTokenBucketRateLimiter.acquire() awaits asyncio.sleep() while still holding the synchronization lock.
This means that when multiple coroutines are waiting for tokens:
- One coroutine acquires the lock.
- It determines it must wait.
- It sleeps while still holding the lock.
- Other waiting coroutines cannot even check token availability until the sleeping coroutine wakes.
This effectively serializes waiting tasks and introduces unnecessary contention/head-of-line blocking.
Suggested fix
Release the lock before sleeping.
while True:
async with lock:
if token available:
consume token
return
wait_time = ...
await asyncio.sleep(wait_time)
This preserves correctness while allowing other waiting coroutines to make progress. I'd be happy to submit a PR if this approach sounds reasonable.
Description
While looking through the async rate limiter implementation, I noticed that
EnhancedTokenBucketRateLimiter.acquire()awaitsasyncio.sleep()while still holding the synchronization lock.This means that when multiple coroutines are waiting for tokens:
This effectively serializes waiting tasks and introduces unnecessary contention/head-of-line blocking.
Suggested fix
Release the lock before sleeping.
This preserves correctness while allowing other waiting coroutines to make progress. I'd be happy to submit a PR if this approach sounds reasonable.