Skip to content

Async rate limiter sleeps while holding the lock, causing unnecessary contention #55

Description

@nouraellm

Description

While looking through the async rate limiter implementation, I noticed that EnhancedTokenBucketRateLimiter.acquire() awaits asyncio.sleep() while still holding the synchronization lock.

This means that when multiple coroutines are waiting for tokens:

  1. One coroutine acquires the lock.
  2. It determines it must wait.
  3. It sleeps while still holding the lock.
  4. Other waiting coroutines cannot even check token availability until the sleeping coroutine wakes.

This effectively serializes waiting tasks and introduces unnecessary contention/head-of-line blocking.

Suggested fix

Release the lock before sleeping.

while True:
    async with lock:
        if token available:
            consume token
            return
        wait_time = ...

    await asyncio.sleep(wait_time)

This preserves correctness while allowing other waiting coroutines to make progress. I'd be happy to submit a PR if this approach sounds reasonable.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions