[madhuchavva]

feat: async sticky bucketing for the multi-user client — non-blocking reads, fire-and-forget writes

Sticky-bucket I/O moves off the evaluating thread: identifier-scoped coalesced reads, merged fire-and-forget writes, prefetch, and flush

opt-in · additive & backward compatible · no new dependencies · JS/Python parity

Branch: mc/async-sticky-read-path → stacked on #234 (concurrency groundwork)
Commits: ff5d0c3, dbb82a8, dd54119 (feature) · e36d138 (review follow-ups)

---

Summary

Adds an asynchronous sticky-bucketing path to the multi-user GrowthBookClient so sticky-bucket
I/O no longer runs on the evaluating thread. Evaluation stays synchronous and CPU-only (matching the
JavaScript and Python SDKs) — reads and writes move to the edges.

The change is additive and opt-in. The legacy single-user GrowthBook + GBContext path is
untouched, and a multi-user client with no sticky-bucket service configured behaves exactly as before.

• AsyncStickyBucketService — a CompletionStage-based sibling of StickyBucketService.
  Natively-async stores (Lettuce RedisFuture, Reactor Mono#toFuture(), …) plug in without
  adaptation; existing blocking StickyBucketService implementations need no porting — the client
  offloads their calls to its executor via SyncOffloadStickyBucketAdapter.
• Identifier-scoped, coalesced reads — only sticky-identifier attributes are sent to the store,
  and concurrent reads for the same identifiers collapse to one store call.
• Fire-and-forget writes backed by an authoritative in-process document map — reads see this
  process's own writes (read-your-writes), and a slow or stale store read can never roll back an
  assignment. Writes are merged and serialized per document key.
• prefetchStickyBuckets(userContext) — JS-style prefetch so a request does its one sticky read
  up front, then evaluates many features/experiments with zero further sticky I/O.
• flushStickyBucketSaves() — awaits all outstanding save attempts (see Durability). Called
  automatically by shutdown().

Evaluation is wired to a StickyBucketDocWriter seam (same shape as the Python SDK's
EvaluationContext.save_sticky_bucket_doc); when no writer is set the evaluator falls back to the
synchronous StickyBucketService.saveAssignments — the legacy path, unchanged.

---

Public API (all additive)

• new interface AsyncStickyBucketService (getAssignments, saveAssignments, batched
  getAllAssignments, shared getKey)
• new class SyncOffloadStickyBucketAdapter — present a blocking StickyBucketService as async
• new functional interface StickyBucketDocWriter — evaluation → client persistence seam
• GrowthBookClient.prefetchStickyBuckets(UserContext) → CompletableFuture<UserContext>
• GrowthBookClient.flushStickyBucketSaves() → CompletableFuture<Void>
• Options: asyncStickyBucketService, asyncExecutor, stickyBucketCacheTtlSeconds,
  stickyBucketCacheSize; new Options.isStickyBucketingConfigured() (sticky bucketing now activates
  for either service flavour). Existing positional constructors are preserved via delegating overloads.

StickyBucketManager is package-private — its invariants are implementation detail, reached only
through the client's methods. No new third-party dependency (JDK + Guava + Lombok + SLF4J, all
already on the classpath).

---

Behaviour changes (multi-user client only; changelog-flagged)

• Reads are identifier-scoped and coalesced. Previously every primitive attribute was sent to the
  store on every evaluation; now only the sticky-identifier attributes are, and concurrent identical
  reads share one call.
• Saves are fire-and-forget, merged, and per-key serialized. Previously a single blocking,
  unmerged saveAssignments ran inside evaluation. Durability is now reached via
  flushStickyBucketSaves() / shutdown().
• A failing sticky read propagates out of evaluation instead of silently degrading to non-sticky
  bucketing — matching the JS/Python SDKs, which never evaluate without sticky data when a store is
  configured. Multi-user clients using a blocking StickyBucketService that relied on silent fallback
  should note this.

---

Durability

flushStickyBucketSaves() (and shutdown()) complete when every pending save has been attempted
— not a guarantee that every assignment reached the store. A write the store rejects is logged and not
retried (JS/Python parity); the authoritative in-process map still holds the document.

---

Concurrency

StickyBucketManager is shared across threads with no synchronized (virtual-thread friendly) and no
side effects inside ConcurrentHashMap compute lambdas; store calls are launched outside all locks.
Design invariants (ported from the Python 2.4.0 design, growthbook-python #128/#129): authoritative
overlay, per-key save serialization, coalesced reads, bounded memory.

Review follow-ups (correctness fixes — commit e36d138)

Code review surfaced five P1 concurrency defects and a few lower-severity issues; all are fixed in this
branch:

1. Eviction could drop an unsaved assignment. recordAndSave now merges the document under the
   per-key SaveState lock inside a validate-retry loop; eviction locks the same state before removing
   it, so a freshly recorded assignment can no longer be pruned between the merge and the in-flight
   transition (previously driveSave could read a null snapshot and silently skip the save once

   1000 documents were resident).

2. Reads lost eviction protection too early. The authoritative overlay is now applied while the
   fetch is still registered (its startSequence still guards eviction); the in-flight entry is
   removed only after completion, and each caller gets its own copy — a slow reader can no longer have
   a needed document pruned before its overlay runs.
3. Eviction exposed stale cached reads. Pruning now invalidates read-cache entries whose attribute
   map addresses an evicted document key, so a cached miss can't outlive the local document it would
   have been overlaid with.
4. Shutdown could strand in-flight reads. close() completes every pending fetch exceptionally
   before the offload executor is torn down, so a queued read task discarded by shutdownNow() can no
   longer leave an evaluation thread blocked forever.
5. Refresh could skip saved assignments. getEvalContext captures one GlobalContext snapshot and
   uses it for both identifier derivation and evaluation, so a mid-read refresh can't pair new rules
   with documents fetched for the old identifiers.

Also: run(experiment, …) now loads the inline experiment's own hashAttribute/fallbackAttribute,
so its persisted assignment is honoured even with no configured or feature-derived identifiers; the
overlay returns a ConcurrentHashMap so a reused prefetched context is safe to evaluate concurrently;
the default getAllAssignments tolerates a null CompletionStage; and the flush Javadoc no longer
overstates durability.

---

Tests

./gradlew :lib:test passes in full on JDK 17 — 2 new test classes (29 tests), deterministic by
construction (hand-completed service doubles, latch-gated executors, no sleeps or real time):

Test	Count	Covers
StickyBucketManagerTest	18	fetch coalescing, authoritative overlay (stale read can't roll back a local write), per-key save serialization with one trailing save, rejection stays dirty + flush retries, failed save not retried but doc stays authoritative, bounded eviction never drops unsaved docs and invalidates stale cached reads, close() unblocks an in-flight read, opt-in TTL cache semantics
GrowthBookClientStickyBucketTest	11	identifier-scoped reads, inline-experiment identifier fetched without configured/derived identifiers, sync-service offload off the caller thread with batched lookups, caller-preloaded docs skip the fetch, prefetch-once-then-many, read-failure propagation, fire-and-forget writes + flush, in-process read-your-writes, shutdown flush, caller-supplied executor never shut down, mutual-exclusion validation

[andywhite37]

Madhu is on leave, so the team is picking this up. The async design in #235–#237 is accepted as written; next steps are rebasing #235 onto main to resolve conflicts, then reviewing and restacking #236 and #237.

[vazarkevych]

@greptile review

[greptile-apps]

Confidence Score: 5/5

[Medium impact] No new blocking issue remains under the supplied stack assessment.

_{Reviews (2) · Last reviewed commit: "fix: close sticky bucket manager evictio..." · Reviewed by Greptile}

[vazarkevych]

@greptile review