Refactor query sizing to water-fill budget across chains#1392
Open
DZakh wants to merge 23 commits into
Open
Conversation
Replace the per-partition/per-query greedy admission scheduler with a per-chain waterfall: CrossChainState.checkAndFetch visits chains furthest-behind first, handing each its remaining share of the shared buffer budget. ChainState turns that budget into a soft target block using a new chain-wide event density (seeded from cumulative progress, smoothed with an EMA per batch), and FetchState.getNextQuery sizes known-density partitions against that target block while splitting whatever budget is left across partitions with unknown density. This concentrates fetch effort on the bottleneck chain per tick instead of scattering a shared item budget across every chain's full candidate query set.
…uery The unknown-density probe split counted partitions with nothing left to query (already at their endBlock/mergeBlock/knownHeight ceiling), inflating the divisor and under-sizing eligible partitions' queries. Add a hasEligibleRange check mirroring pushQueriesForRange's own gate to exclude them. Splitting partitions into known/unknown passes also broke the original idsInAscOrder query ordering that several tests assert on positionally; restore it by sorting the final query list back into partition order. Update FetchState_test.res fixtures accordingly, including a case that needed distinct expected values across three eligibility scenarios that previously shared one fixture.
Query creation now splits the chain's range budget evenly across in-range partitions each round, rather than sizing every known-density partition against the full chain target while unknown-density partitions fought over the leftover. A partition already holding more budget than its even share (e.g. from an earlier tick's in-flight query) sits out a round so its share flows to the others, and the split is recomputed each round against the shrinking set of partitions still needing more. Also: - Rename estResponseSize -> itemsTarget throughout, since the field is now both the server-side maxNumLogs-style cap and the budget reservation/consumption unit, not just an estimate. - Bucket queries by partition index as they're created instead of sorting the whole result at the end of every tick. - Only trust a partition's density once it has two responses (matching the existing chunking-heuristic gate); a single response is too noisy to size the next query from. - Smooth the chain-wide density EMA as (old + new) / 2 instead of (2*old + new) / 3.
walk, tighten round bound, add coverage - getNextQuery walked every partition's mutPendingQueries twice (once for chainReserved, again to seed reservedByPartition per partition). Merge into a single pass. - Replace the unproven roundsRef < 1000 safety cap with a provable bound: every active partition either finishes or advances its chunk count each round, capped at maxPendingChunksPerPartition, and all active partitions progress in lockstep (not one at a time), so no partition can outlive maxPendingChunksPerPartition + 1 rounds. - Add ChainState_test.res covering the chain density seed (from resumed progress) and the EMA blend. - Add a CrossChainState_test.res case pinning the waterfall's actual cross-chain budget flow: a chain whose real range caps its consumption below its share leaves the remainder for the next chain.
Each round computed ipb = rangeBudget/n once, but then capped every partition's actual budget at min(rangeBudget, ipb - reserved) and decremented rangeBudget after each partition — so a partition processed earlier in the same round (e.g. one forced to overshoot its share via the "at least one full chunk" rule) shrank the pool for whoever came after it. Same reservations, different iteration order, different split (and total consumption could even exceed rangeBudget depending on order). Fix: every partition's share for a round is ipb - reserved, fixed for the whole round; rangeBudget is only re-derived once, from the round's actual total consumption, after every partition has had its fixed shot. A partition can still overshoot its own share, but it can no longer steal from another partition in the same round. Also fixes a SourceManager_test.res assertion that was pinned to the old order-dependent rounding artifact (three identical partitions splitting a budget three ways used to get 16667/16667/16666; they now all get 16667, as they should since they're indistinguishable).
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThis PR replaces estimated response-size budgeting with ChangesItemsTarget budgeting and chain density
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant CrossChainState
participant ChainState
participant FetchState
CrossChainState->>ChainState: getNextQuery(chainTargetItems=remaining + pendingBudget)
ChainState->>FetchState: getNextQuery(chainTargetBlock, chainTargetItems)
FetchState-->>ChainState: Ready(queries)
ChainState-->>CrossChainState: action and consumed itemsTarget
CrossChainState->>CrossChainState: remaining -= consumed
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (2)
packages/envio/src/FetchState.res (1)
1542-1546: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDrop the refactor-history note from this comment.
Keep the invariant, but remove the “walking
mutPendingQueriestwice ... was pure waste” part; it narrates the refactor rather than current behavior.As per coding guidelines,
**/*.res: “Never narrate the refactor itself.”Suggested cleanup
- // and for seeding reservedByPartition (the per-partition running total the - // water-fill rounds check against) — walking mutPendingQueries twice for - // the same sum was pure waste. + // and for seeding reservedByPartition, the per-partition running total the + // water-fill rounds check against.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/envio/src/FetchState.res` around lines 1542 - 1546, Remove the refactor-history wording from the comment in FetchState.res and keep only the invariant explanation. Update the note near the partition budget calculation so it describes the current behavior around existing pending itemsTarget sums, chainReserved, and reservedByPartition without mentioning that mutPendingQueries used to be walked twice or that it was “pure waste.”Source: Coding guidelines
scenarios/test_codegen/test/lib_tests/CrossChainState_test.res (1)
338-349: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winCollapse this test to one assertion.
This test uses two
t.expectcalls for one outcome. Combine the dispatched totals and pending budgets into one value.As per coding guidelines, "
**/*_test.res: Always use single assert to check the whole value instead of multiple asserts for every field".Proposed refactor
- t.expect( - ( - dispatchedItemsByChain->Utils.Dict.dangerouslyGetByIntNonOption(1), - dispatchedItemsByChain->Utils.Dict.dangerouslyGetByIntNonOption(2), - ), - ~message="Chain 1's real range caps it at 200 regardless of its share of the 3000-item pool; chain 2 gets the rest", - ).toEqual((Some(200.), Some(2800.))) - - t.expect( - (a->ChainState.pendingBudget, b->ChainState.pendingBudget), - ~message="Each chain's pendingBudget reflects exactly what it was just dispatched", - ).toEqual((200., 2800.)) + t.expect( + { + "dispatchedItems": ( + dispatchedItemsByChain->Utils.Dict.dangerouslyGetByIntNonOption(1), + dispatchedItemsByChain->Utils.Dict.dangerouslyGetByIntNonOption(2), + ), + "pendingBudget": (a->ChainState.pendingBudget, b->ChainState.pendingBudget), + }, + ~message="Chain 1 is capped at 200, chain 2 gets the remainder, and pendingBudget mirrors dispatch", + ).toEqual({ + "dispatchedItems": (Some(200.), Some(2800.)), + "pendingBudget": (200., 2800.), + })🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scenarios/test_codegen/test/lib_tests/CrossChainState_test.res` around lines 338 - 349, This test currently verifies one outcome with two separate t.expect calls; collapse the CrossChainState test into a single assertion that checks the whole result at once. In the test around dispatchedItemsByChain, ChainState.pendingBudget, and the two chain values (a and b), combine the dispatched totals and pending budgets into one composite value and assert it in a single t.expect so the entire outcome is covered together.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/envio/src/ChainState.res`:
- Around line 662-672: The density update in ChainState.res is seeding
cs.chainDensity too early from progress-only batches with zero events. Update
the logic around deltaBlocks and the chainDensity assignment so it only
initializes Some(...) when the current batch actually processed events, and
otherwise leaves chainDensity as None until the first non-zero event batch
arrives. Keep the adjustment inside the existing chainDensity update path in the
block using chainAfterBatch.totalEventsProcessed, cs.numEventsProcessed, and
cs.chainDensity.
In `@packages/envio/src/FetchState.res`:
- Around line 1737-1754: The water-fill calculation in FetchState.res is
subtracting each partition’s reservation from a per-partition share instead of
solving for a common fill level across underfilled partitions. Update the round
logic around rangePartitions, reservedByPartition, and emitQueries so the budget
is distributed by finding a level where sum(max(0, level - reserved)) matches
rangeBudget, rather than using rangeBudget / n minus reserved. Keep the existing
reservation bookkeeping, but base the per-partition budget on the computed fill
level so partially reserved partitions don’t strand remaining budget.
- Around line 1566-1598: The gap-fill path in FetchState.res is still being
counted against the tick’s water-fill budget, which can starve new-range
queries. Update the gap-fill accounting around pushGapFillQueries so it no
longer contributes to gapFillCost, and make sure any budget logic in
rangeBudget/reservedByPartition excludes these gap repairs from the per-tick
budget. Keep the change localized near the loop that processes
p.mutPendingQueries and the related budget helpers so the “does not consume tick
budget” behavior is consistent everywhere.
- Around line 1700-1712: The pending-budget handling in the query construction
path can emit zero-target requests when Math.round(budget) evaluates to 0, which
later gets executed by SourceManager with minItemsTarget. Update the itemsTarget
calculation in FetchState’s bucket push flow to use a ceil/max-based value
instead of rounding so the recorded target never drops to zero and stays
consistent with the request cap.
---
Nitpick comments:
In `@packages/envio/src/FetchState.res`:
- Around line 1542-1546: Remove the refactor-history wording from the comment in
FetchState.res and keep only the invariant explanation. Update the note near the
partition budget calculation so it describes the current behavior around
existing pending itemsTarget sums, chainReserved, and reservedByPartition
without mentioning that mutPendingQueries used to be walked twice or that it was
“pure waste.”
In `@scenarios/test_codegen/test/lib_tests/CrossChainState_test.res`:
- Around line 338-349: This test currently verifies one outcome with two
separate t.expect calls; collapse the CrossChainState test into a single
assertion that checks the whole result at once. In the test around
dispatchedItemsByChain, ChainState.pendingBudget, and the two chain values (a
and b), combine the dispatched totals and pending budgets into one composite
value and assert it in a single t.expect so the entire outcome is covered
together.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 60a3f8ee-100d-4866-9248-34fc33f7829e
📒 Files selected for processing (13)
packages/envio/src/ChainState.respackages/envio/src/ChainState.resipackages/envio/src/CrossChainState.respackages/envio/src/FetchState.respackages/envio/src/sources/SourceManager.resscenarios/test_codegen/test/ClientAddressFilter_test.resscenarios/test_codegen/test/EventBlockFilter_test.resscenarios/test_codegen/test/IndexerState_test.resscenarios/test_codegen/test/lib_tests/ChainState_test.resscenarios/test_codegen/test/lib_tests/CrossChainState_test.resscenarios/test_codegen/test/lib_tests/FetchState_onBlock_test.resscenarios/test_codegen/test/lib_tests/FetchState_test.resscenarios/test_codegen/test/lib_tests/SourceManager_test.res
…exer-query-control-kebggn # Conflicts: # packages/envio/src/ChainState.res # packages/envio/src/ChainState.resi
…1394) The per-partition round budget was `ipb - reserved`, where `ipb` was an even share of only the *remaining fresh* budget (`rangeBudget / n`) while `reserved` accumulated each partition's full footprint (existing in-flight + gap-fill + this call's prior-round emissions). Those two are on different scales, so once a chunked partition's running reservation passed a later round's fresh share, `ipb - reserved` went negative and the partition was dropped — leaving budget unspent even though it still had range to fetch and a sibling had just freed its share by filling early. Compute the round's level as a real water-fill line — (remaining fresh budget + the still-not-filled partitions' current footprint) / count — and top each partition up toward it. A partition already above the line gets nothing (its head start is its whole share); the rest absorb the leftover, so the budget is fully used and per-partition totals stay even. The loop now runs until either the not-filled set drains or the whole fresh budget is reserved, dropping the redundant round cap: a partition survives a round only by advancing chunksUsedThisCall (bounded by maxPendingChunksPerPartition) or consuming budget, so it terminates on its own. Add a regression test: a range-capped partition and a deep partition splitting a 900-item budget — the deep one now absorbs the capped one's freed share (4 chunks / 720 items) instead of stopping at 2. Claude-Session: https://claude.ai/code/session_0134TTgxQ3ci5mWUnt928yr9 Co-authored-by: Claude <noreply@anthropic.com>
An unknown-density partition's open-ended probe was sized to its full even share of the chain's budget with no ceiling. When such a chain leads the furthest-behind waterfall, that share is the entire cross-chain buffer pool, so its single probe consumed 100% of the remaining budget and starved any sibling chain needing its own first probe in the same tick (e.g. multiple chains entering the reorg threshold together). Cap the probe at maxItemsTarget (10_000), restoring the old bounded-default ceiling. The leftover budget flows to the next chain via checkAndFetch's remaining subtraction, exactly as before. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GBSAcMXTNy2K8i16bkJELo
…control-kebggn' into claude/multichain-indexer-query-control-kebggn
The resume-seed path only sets chainDensity once numEventsProcessed > 0, but the per-batch EMA update seeded Some(0.) after any progress-only batch (blocks advanced, no events), contradicting that documented behavior and making the first real batch blend against 0 instead of seeding from its own density. Guard the EMA seed on the batch having events, matching the resume path. The Some(oldDensity) blend is unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GBSAcMXTNy2K8i16bkJELo
…exer-query-control-kebggn
* Enforce reservation == server cap and stop chunking without trusted density - Floor itemsTarget at 1 at creation (densityItemsTarget, water-fill chunk loop, probe) so a query's budget reservation always equals the maxNumLogs-style cap sent to the server; drop SourceManager's 2000-item fallback that let density-0 queries return up to 2000 unaccounted items. - Emit density-priced chunks only for a trusted positive density; density-0 and unknown-density partitions get a single open-ended probe sized at the even split of the tick's fresh budget (maxItemsTarget cap removed). This removes the chunkCost=0 path that flooded 10 free hard-bounded chunks per partition and froze the 1.8x range growth. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01McpcXkR3pPWfEcq4mCj9Sw * Extract getTrustedDensity helper for water-fill chunk sizing Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01McpcXkR3pPWfEcq4mCj9Sw * Make query itemsTarget an int and trim redundant comments The ceil-to-int conversion now happens once at query creation, so the reservation, the budget accounting, and the server cap all use the same integer value; SourceManager passes it through untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01McpcXkR3pPWfEcq4mCj9Sw * Price gap-fill by trusted density with available-density fallback Gap queries now use getTrustedDensity: chunks only on a trusted positive density (same rule as the water-fill); a trusted-zero density prices the whole gap as one open query, and a partition with no density signal prices it by available density — its equal-divide budget spread over the remaining range this tick — so a small gap reserves proportionally little instead of a noisy one-sample estimate or a NaN from dividing by a zero range. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01McpcXkR3pPWfEcq4mCj9Sw * Cap follower chains at the leader's target progress in the waterfall Chains beyond the most-behind one in the budget waterfall are now capped at that leader's target progress, mapped onto their own block range (ChainState.progressAtBlock/blockAtProgress), so no chain runs further ahead than the chain the shared buffer pool is prioritizing. A chain visited after the pool is exhausted simply sits out the round — its reservations release as responses land, so the next tick redistributes. FetchState's dynamic-contract partition merge now inherits the sum of its parents' trusted densities (weighted onto the merged partition's min query range) instead of resetting to 0, so a merge with density history doesn't regress to an unpriced probe. Update E2E/rollback tests to the now-serialized cross-chain query dispatch (most-behind chain queries first; siblings follow once its response releases budget) and to give density-dependent chunking tests a nonzero item count to trust. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01McpcXkR3pPWfEcq4mCj9Sw * Cap a clamped chain's fresh budget at its density-priced range cost When a chain's target block is clamped (head, endBlock, or the cross-chain alignment cap), a known-density chain's fresh budget is now capped at density x clamped range (in-flight reservations stay on top so they don't crowd out new partitions). The unused remainder stays in the waterfall's pool and flows to the next chain in the same tick, instead of being held by an oversized probe until the response lands. This also removes the drain loop the infinite-reorg-loop test needed: the non-reorg chain's post-rollback refetch now reserves only its real range cost, so the reorg chain gets budget immediately. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01McpcXkR3pPWfEcq4mCj9Sw * Give head-bound queries 2x density headroom in the budget cap A query clamped at the head sized exactly at density x range truncates at the server cap whenever the range is slightly denser than the estimate, forcing an immediate catch-up query for the last few blocks. Double the range cost for head-bound targets so one query usually suffices; the extra reservation releases as soon as the response lands. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01McpcXkR3pPWfEcq4mCj9Sw * Refine chain budget caps: endBlock ceiling, 5k probe cap, 3x head headroom - targetBlock now clamps at endBlock (when below the head) via a shared fetchCeiling helper, so endBlock'd chains stop sizing and aligning against range they'll never fetch. - A chain with no positive density signal caps its fresh budget at 5k, so one unknown chain measuring its first responses no longer holds the whole cross-chain pool. - Head/endBlock-bound queries get 3x (was 2x) density headroom against truncating at the server cap and needing a catch-up query. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01McpcXkR3pPWfEcq4mCj9Sw * Fix clippy::useless_borrows_in_formatting across cli package Remove redundant & references in format!/anyhow! arguments flagged by the CI-pinned clippy (rust 1.97). Pre-existing on the base branch, unrelated to the SourceManager/waterfall changes in this PR — fixed here since it was blocking cargo-test from going green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01McpcXkR3pPWfEcq4mCj9Sw * Fix clippy::to_string_in_format_args exposed by the previous fix Removing the redundant & in anyhow!'s self.id.to_string() surfaced a second lint on the same line: ChainId (u64) already implements Display, so .to_string() inside the format arg is itself redundant. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01McpcXkR3pPWfEcq4mCj9Sw --------- Co-authored-by: Claude <noreply@anthropic.com>
main's cargo-test job started failing clippy (-D warnings) on pre-existing code after a stable-toolchain drift (no rust-toolchain pin), unrelated to this PR's scheduling changes but inherited via the origin/main merge. Removed the redundant `&` in format!/anyhow! args across 6 files, and dropped a now-also-flagged explicit .to_string() on a Display type in validation.rs. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GBSAcMXTNy2K8i16bkJELo
…control-kebggn' into claude/multichain-indexer-query-control-kebggn # Conflicts: # packages/cli/src/hbs_templating/hbs_dir_generator.rs
- Replace the per-round mean line in FetchState.getNextQuery with an exact water level (sum of top-ups equals the poured budget), so uneven in-flight reservations can no longer inflate other partitions' allotments past the fresh budget - Size unknown-density probes by their water-fill allotment instead of a fixed pre-round even split, so leftover budget reaches the partitions without reservations instead of being stranded - Gate the 3x head headroom on the chain having caught up once (isReady) - Blend chain density weighted by the batch's block span instead of a flat (old + new) / 2 - Clamp progressAtBlock at 0 for the initial -1 fetch frontier - Skip chains with no known height in the cross-chain waterfall so they wait for a block instead of setting a degenerate alignment line Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvUhc8DhbxjGPEtHoDNThJ
…exer-query-control-kebggn
Small batches (a few blocks) should barely nudge the chain density estimate, while anything spanning 100+ blocks is a trustworthy fresh sample. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvUhc8DhbxjGPEtHoDNThJ
…control-kebggn' into claude/multichain-indexer-query-control-kebggn
…exer-query-control-kebggn
* Add chunk itemsTarget headroom and budget-driven chunk emission Chunk reservations now carry a headroom multiplier over the density estimate (1.5x during backfill, 3x in realtime, chosen in CrossChainState.checkAndFetch and threaded down to FetchState.getNextQuery), so a denser-than-expected range doesn't truncate at the server cap. Open-ended probes stay allotment-sized. The emit loop replaces the precomputed chunkCost/affordable estimate with per-chunk actual itemsTarget accounting: the first chunk always emits full-size, subsequent chunks only while they fit the budget, and the min-one-chunk force applies once per call instead of once per water-fill round. Cap-hit truncations (partial response with itemsCount >= itemsTarget) no longer update the chunk range history — they reflect our own reservation, not server capacity. Sub-cap partials still do. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KpFcYPn8UbQfaEfW6gjant * Restore min-one-chunk per water-fill round A leftover re-pour forces a full chunk again, so the budget never strands on chunk quantization; the overshoot stays bounded at one chunk per partition per round and self-corrects via the reported reservations. Drops the per-call emittedThisCall bookkeeping. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KpFcYPn8UbQfaEfW6gjant --------- Co-authored-by: Claude <noreply@anthropic.com>
* Contain queries to the chain target block and rework cold-start sizing - No chunk or gap-fill query starts past chainTargetBlock; emitted chunks keep their full span, with endBlock/mergeBlock staying the hard bounds. Skipped gaps regenerate from the pending-walk and fill once the target reaches them. - A chain with no density signal targets frontier + coldTargetRange (init 20k), doubling whenever it goes idle without producing a signal, capped at the fetch ceiling. The cross-chain waterfall clamps a cold chain to min(5k, targetBufferSize), replacing the internal probe clamp, and a cold leader no longer sets the alignment line. - Query sizing uses effectiveDensity = max(processing EMA, ready-buffer density), so a dense buffer overrides a stale-low EMA and ready items alone take a chain out of cold mode. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018CiEziFbVe1P3Y6iuyfT5Z * Replace cold-window doubling with a fixed 20k range Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018CiEziFbVe1P3Y6iuyfT5Z * Span ready-buffer density from the processing block number The buffer is consumed at batch creation while committed progress only catches up after the batch commits, so mid-batch the density's numerator shrank without the denominator following. Track the in-flight batch's progress as processingBlockNumber (advanced in advanceAfterBatch, caught up in applyBatchProgress, rewound on rollback) and use it as the span's lower boundary. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018CiEziFbVe1P3Y6iuyfT5Z * Warm the chain with seed events in the partition-merge E2E test A chain with no density signal now targets frontier + 20k, which gates the far DC partitions this test fetches in parallel. Seed 100 events in the registering response so the chain has a density signal and enough range budget for DC2's full 10-chunk pipeline; cold gating itself is covered by unit tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018CiEziFbVe1P3Y6iuyfT5Z --------- Co-authored-by: Claude <noreply@anthropic.com>
…exer-query-control-kebggn
* Reserve budget at honest itemsEst instead of headroomed itemsTarget Queries now carry both itemsTarget (server-side cap, sized with the chunk headroom multiplier) and itemsEst (raw density estimate). Reservations, pendingBudget, and water-fill footprints use itemsEst, so headroom no longer throttles pipeline depth. The extra 3x budget cap for caught-up chains is dropped — truncation safety lives solely in the itemsTarget cap, keeping realtime headroom at 3x instead of compounding to 9x. Aligned chains may now run 5% past the leader's line to stop clamp flapping when progress tracks closely. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QyDQ2imktXA4jL8PtWYSmd * Raise default target buffer to 100k and chunk pipeline cap to 12 Measured on the erc20 template against real HyperSync data: at 50k the dense chain's buffer drained to zero in a quarter of samples (processing starved on fetching), while at 100k it almost never does and throughput matches the processing ceiling. Beyond 100k there's no further gain — 300k only grows the resident buffer. The chunk cap rarely binds at 12 but gives the pipeline headroom at the larger budget. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QyDQ2imktXA4jL8PtWYSmd --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the per-query admission model with a waterfall-based budget distribution across chains. Instead of each query carrying an estimated response size and being admitted independently, chains are visited in priority order (furthest-behind first), each receives the remaining budget plus its own pending queries' share, and sizes its queries against a chain-density-derived target block. This allows chains with low density or short remaining ranges to naturally leave budget for others, rather than starving near-head chains with a per-query pool.
Key Changes
Query sizing model: Renamed
estResponseSize→itemsTargetand removed cross-chain scheduler fields (chainId,progress) fromFetchState.query. Queries are now sized againstchainTargetBlock(a soft per-tick horizon) andchainTargetItems(the budget handed to each chain), not a global pool.Density calculation: Replaced
calculateEstResponseSize(which scaled a default estimate by partition count) withdensityItemsTarget, which computes items from a partition's real event density. Partitions with no signal yet are sized against their even split of the chain's range budget instead.Gap-fill refactor: Extracted
pushGapFillQueriesto unconditionally fill holes between pending chunks (already-committed range, not subject to this tick's budget). Returns the cost so the caller can track total consumption.Water-fill rounds:
getNextQuerynow accepts~chainTargetBlockand~chainTargetItemsparameters and implements a multi-round algorithm:CrossChainState dispatch:
checkAndFetchnow visits chains inpriorityOrder(furthest-behind first), hands each the remaining budget plus its pending share, and subtracts actual consumption before moving to the next chain. Removed the per-query pool andcompareByProgresssort.ChainState density tracking: Added
chainDensityfield (seeded from resumed progress, refined per-batch with EMA) to turn a chain's item budget into a target block for query sizing.Notable Implementation Details
waterFillStatetracks per-partition mutable state (cursor, chunks used, pending count) threaded through the water-fill rounds.computeQueryEndBlockis inlined to avoid recomputation across gap-fill and water-fill phases.existingReservedByPartitionis computed once and reused for both the call-wide ceiling and per-partition running totals, avoiding duplicate walks ofmutPendingQueries.~chainTargetBlockand~chainTargetItemstogetNextQuery, and to verify water-fill behavior (e.g., a low-density chain leaving budget for others).https://claude.ai/code/session_01GBSAcMXTNy2K8i16bkJELo
Summary by CodeRabbit
itemsTarget) rather than estimated response sizes.