Fix hub subscriber race, SOURCE_CHAN_SIZE handling, skipped-block serving, retry count and Range.Equals#58
Draft
sduchesneau wants to merge 5 commits into
Draft
Fix hub subscriber race, SOURCE_CHAN_SIZE handling, skipped-block serving, retry count and Range.Equals#58sduchesneau wants to merge 5 commits into
sduchesneau wants to merge 5 commits into
Conversation
An unparseable value was assigned anyway, leaving every subscription channel with capacity 0 and disconnecting all clients with ErrSubscriptionChannelFull.
subscribe runs under the forkable read lock only (all SourceFrom* paths), so concurrent subscribers raced on the slice and a subscription could be lost. The new mutex is a leaf lock, acquired last and never while calling out.
Chains like Solana or NEAR skip block numbers; requiring an exact match made the hub unable to serve a skipped start block, stalling the consumer on the file source. Requests below the lowest held block still fail so callers fall back to another source.
The error path bumped attempt on top of the loop post statement, halving the effective retries. Also make 'attempts' mean the total number of tries.
endBlock was compared by pointer, so bounded ranges built separately never compared equal and IsNext always returned false.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes
Five independent correctness bugs, one commit each (branch
fix/hub-forkable-misc-bugsoffdevelop):Protect hub subscribers slice with a mutex (
hub/hub.go)All
SourceFrom*paths callsubscribeunder the forkable read lock only, whilebroadcastBlock/unsubscriberun under the write lock. Two concurrent subscribers raced on theh.subscribersappend and one subscription could be silently lost (client hangs, never receives blocks). Added a hub-level leaf mutex protecting the slice insubscribe,unsubscribe,broadcastBlockand the shutdown callback; lock ordering is safe because it is acquired last and never held while calling out.Keep default SOURCE_CHAN_SIZE on invalid value (
hub/hub.go)An unparseable
SOURCE_CHAN_SIZElogged "ignoring" but assigned the zero value anyway, giving every subscription channel capacity 0 and disconnecting all clients withErrSubscriptionChannelFull. Parsing is now insourceChanSizeFromEnv, which falls back to the default on invalid input.Start blocksFromNum at first block >= num (
forkable/forkable.go)Chains like Solana/NEAR skip block numbers; requiring an exact match made the hub unable to serve a request starting at a skipped number, stalling consumers on the file source. Now consistent with
blocksFromNumWithForks(>=semantics). Requests below the lowest held block still error so callers fall back to another source instead of silently skipping blocks.Fix double attempt increment in retryable getter (
tracker.go)RetryableBlockRefGetterincrementedattemptboth in the loop post statement and in the error branch, halving the configured retries. Removed the extra increment and madeattemptsmean the total number of tries (attempts=4 → exactly 4 calls);attempts == -1(infinite) unchanged.Compare Range end blocks by value in Equals (
range.go)endBlockwas compared by pointer, so bounded ranges built separately never compared equal andIsNextalways returned false for bounded ranges. Now compares by value (both nil, or both non-nil and equal).Tests
hub/hub_subscribe_race_test.go: 20 concurrent subscribers while blocks broadcast; pre-fix trips-race(multiple DATA RACE warnings) and times out on a lost subscription; post-fix clean.hub/source_chan_size_test.go:t.Setenvtable for unset/empty/garbage/valid values + end-to-endNewForkableHubcheck (pre-fix: channel size 0).forkable/blocks_from_num_test.go: buffer 3,4,6,7 (5 skipped); request 5 → serves 6,7; exact request unchanged; below-lowest and above-head still error.tracker_test.go: counting fake getter; attempts=4 → exactly 4 tries; early-success and canceled-context cases.range_test.go:Equals/IsNexttables including equal-value bounded ranges with distinct pointers.Full suite:
go test ./...passes.go test -race ./...passes for all changed packages; the only-racefailures are pre-existing data races in test code ofbstream(eternalsource/filesource/joiningsource tests) andblockstream(source_test.go), reproduced identically on a cleandevelopcheckout.🤖 Generated with Claude Code