Scan protocol traits - #10047
Open
joseph-isaacs wants to merge 3 commits into
Open
Scan protocol traits#10047joseph-isaacs wants to merge 3 commits into
joseph-isaacs wants to merge 3 commits into
Conversation
Add explicit planner and morsel state machines, deferred stage composition, and byte IO contracts in separate modules. Keep live execution worker-local and IO owner identities independent of scan scopes. Include protocol and construction tests. Signed-off-by: "Joe Isaacs" <joe.isaacs@live.co.uk>
This was referenced Sep 25, 2026
Merging this PR will not alter performance
|
joseph-isaacs
added this pull request to stack #10050
September 25, 2026 12:50
I, Joe Isaacs <joe.isaacs@live.co.uk>, hereby add my Signed-off-by to this commit: f82b9c0 Format the new planning modules with the CI-pinned nightly formatter. This corrects the formatting check and the quoted sign-off on the original commit without rewriting history. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
joseph-isaacs
marked this pull request as ready for review
October 5, 2026 10:10
Require Send live planners and morsels, publish IO requests once, and construct successor planners directly. Add aligned byte requests, IO sessions and completion polling, and the blocking reader adapter. Update the protocol and adapter tests for these contracts. Signed-off-by: "Joe Isaacs" <joe.isaacs@live.co.uk> Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
robert3005
approved these changes
Oct 5, 2026
robert3005
left a comment
Contributor
There was a problem hiding this comment.
Let's go with this and tune as we learn more
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce composable scan planning through explicit IO and CPU state machines. Live
PlannerandMorselimplementations areSendand can move between threads with their owning run.Next<Input>constructs a live successor directly; constructors store inputs, while substantial work belongs incompute().Requests are published exactly once through
compute()returningNeedsIO(batch).state()reportsDone,NeedsCompute, orWaiting; the driver tracks outstanding fetches, and waiting without one is a protocol error.IoSlotexposesis_waiting()instead of returning outstanding request batches.Add the protocol under
vortex_scan::planningand byte request/result contracts undervortex_io::request. Byte ranges carry alignment,IoSourcesupports completion polling with a waker, andIoServicecreates per-scope sessions. Include a simple blockingReadAtIoSourceadapter that declines optional hints. IO owner identities remain independent of scan scopes. Existing scan execution paths are unchanged.Tests cover request bookkeeping, alignment, direct successor construction and constructor errors, and the reader adapter's registration, delivery, and cleanup behavior. No local builds, tests, lint, or formatting commands were run for this update.
First of three draft PRs: protocol and byte-source adapter → simple driver → footer opening/pruning. Data planners, IO priority/coalescing, and engine integration remain outside this PR. The later stacked PRs will need to adopt these updated contracts.
Stack order: