Skip to content

feat(store): add key_prefix_from cursor to Query for efficient prefix scans - #108

Open
royalcala wants to merge 2 commits into
n0-computer:mainfrom
royalcala:feat/query-key-prefix-from
Open

feat(store): add key_prefix_from cursor to Query for efficient prefix scans#108
royalcala wants to merge 2 commits into
n0-computer:mainfrom
royalcala:feat/query-key-prefix-from

Conversation

@royalcala

Copy link
Copy Markdown

Summary

Adds an optional from cursor field to the Query struct that enables B-tree-level seeks during prefix scans.

Motivation

The current key_prefix filter always starts iteration from the beginning of the prefix range. For applications maintaining a cursor position (e.g., syncing event logs), this forces O(n) client-side skipping via offset. With this change, the B-tree seeks directly to the cursor position in O(log n).

Changes

  • Add from: Option<Bytes> field to Query and QueryBuilder (store.rs)
  • Add key_prefix_from(prefix, from) builder method (store.rs)
  • Pass cursor through IndexKind::KeyAuthor to ByKeyBounds::new (util.rs, bounds.rs, query.rs)
  • When from is set with a Prefix filter, the lower bound becomes the cursor position instead of the prefix start

Backward Compatibility

Fully backward-compatible. When from is None (default), behavior is identical to current key_prefix. The wire format adds an optional field that defaults to None.

Usage

// Iterate from a known cursor position
let stream = doc.get_many(
    Query::all()
        .sort_by(SortBy::KeyAuthor, SortDirection::Asc)
        .key_prefix_from("evt:", "evt:01JABC...")
        .limit(100)
).await?;

Adds an optional  field to the Query struct that serves as a cursor
position for prefix scans. When set together with a KeyFilter::Prefix,
the B-tree seek starts at the cursor position instead of the prefix start,
enabling efficient iteration over large datasets from a known position.

- Add  to Query and QueryBuilder
- Add  builder method
- Pass cursor through IndexKind::KeyAuthor to ByKeyBounds
- Use cursor as lower bound for B-tree range scan (O(log n))
- Add unit test for ByKeyBounds with cursor lower bound
- Add debug_assert that cursor starts with prefix
- Document SortBy::KeyAuthor requirement in doc comment
- Document that from is ignored on AuthorKey index path
@n0bot n0bot Bot added this to iroh Jun 15, 2026
@github-project-automation github-project-automation Bot moved this to 🚑 Needs Triage in iroh Jun 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: 🚑 Needs Triage

Development

Successfully merging this pull request may close these issues.

1 participant