Composable footer opening and file pruning - #10049
joseph-isaacs wants to merge 1 commit into
Conversation
Add separate FooterOpen and FooterPrune stages and a caller-supplied continuation. Exercise real footer IO and conservative file-statistics pruning with a diagnostic range morsel confined to test support. Keep existing file and scan entry points unchanged. Signed-off-by: "Joe Isaacs" <joe.isaacs@live.co.uk> Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
d704c9c to
de8be02
Compare
Merging this PR will degrade performance by 10.75%
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| ❌ | bitpack_blocked_compress_avx2 |
6.8 µs | 7.6 µs | -10.75% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing ji/scan-protocol-footer (de8be02) with ji/scan-protocol-driver (493b7d9)
Footnotes
-
2403 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
Add
FooterOpenandFooterPruneundervortex_file::planning. The first stage reads and validates a footer through the planning IO protocol; the second rejects empty files and files whose statistics prove the predicate cannot match. Surviving files pass to a caller-suppliedNext<OpenedFile>.The stages are separate from existing file-opening and scan implementations and use the file crate's private helpers without widening their visibility. End-to-end tests finish with a diagnostic
{start, end}morsel that exists only in test support; this PR does not evaluate file data or switch public scan entry points.Tests cover cached and large footers, known/unknown size, stable requests, failures, conservative pruning with missing statistics, and footer-only reads. Builds, tests, formatting, and lint checks have not been run.
Third of three draft PRs: traits → simple driver → footer opening/pruning. Based on
ji/scan-protocol-driver. Further planners and production scheduling are follow-on work.Stack order: