Repository navigation
perf(layout): lower zoned pruning predicates once - #10428
Merged
Merged
Conversation
The zoned pruning result lowered its stats predicate to the zone map's stats table each time it computed a mask. With dynamic expressions, the predicate is recomputed on each version change, so it was lowered again each time. Lower the predicate once in the shared pruning future and keep the lowered form in `PruningResult`. `ZoneMap::prune` keeps its behaviour for external callers. Ported from langchain-ai/vortex 5f3f68d by Neville Li. The NaN-stat shortcut for non-float columns is not ported, because the stats rewrite no longer emits NaN checks for non-float inputs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Robert Kruszewski <github@robertk.io>
Merging this PR will degrade performance by 35.31%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | decode_primitives[i64, (1000, 32)] |
25 µs | 38.6 µs | -35.31% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing rk/zoned-prepare-pruning-predicate (ba16698) with develop (9c7b782)
Footnotes
-
184 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
joseph-isaacs
approved these changes
Oct 12, 2026
joseph-isaacs
added
the
action/bench-sql
github-actions
Bot
removed
the
action/bench-sql
Contributor
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
Contributor
Benchmarks: TPC-H SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.006x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.000x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.001x ➖, 1↑ 0↓)
duckdb / parquet / ns (1.005x ➖, 0↑ 0↓)
No file size changes detected. |
The zoned reader lowers a pruning predicate once and evaluates the lowered form again on each dynamic update. Add a test that changes the dynamic value after lowering and checks that the pruning mask follows it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Robert Kruszewski <github@robertk.io>
Rename `PruningResult::predicate` to `lowered_predicate`, so the name tells that the predicate is already lowered to the stats table, and remove the comment that the new name makes unnecessary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Robert Kruszewski <github@robertk.io>
Contributor
Benchmarks: PolarSignals Profiling 📖Commits: PR datafusion / vortex-file-compressed / ns (1.010x ➖, 0↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: FineWeb NVMe 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.022x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.993x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.029x ➖, 1↑ 2↓)
duckdb / parquet / ns (1.011x ➖, 0↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: TPC-DS SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.999x ➖, 1↑ 1↓)
datafusion / parquet / ns (0.996x ➖, 1↑ 0↓)
duckdb / vortex-file-compressed / ns (1.008x ➖, 1↑ 3↓)
duckdb / parquet / ns (1.009x ➖, 1↑ 2↓)
No file size changes detected. |
Contributor
Benchmarks: Clickbench on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.002x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.005x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.987x ➖, 4↑ 1↓)
duckdb / parquet / ns (1.003x ➖, 0↑ 1↓)
No file size changes detected. |
Contributor
Benchmarks: TPC-H SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.003x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.001x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.007x ➖, 0↑ 1↓)
duckdb / parquet / ns (0.986x ➖, 1↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: TPC-H SF=1 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.977x ➖, 1↑ 2↓)
datafusion / parquet / ns (0.982x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.035x ➖, 0↑ 3↓)
duckdb / parquet / ns (1.025x ➖, 0↑ 0↓)
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Only compute lowered stats predicate once. Avoids relowering on every dynamic
version update
Signed-off-by: Robert Kruszewski github@robertk.io