Repository navigation
Write the list layout when the enabled editions include it - #10563
Conversation
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Merging this PR will regress 7 benchmarks
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ ℹ️ | Simulation | test_scan_filter_to_arrow_array[int_float] |
3.5 ms | 4.1 ms | -13.51% |
| ❌ | WallTime | words_gather_dispatch_avx512[65536] |
896 ns | 1,021 ns | -12.24% |
| ❌ | WallTime | bitpack_blocked_compress_avx2 |
6.7 µs | 7.6 µs | -12.19% |
| ❌ ℹ️ | Simulation | test_repeated_scan_filter[int_float] |
3.4 ms | 3.9 ms | -10.93% |
| ❌ ℹ️ | Simulation | test_repeated_scan_filter[int] |
2.5 ms | 2.7 ms | -10.35% |
| ❌ | WallTime | threshold_neon[8] |
1.4 µs | 1.5 µs | -10.05% |
| ❌ ℹ️ | Simulation | test_scan_filter_to_arrow_array[int] |
2.5 ms | 2.8 ms | -10.04% |
| ⚡ | Simulation | embedding_code_runs[4096] |
1,940.1 µs | 160 µs | ×12 |
| ⚡ | Simulation | embedding_code_runs[2048] |
997.1 µs | 108.2 µs | ×9.2 |
| ⚡ | Simulation | embedding_code_runs[1024] |
534.9 µs | 82.7 µs | ×6.5 |
| ⚡ | Simulation | embedding_code_runs[512] |
306 µs | 74.1 µs | ×4.1 |
| ⚡ | Simulation | isolated_runs[u8, (512, 0)] |
304.7 µs | 74.2 µs | ×4.1 |
| ⚡ | Simulation | embedding_runs[1536] |
890.8 µs | 236 µs | ×3.8 |
| ⚡ | Simulation | embedding_runs[1024] |
605.3 µs | 168.2 µs | ×3.6 |
| ⚡ | Simulation | fragmented_runs[1024] |
605.2 µs | 168.6 µs | ×3.6 |
| ⚡ | Simulation | isolated_runs[u16, (512, 0)] |
322.1 µs | 95 µs | ×3.4 |
| ⚡ | Simulation | embedding_runs[768] |
498.2 µs | 170.3 µs | ×2.9 |
| ⚡ | Simulation | isolated_runs[u8, (256, 0)] |
190.5 µs | 70.7 µs | ×2.7 |
| ⚡ | Simulation | isolated_runs[u8, (256, 1020)] |
193.9 µs | 73.3 µs | ×2.6 |
| ⚡ | Simulation | isolated_runs[u32, (512, 0)] |
353.5 µs | 135.8 µs | ×2.6 |
| ... | ... | ... | ... | ... | ... |
ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing mk/list-layout-from-editions (2d1339f) with develop (06adefd)2
Footnotes
-
12 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
-
No successful run was found on
develop(8a1f2cb) during the generation of this report, so 06adefd was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
CI run 38936290519 failed before timing: rewrite_list_layout still called WriteStrategyBuilder::with_list_layout(), removed by upstream #10563 and inherited through the merged #10540 base. Keep the base's edition-driven writer policy and adapt the benchmark helper instead of restoring the API. Enable the experimental list edition only for list/list-elements modes. Flat mode leaves it disabled, preserving the flat control. Drop the obsolete environment guard and explicit writer toggle. Scanner runtime is unchanged, so the completed local 48-case merged-comparison remains applicable. Validation: built both release_debug examples with --locked; rewrote all 1,000,000 rows in flat, list, list-elements/1 MiB and list-elements/64 KiB modes; all four full Arrow equality and layout-kind checks passed, as did four V2 selection comparisons against full V1 scans followed by take. Pinned formatting and diff whitespace checks passed. No new Rust-library checks were needed for this example-only API adaptation. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Summary
The default writer now writes the list layout whenever the session's enabled editions include
vortex.list, so the editions are the only switch.with_list_layout()is removed, with no opt-out in its place. No default edition includesvortex.list, so default writes are unchanged.Changes
WriteStrategyBuilder::from_sessionturns the list layout on when the enabled editions includevortex.list.WriteStrategyBuilder::with_list_layout().write_read_roundtrip_with_layoutis removed.API Changes
WriteStrategyBuilder::with_list_layout()(added in Make the list layout opt-in and edition-gated #10529, unreleased).