Skip to content

Write the list layout when the enabled editions include it - #10563

Merged
mhk197 merged 4 commits into
developfrom
mk/list-layout-from-editions
Oct 16, 2026
Merged

mhk197 merged 4 commits into
developfrom
mk/list-layout-from-editions

Conversation

@mhk197

@mhk197 mhk197 commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The default writer now writes the list layout whenever the session's enabled editions include vortex.list, so the editions are the only switch. with_list_layout() is removed, with no opt-out in its place. No default edition includes vortex.list, so default writes are unchanged.

Changes

  • WriteStrategyBuilder::from_session turns the list layout on when the enabled editions include vortex.list.
  • Removes WriteStrategyBuilder::with_list_layout().
  • vortex-file's test session enables every registered layout, so its list tests now write the list layout; the flat/list switch in write_read_roundtrip_with_layout is removed.

API Changes

Signed-off-by: Matt Katz <mhkatz97@gmail.com>
@mhk197 mhk197 added the changelog/chore A trivial change label Oct 16, 2026
@mhk197
mhk197 added this pull request to stack #10564 October 16, 2026 15:47
Comment thread vortex-file/src/strategy.rs Outdated
@codspeed

codspeed Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Merging this PR will regress 7 benchmarks

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ 19 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

ℹ️ 40 benchmarks have no flame graph

The flamegraph could not be generated because we failed to resolve the symbols in the profiling trace. This happens when we fail to harvest information about language frames.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 46 improved benchmarks
❌ 7 regressed benchmarks
✅ 2334 untouched benchmarks
⏩ 12 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ ℹ️ Simulation test_scan_filter_to_arrow_array[int_float] 3.5 ms 4.1 ms -13.51%
❌ WallTime words_gather_dispatch_avx512[65536] 896 ns 1,021 ns -12.24%
❌ WallTime bitpack_blocked_compress_avx2 6.7 µs 7.6 µs -12.19%
❌ ℹ️ Simulation test_repeated_scan_filter[int_float] 3.4 ms 3.9 ms -10.93%
❌ ℹ️ Simulation test_repeated_scan_filter[int] 2.5 ms 2.7 ms -10.35%
❌ WallTime threshold_neon[8] 1.4 µs 1.5 µs -10.05%
❌ ℹ️ Simulation test_scan_filter_to_arrow_array[int] 2.5 ms 2.8 ms -10.04%
⚡ Simulation embedding_code_runs[4096] 1,940.1 µs 160 µs ×12
⚡ Simulation embedding_code_runs[2048] 997.1 µs 108.2 µs ×9.2
⚡ Simulation embedding_code_runs[1024] 534.9 µs 82.7 µs ×6.5
⚡ Simulation embedding_code_runs[512] 306 µs 74.1 µs ×4.1
⚡ Simulation isolated_runs[u8, (512, 0)] 304.7 µs 74.2 µs ×4.1
⚡ Simulation embedding_runs[1536] 890.8 µs 236 µs ×3.8
⚡ Simulation embedding_runs[1024] 605.3 µs 168.2 µs ×3.6
⚡ Simulation fragmented_runs[1024] 605.2 µs 168.6 µs ×3.6
⚡ Simulation isolated_runs[u16, (512, 0)] 322.1 µs 95 µs ×3.4
⚡ Simulation embedding_runs[768] 498.2 µs 170.3 µs ×2.9
⚡ Simulation isolated_runs[u8, (256, 0)] 190.5 µs 70.7 µs ×2.7
⚡ Simulation isolated_runs[u8, (256, 1020)] 193.9 µs 73.3 µs ×2.6
⚡ Simulation isolated_runs[u32, (512, 0)] 353.5 µs 135.8 µs ×2.6
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing mk/list-layout-from-editions (2d1339f) with develop (06adefd)2

Open in CodSpeed

Footnotes

  1. 12 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on develop (8a1f2cb) during the generation of this report, so 06adefd was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
@mhk197
mhk197 marked this pull request as ready for review October 16, 2026 16:06
@mhk197
mhk197 requested a review from robert3005 October 16, 2026 16:12
@mhk197
mhk197 merged commit a805d3e into develop Oct 16, 2026
96 of 97 checks passed
@mhk197
mhk197 deleted the mk/list-layout-from-editions branch October 16, 2026 16:23
joseph-isaacs added a commit that referenced this pull request Oct 16, 2026
CI run 38936290519 failed before timing: rewrite_list_layout still called
WriteStrategyBuilder::with_list_layout(), removed by upstream #10563 and
inherited through the merged #10540 base. Keep the base's edition-driven
writer policy and adapt the benchmark helper instead of restoring the API.

Enable the experimental list edition only for list/list-elements modes.
Flat mode leaves it disabled, preserving the flat control. Drop the obsolete
environment guard and explicit writer toggle. Scanner runtime is unchanged,
so the completed local 48-case merged-comparison remains applicable.

Validation: built both release_debug examples with --locked; rewrote all
1,000,000 rows in flat, list, list-elements/1 MiB and list-elements/64 KiB
modes; all four full Arrow equality and layout-kind checks passed, as did
four V2 selection comparisons against full V1 scans followed by take.
Pinned formatting and diff whitespace checks passed. No new Rust-library
checks were needed for this example-only API adaptation.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants