Skip to content

feat(array): min/max for fixed-size lists, so UUID columns get zone-map stats - #10457

Open
XiangpengHao wants to merge 3 commits into
vortex-data:developfrom
XiangpengHao:xiangpeng/fsl-min-max
Open

XiangpengHao wants to merge 3 commits into
vortex-data:developfrom
XiangpengHao:xiangpeng/fsl-min-max

Conversation

@XiangpengHao

Copy link
Copy Markdown
Contributor

Summary

vortex.uuid is stored as FixedSizeList(U8, 16), and the min/max aggregate skips fixed-size lists as a stop-gap (#8243, for #8189). UUID zone maps and file footers therefore keep only null counts, and uuid_col = x prunes nothing, which hurts sorted UUIDv7 keys most. This implements min/max for fixed-size lists, so UUID columns get the same zone-map and footer statistics, and the same pruning, as Parquet's UUID logical type.

Changes

  • min/max for FixedSizeList (aggregate_fn/fns/min_max/fixed_size_list.rs): lists of fixed-width elements (bool, primitive, decimal), ordered lexicographically by the compare kernel's own row comparator (compare/nested.rs), so statistics and comparisons always agree. Null lists are skipped, null elements order first, floats use total order. Non-nullable u8 lists (the UUID case) compare as byte slices, the same order as Postgres, DuckDB and Parquet.
  • Size cap: lists wider than MAX_FIXED_SIZE_LIST_MIN_MAX_BYTES (64 bytes per value) get no min/max, so embedding-style columns don't grow zone maps for pruning they can't use. The result is the same for every encoding.
  • vortex.uuid gets min/max through its storage; no UUID-specific code.
  • DuckDB COPY … RETURN_STATS converts UUID bounds to UUID and fixed-size-list bounds to ARRAY instead of failing on them; the JNI writer converts them to java.util.UUID and List.
  • Tests: vortex/tests/uuid_pruning.rs (a UUID column over four zones; uuid = x keeps only the zone that can hold it, footer min/max, can_prune), fixed_size_list::tests (ordering vs Scalar::partial_cmp and compare, nulls, floats, sliced and dictionary inputs, the cap at its boundary), vortex-uuid min/max order, DuckDB conversion tests and copy_return_stats.slt. JNIWriterTest.testUuidAndFixedSizeListBounds is added but was not run locally (no JDK).

API Changes

Files written with this change carry min/max for UUID and narrow fixed-size-list columns. Engines that receive file statistics (DuckDB RETURN_STATS, the JNI writer's column statistics) now see bounds for those columns.

…ap stats

`vortex.uuid` is stored as `FixedSizeList(U8, 16)`, and the min/max
aggregate skipped fixed-size lists as a stop-gap (vortex-data#8243), so UUID zone
maps and footers kept only null counts and `uuid = x` pruned nothing.

Compute min/max for fixed-size lists of fixed-width elements (bool,
primitive, decimal), ordered by the compare kernel's own row comparator
so statistics and comparisons always agree. Non-nullable byte lists such
as UUIDs compare as byte slices. Lists wider than 64 bytes per value get
none, so embedding-style columns do not grow zone maps.

DuckDB's COPY RETURN_STATS and the JNI writer now convert UUID and
fixed-size-list bounds instead of failing on them.

Signed-off-by: XiangpengHao <xiangpeng@spiraldb.com>
@XiangpengHao XiangpengHao added the changelog/feature A new feature label Oct 12, 2026
Signed-off-by: XiangpengHao <xiangpeng@spiraldb.com>
@codspeed

codspeed Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Merging this PR will degrade performance by 24.52%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ 1 benchmark measured no execution time

Nothing ran under measurement, usually because the compiler removed the code under test. This result is not comparable, so it counts as unchanged.

Preventing compiler optimizations

⚠️ 19 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 2 regressed benchmarks
✅ 2211 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Simulation decode_primitives[i64, (1000, 32)] 24.7 µs 38 µs -35.15%
❌ WallTime bitpack_blocked_compress_avx2 6.7 µs 7.6 µs -12.14%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing XiangpengHao:xiangpeng/fsl-min-max (5b9c1d0) with develop (fdba755)

Open in CodSpeed

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant