perf(array): forward probe_scalar through pass-through encodings

[joseph-isaacs]

Summary

Eleven encodings that cost almost nothing per row themselves were still serving reads from the default probe_scalar, which forwards to scalar_at and reads children with a one-off execute_scalar. Preparation retained below them was rebuilt on every row, so an encoding that keeps decoded state only paid off when it sat at the root of a tree. A column stored as Dict(Pco) or Chunked(Pco) threw the decoded page away between reads; the same PCO array read directly kept it.

Each of Slice, Dict, Chunked, Shared, Masked, Extension, FoR, ZigZag, ALP, FSST and DateTimeParts now implements probe_scalar once and reads its children through state.slot(..), with scalar_at delegating via ProbeState::once so one-off and repeated reads share a body. Most use only the retained child slots; Shared and FSST retain a probe over their dynamically resolved child array.

This composes with #9844: this PR carries retention across a parent, while #9844 gives the children decoded state worth retaining.

Changes

• vortex-array: Slice, Dict, Chunked, Shared, Masked and Extension read their children through the probe's slots. Chunked resolves the chunk first and probes only that slot, so a probe walking one chunk keeps that chunk's preparation and never touches the others.
• encodings: FoR, ZigZag, ALP, FSST and DateTimeParts likewise. FoR probes both encoded values and per-chunk references, accounting for slice offsets. DateTimeParts reads its three parts through one helper.
• ALP, FSST, Masked and DateTimeParts check nullness through ProbeState::is_valid, as required by the current probe API.
• Regression tests cover masked nulls and retained FoR reads across sliced chunk boundaries.
• Keep the explicit scalar_at adapters during migration: the default probe_scalar still calls scalar_at, so adding the reverse default would allow implementations with neither method to recurse indefinitely.
• Two need more than a slot read. Shared resolves to either its source or its computed cache, and the swap can happen between reads, so its state holds a probe and rebuilds it when the array it was built over is no longer current. FSST rebuilds its codes VarBinArray from a buffer and the offsets slot on every read, so its state holds one probe over that array instead.
• ALP's patched path still reads through Patches::get_patched; only the unpatched read, the common one, goes through the probe. Worth a follow-up alongside the other Patches callers.
• Tests: test_repeated_probe_consistency in the conformance suite asserts a retained probe agrees with one-off reads over a backwards, then forwards, then sparse visit order, so a cache that outlives its row or a slot read through the wrong index fails there. It runs for every encoding already wired into test_array_consistency, which covers these eleven and guards future migrations.

Historical measurements (before the develop rebase)

The same PCO array wrapped in one parent, 65,536 rows, 1024-value pages, 64 random lookups per pass, release build on a 4-core x86_64 cloud box. Both columns have #9844 merged in, so the only difference is this PR:

array	#9844 only	with this PR
pco	13.3x	12.6x
slice(pco)	1.0x	16.6x
dict(pco)	1.0x	10.1x
chunked(pco)	1.0x	14.1x
runend(pco)	6.6x	6.4x

Each figure is a one-off read divided by a repeated-probe read, clustered lookups inside a four-page window. The pco and runend(pco) rows are unwrapped controls and do not move; the three wrapped rows go from no benefit at all to within reach of the unwrapped array. Scattered lookups over the whole array show the same shape at smaller ratios, since they rarely hit a page twice whatever retention is in place.

On the original develop baseline this change was flat: only Primitive and Struct implemented probe_scalar there, and neither had state worth keeping. What it buys today is that a child's validity resolution is no longer redone per row; what it unlocks is every encoding that retains decoded state.

Original implementation validation: cargo test for vortex-array, vortex-fsst, vortex-alp, vortex-zigzag, vortex-fastlanes and vortex-datetime-parts (4402 tests, 0 failures), and cargo fmt with the pinned nightly. Workspace clippy was not run.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Ui7E5mtZLgh4j41ugrLXbS

---

Generated by Claude Code

Current revision: rebased onto 48162f0343, with null-handling and chunked FoR compatibility fixes. Local checks were not rerun; GitHub CI validates this revision.

[codspeed]

Merging this PR will regress 5 benchmarks

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 6 improved benchmarks
❌ 5 regressed benchmarks
✅ 2092 untouched benchmarks
🆕 2 new benchmarks
⏩ 503 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	WallTime	mul_u64_nonnull_neon	29.1 µs	39.5 µs	-26.49%
❌	WallTime	mul_i64_nonnull_neon	33.1 µs	38.8 µs	-14.66%
❌	WallTime	multiply_shapes_neon[(32768, PerRowPerRow)]	33.1 µs	38.7 µs	-14.59%
❌	WallTime	dict_canonicalize_gt_u8_neon[1000000]	488.3 µs	547.6 µs	-10.84%
❌	WallTime	dict_canonicalize_gt_u8_avx2[1000000]	422.2 µs	472.1 µs	-10.58%
⚡	Simulation	once[dict]	1,118 µs	492.2 µs	×2.3
⚡	Simulation	all_valid_exclusive[65536]	6.9 ms	3.5 ms	+95.38%
⚡	Simulation	all_valid_exclusive[4096]	438.3 µs	231 µs	+89.74%
⚡	Simulation	nullable_exclusive[4096]	350.5 µs	192.5 µs	+82.05%
⚡	Simulation	nullable_exclusive[65536]	4.9 ms	2.8 ms	+76.61%
⚡	Simulation	repeated[dict]	784.3 µs	521.6 µs	+50.37%
🆕	Simulation	outlined_nullable_exclusive[4096]	N/A	368.1 µs	N/A
🆕	Simulation	outlined_nullable_exclusive[65536]	N/A	5.2 ms	N/A

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing ji/epic-euler-88rquc (4394b23) with develop (48162f0)²}

Footnotes

1. 503 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on develop (8b7a257) during the generation of this report, so 48162f0 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[robert3005]

any idea how we can default this?

[joseph-isaacs]

we cannot have to do this one-by-one until all valid check are gone

[robert3005]

I think you need to rebase