Repository navigation
Perform validation and null view replacement in two passes - #10148
Conversation
Signed-off-by: Robert Kruszewski <github@robertk.io>
Merging this PR will regress 2 benchmarks
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | WallTime | scalar_subtract_neon |
12.2 µs | 14 µs | -13.13% |
| ❌ | WallTime | dict_canonicalize_gt_u8_neon[1000000] |
493.5 µs | 562.1 µs | -12.2% |
| ⚡ | Simulation | all_valid_exclusive[65536] |
6.9 ms | 3.5 ms | +95.37% |
| ⚡ | Simulation | all_valid_exclusive[4096] |
438.2 µs | 231.1 µs | +89.6% |
| ⚡ | Simulation | nullable_exclusive[4096] |
350.3 µs | 192.8 µs | +81.67% |
| ⚡ | Simulation | nullable_exclusive[65536] |
4.9 ms | 2.8 ms | +76.58% |
| 🆕 | Simulation | outlined_nullable_exclusive[4096] |
N/A | 368.6 µs | N/A |
| 🆕 | Simulation | outlined_nullable_exclusive[65536] |
N/A | 5.2 ms | N/A |
| Simulation | bench_compare_sliced_dict_primitive[(3333, 10000)] |
79.2 µs | < 1 ns | N/A |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing rk/validate (cb9d840) with develop (9a08b82)
Footnotes
-
503 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
…n UTF-8 per buffer Raise the VarBinView whole-buffer UTF-8 budget from a fixed cost per valid view to that plus the bytes the valid outlined views reference, which is what checking the views one by one would cost. Long strings now take the whole-buffer path instead of falling back to per-view checks. Port the VarBin fast path from #10271: when the offsets never decrease and the referenced byte range is valid UTF-8, check only that every offset falls on a char boundary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011fgXLsky3Vu1DGW2H1MQQT Signed-off-by: Claude <noreply@anthropic.com>
…icking The per-string UTF-8 loop sliced the bytes with each offset pair, so offsets that decrease, or that point past the end of the bytes before the last offset, panicked instead of returning an error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011fgXLsky3Vu1DGW2H1MQQT Signed-off-by: Claude <noreply@anthropic.com>
Covers short, long, multibyte and sliced strings at several null densities, to compare whole-buffer and per-string UTF-8 validation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011fgXLsky3Vu1DGW2H1MQQT Signed-off-by: Claude <noreply@anthropic.com>
…s from the budget When the views are owned and the validity has nulls, validate the valid views and replace the null views in one pass. The second pass over many short null runs cost more than a branch per view. Go back to a fixed per-view budget for the whole-buffer UTF-8 check. Counting the referenced bytes sent long strings to the whole-buffer check, which is slower for them: the call per view is cheap next to the string, and the char boundary checks read a buffer larger than the cache a second time. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011fgXLsky3Vu1DGW2H1MQQT Signed-off-by: Claude <noreply@anthropic.com>
Drop the separate benchmark file and add a single outlined, nullable case to varbinview_try_new, which covers the whole-buffer UTF-8 check and the fused null replacement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011fgXLsky3Vu1DGW2H1MQQT Signed-off-by: Claude <noreply@anthropic.com>
When validating VarBinViewArray first validate the buffers and inline strings
and only later iterate nulls and replace them with sentinel values