Skip to content

Move BitPacked serde logic from VTable to Plugin - #9937

Merged
mhk197 merged 4 commits into
developfrom
mk/bitpacked-stack-01-wire-boundary
Sep 30, 2026
Merged

mhk197 merged 4 commits into
developfrom
mk/bitpacked-stack-01-wire-boundary

Conversation

@mhk197

@mhk197 mhk197 commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Move metadata, serialization and deserialization of fastlanes.bitpacked into BitPackedPlugin. The VTable retains only the required error-returning serde methods and in-memory array validation. BitPackedPlugin is registered on the default session instead of the vtable-derived plugin by default.

Users must now register BitPackedPlugin instead of BitPacked on session.

This refactor is made in preparation for introducing a new version of BitPackedArray

@codspeed

codspeed Bot commented Sep 18, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ 1 benchmark measured no execution time

Nothing ran under measurement, usually because the compiler removed the code under test. This result is not comparable, so it counts as unchanged.

Preventing compiler optimizations

✅ 2085 untouched benchmarks
⏩ 503 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚠️ Simulation bench_compare_sliced_dict_primitive[(3333, 10000)] < 1 ns < 1 ns N/A

Comparing mk/bitpacked-stack-01-wire-boundary (0cd02cd) with develop (4224390)2

Open in CodSpeed

Footnotes

  1. 503 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on develop (0751940) during the generation of this report, so 4224390 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@mhk197
mhk197 added this pull request to stack #9946 September 18, 2026 19:27
@mhk197
mhk197 force-pushed the mk/bitpacked-stack-01-wire-boundary branch from 950de57 to 9681199 Compare September 18, 2026 19:42
@mhk197 mhk197 changed the title refactor: isolate bitpacked v1 serialization refactor: isolate bitpacked v1 serde in the plugin Sep 18, 2026
@mhk197
mhk197 deployed to duckdb-build September 18, 2026 19:42 — with GitHub Actions Active
@mhk197 mhk197 added the changelog/break A breaking API change label Sep 23, 2026
@mhk197
mhk197 force-pushed the mk/bitpacked-stack-01-wire-boundary branch from 349b196 to 04fa22c Compare September 23, 2026 17:31
@mhk197
mhk197 marked this pull request as ready for review September 23, 2026 17:37
@mhk197 mhk197 changed the title refactor: isolate bitpacked v1 serde in the plugin Move BitPacked serde logic from VTable to Plugin Sep 23, 2026
@mhk197
mhk197 removed this pull request from stack #9946 September 23, 2026 18:19
@mhk197
mhk197 added this pull request to stack #10006 September 23, 2026 18:22
@mhk197
mhk197 removed this pull request from stack #10006 September 23, 2026 19:43
@mhk197
mhk197 added this pull request to stack #10012 September 23, 2026 19:43
mhk197 added a commit that referenced this pull request Sep 28, 2026
First PR of a stack prototyping a FoR encoding with one reference per
1024-element chunk, following the plugin approach used for
`DecimalByteParts` and #9937.

## Summary

Move metadata, serialization and deserialization of `fastlanes.for` into
a new `FoRPlugin`. The VTable keeps only the required serde methods,
which now return an error, plus in-memory validation.
`vortex_fastlanes::initialize` registers `FoRPlugin` instead of the
vtable-derived plugin.

The wire format is unchanged: the metadata is still just the reference
`ScalarValue` proto bytes, with a single `encoded` child.

Users who register `FoR` directly on a session must now register
`FoRPlugin` to get serde.

---------

Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Signed-off-by: "Matt Katz" <mhkatz97@gmail.com>

Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Signed-off-by: "Matt Katz" <mhkatz97@gmail.com>
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
@mhk197
mhk197 force-pushed the mk/bitpacked-stack-01-wire-boundary branch from 04fa22c to 314bd32 Compare September 30, 2026 19:16
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
@mhk197
mhk197 merged commit 0675f59 into develop Sep 30, 2026
121 of 123 checks passed
@mhk197
mhk197 deleted the mk/bitpacked-stack-01-wire-boundary branch September 30, 2026 19:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/break A breaking API change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants