Skip to content

Move array serde from VTable to standalone ArrayPlugin impls - #10111

Draft
mhk197 wants to merge 2 commits into
developfrom
mk/standalone-array-plugins
Draft

mhk197 wants to merge 2 commits into
developfrom
mk/standalone-array-plugins

Conversation

@mhk197

@mhk197 mhk197 commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Remove the blanket impl<V: VTable> ArrayPlugin for V.
  • Remove serialize / deserialize from the VTable trait. The vtable now only covers in-memory behaviour.
  • Every serializable encoding implements ArrayPlugin directly on its vtable type (impl ArrayPlugin for Primitive, impl ArrayPlugin for FoR, ...). There are no new plugin structs, and session.arrays().register(X) is unchanged.
  • Each impl lives in a plugin.rs child module of the file that holds impl VTable for X, e.g. primitive/vtable/plugin.rs or alp/array/plugin.rs. Because it is a child module, it can use the parent's private metadata types without visibility changes.
  • The old serde bodies are ported verbatim, so the wire format is unchanged.

Encodings that were never serializable lose the two methods and get no plugin: Filter, Slice, Shared, Interleave, ScalarFn, ForeignArray, TransposedBool, PiecewiseSequence and the test-harness vtables.

  • The standalone DecimalBytePartsPlugin struct is removed. Its v1/v2 serde now lives in impl ArrayPlugin for DecimalByteParts, and initialize registers DecimalByteParts. The bare_vtable_refuses_serde test is deleted because its premise no longer holds.
  • ALPPatchedPlugin and BitPackedPatchedPlugin are unchanged apart from delegating to ArrayPlugin::deserialize(&ALP/&BitPacked, ...).

Behaviour changes

  • PiecewiseSequence is no longer registered in ArraySession::default(), and TransposedBool is no longer registered by vortex_fastlanes::initialize. Neither could serialize or deserialize before. Serializing them now fails with "not registered for serialization" instead of "does not support serialization", and serialization_is_not_supported is updated to match.
  • PythonVTable gets a minimal ArrayPlugin whose serialize returns Ok(None) and whose deserialize bails, the same as before. This keeps vortex._lib.registry.register working.

Signed-off-by: Matt Katz <mhkatz97@gmail.com>
@codspeed

codspeed Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Merging this PR will degrade performance by 13.28%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
❌ 4 regressed benchmarks
✅ 2057 untouched benchmarks
⏩ 503 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Simulation density_sweep_single_slice[0.9] 32.1 µs 48.3 µs -33.57%
❌ Simulation sum_v2_i64 193.9 µs 224.7 µs -13.7%
❌ Simulation sum_i64 194 µs 224.4 µs -13.56%
❌ WallTime dict_canonicalize_gt_u8_neon[1000000] 488.7 µs 547.2 µs -10.68%
⚡ WallTime compare_u64_avx2 4.1 µs 3.7 µs +10.8%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing mk/standalone-array-plugins (e50242a) with develop (292cf6f)

Open in CodSpeed

Footnotes

  1. 503 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Signed-off-by: Matt Katz <mhkatz97@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant