Skip to content

refactor(array): give canonical decimals one integer child - #10230

Draft
connortsui20 wants to merge 19 commits into
ct/narrow-compressionfrom
ct/decimal-integer-child
Draft

connortsui20 wants to merge 19 commits into
ct/narrow-compressionfrom
ct/decimal-integer-child

Conversation

@connortsui20

@connortsui20 connortsui20 commented Oct 2, 2026 •

Copy link
Copy Markdown
Member

Stacked on #10333. This experimental draft remains at the top of the stack.

A decimal with precision 76 can contain values that fit in 32 bits. Giving its canonical representation a plain 256-bit integer buffer would expand the value buffer eightfold before any computation.

Decimal precision now determines the logical dtype of one signed integer child. Narrow retains smaller stored values and encoded children during canonicalization, so selection, comparison, and supported aggregates can work through the child. Consumers that need native buffers explicitly materialize values at the stored width.

The experiment adds signed 128-bit and 256-bit integer extensions and retains the existing Decimal wire representation.

@connortsui20
connortsui20 added this pull request to stack #10231 October 2, 2026 18:47
@connortsui20 connortsui20 added the changelog/break A breaking API change label Oct 2, 2026
@connortsui20
connortsui20 force-pushed the ct/decimal-integer-child branch from bfbb921 to c55281d Compare October 2, 2026 20:22
@codspeed

codspeed Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Merging this PR will regress 3 benchmarks

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 21 improved benchmarks
❌ 3 regressed benchmarks
✅ 2122 untouched benchmarks
⏩ 476 skipped benchmarks1
🗄️ 1 archived benchmark run2

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Simulation random_i128[0.01] 9.6 µs 23.1 µs -58.38%
❌ Simulation patterns_i128[Contiguous] 20.2 µs 30.2 µs -33.07%
❌ Simulation in_place_non_nullable[16384] 37.9 µs 49.3 µs -23.06%
⚡ Simulation random_i128[0.8] 177.3 µs 22.5 µs ×7.9
⚡ Simulation random_i128[0.95] 161.9 µs 22.6 µs ×7.2
⚡ Simulation random_i256[0.95] 127 µs 22.6 µs ×5.6
⚡ Simulation random_i256[0.8] 125.8 µs 22.6 µs ×5.6
⚡ Simulation random_i256[0.5] 122.1 µs 22.5 µs ×5.4
⚡ Simulation patterns_i128[Random] 119.2 µs 22.5 µs ×5.3
⚡ Simulation random_i128[0.5] 119.2 µs 22.6 µs ×5.3
⚡ Simulation patterns_i128[Runs] 119.8 µs 23.6 µs ×5.1
⚡ Simulation cached_indices_i128[0.1] 41.2 µs 21.1 µs +95.8%
⚡ Simulation random_i256[0.01] 39.2 µs 23.2 µs +68.74%
⚡ Simulation decompress[u32, (1000, 256)] 54.9 µs 32.9 µs +66.97%
⚡ Simulation cached_indices_i128[0.01] 46.5 µs 30.8 µs +50.64%
⚡ Simulation cached_indices_i32[0.1] 102.6 µs 78.5 µs +30.68%
⚡ Simulation take_map[(0.1, 1.0)] 289.3 µs 221.7 µs +30.52%
⚡ Simulation in_place_nullable[16384] 32.7 µs 26.3 µs +24.53%
⚡ Simulation random_i64[0.8] 167 µs 136.9 µs +21.94%
⚡ Simulation random_i64[0.95] 174.4 µs 143.9 µs +21.21%
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ct/decimal-integer-child (173be79) with ct/narrow-compression (71f9458)

Open in CodSpeed

Footnotes

  1. 476 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. 1 benchmark was run, but is now archived. If it was deleted in another branch, consider rebasing to remove it from the report. Instead if it was added back, click here to restore it. ↩

@connortsui20
connortsui20 force-pushed the ct/decimal-integer-child branch 3 times, most recently from 54c5826 to cce61ed Compare October 2, 2026 21:48
@connortsui20
connortsui20 removed this pull request from stack #10231 October 5, 2026 10:33
@connortsui20
connortsui20 force-pushed the ct/decimal-integer-child branch from cce61ed to e29d26c Compare October 5, 2026 10:34
@connortsui20
connortsui20 added this pull request to stack #10300 October 5, 2026 10:34
@connortsui20
connortsui20 force-pushed the ct/decimal-integer-child branch from e29d26c to 7571adf Compare October 5, 2026 10:53
@connortsui20
connortsui20 force-pushed the ct/decimal-integer-child branch 2 times, most recently from c21b7df to c0d97fe Compare October 5, 2026 11:12
@connortsui20
connortsui20 removed this pull request from stack #10300 October 5, 2026 15:21
@connortsui20
connortsui20 force-pushed the ct/decimal-integer-child branch from c0d97fe to 4931e50 Compare October 5, 2026 15:26
@connortsui20
connortsui20 added this pull request to stack #10307 October 5, 2026 15:26
@connortsui20
connortsui20 removed this pull request from stack #10307 October 6, 2026 13:09
@connortsui20
connortsui20 force-pushed the ct/decimal-integer-child branch from 4931e50 to 76afa79 Compare October 6, 2026 13:09
@connortsui20
connortsui20 changed the base branch from ct/narrow-encoding to ct/narrow-compression October 6, 2026 13:10
@connortsui20
connortsui20 added this pull request to stack #10334 October 6, 2026 13:10
@connortsui20
connortsui20 force-pushed the ct/decimal-integer-child branch from 76afa79 to d936b02 Compare October 6, 2026 13:27
@connortsui20
connortsui20 removed this pull request from stack #10334 October 6, 2026 13:44
@connortsui20
connortsui20 force-pushed the ct/decimal-integer-child branch from d936b02 to 55b8621 Compare October 6, 2026 13:44
@connortsui20
connortsui20 added this pull request to stack #10335 October 6, 2026 13:44
@connortsui20
connortsui20 removed this pull request from stack #10335 October 6, 2026 14:12
@connortsui20
connortsui20 force-pushed the ct/decimal-integer-child branch from 55b8621 to cc5e3a0 Compare October 6, 2026 14:15
@connortsui20
connortsui20 force-pushed the ct/decimal-integer-child branch from cc5e3a0 to b82e21b Compare October 6, 2026 14:16
@connortsui20
connortsui20 added this pull request to stack #10336 October 6, 2026 14:17
@connortsui20
connortsui20 removed this pull request from stack #10336 October 6, 2026 14:18
Decimal precision selects the logical integer width. Narrow keeps smaller stored values and encoded children through canonicalization, with native buffer consumers decoding only when needed.

Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
@connortsui20
connortsui20 force-pushed the ct/decimal-integer-child branch from b82e21b to 1b342f3 Compare October 6, 2026 14:21
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
@connortsui20
connortsui20 added this pull request to stack #10337 October 6, 2026 14:23

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/break A breaking API change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant