You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[python] Fix MAP and MULTISET type JSON serialization - #9682
Fix MapType.to_dict() and MultisetType.to_dict() to emit canonical MAP / MULTISET tags, with NOT NULL when needed. Preserve child types, nullability, and existing legacy JSON parsing.
Validation
312 local tests passed with paimon-rust 4178002a, including MAP native reads and 12 MULTISET Rust schema checks.
MULTISET: 5 regression failures and 12 Rust schema failures before the fix; all pass afterward.
flake8, license checks, and git diff --check passed.
XiaoHongbo-Hope
changed the title
[python][ray] Rebase exact-range self-merge updates
[python] Fix MAP type JSON serialization for native schema interoperability
Sep 18, 2026
XiaoHongbo-Hope
changed the title
[python] Fix MAP type JSON serialization for native schema interoperability
[python] Fix MAP type JSON serialization
Sep 18, 2026
The MAP fix itself looks right: MAP / MAP NOT NULL matches the canonical Java tag (MapType.java:90), the parser accepts both the canonical and legacy MAP<...> forms, and the tests cover the nullability matrix.
One same-class gap is left behind: MultisetType.to_dict() (paimon-python/pypaimon/schema/data_types.py:221-227) still emits the embedded-element form {"type": "MULTISET<...>", "element": ..., "nullable": ...}. Since _resolved_schema_json serializes every field of the table schema, any table with a MULTISET column would hit the same native-plan failure this PR fixes for MAP:
Invalid table schema JSON: unknown variant `MULTISET<...>`,
expected one of `ARRAY`, `MAP`, `MULTISET`, `ROW`, `VECTOR`
The canonical tag already exists on the Java side (MultisetType.java:83 emits MULTISET / MULTISET NOT NULL), and the Python parser's MULTISET branch (data_types.py:573-578) accepts both forms, so switching the tag would be backward compatible. Could you extend the same fix and tests to MULTISET here, or file a follow-up so the native path is not left half-fixed?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
MAP and MULTISET fields can cause native planning to fall back to Python:
Fix
MapType.to_dict()andMultisetType.to_dict()to emit canonicalMAP/MULTISETtags, withNOT NULLwhen needed. Preserve child types, nullability, and existing legacy JSON parsing.Validation
4178002a, including MAP native reads and 12 MULTISET Rust schema checks.git diff --checkpassed.