Skip to content

Support v3 variant data type #666

Description

@zhjwpku
No description provided.

Activity

  1. self-assigned this
    on May 21, 2026
  2. nssalian commented on May 21, 2026

    @nssalian

    Thanks for creating this @zhjwpku , do you mind adding sub tasks around non shredding first, then shredding as well. You can look here under the go tracker Track A to have something similar for cpp

  3. zhjwpku commented on May 21, 2026

    @zhjwpku
    CollaboratorAuthor

    Thanks for creating this @zhjwpku , do you mind adding sub tasks around non shredding first, then shredding as well. You can look here under the go tracker Track A to have something similar for cpp

    Just dug into some details about Variant Shredding [1].

    [1] https://parquet.apache.org/docs/file-format/types/variantshredding/

  4. nssalian commented on May 21, 2026

    @nssalian

    @zhjwpku I would be best to add support for Variant as a type first which is usually writing unshredded variant data. Shredded writes can follow.

  5. wgtmac commented on May 21, 2026

    @wgtmac
    Member

    @nssalian Thanks for your interest! Do you have a concrete plan for the variant type support? I'm not quite sure if we need variant encoding in the iceberg library. As we depend on arrow-cpp for parquet, I'm not sure if we want to reuse variant encoding from arrow-cpp. (cc @HuaHuaY who is adding variant encoding to arrow-cpp)

  6. nssalian commented on May 21, 2026

    @nssalian

    @wgtmac thanks for weighing in. Yes, the plan is to wire arrow-cpp's variant into iceberg-cpp's type system, not duplicate the codec. iceberg-go follows the same pattern (defers to arrow-go's variant).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions