Export partition to apache iceberg by arthurpassos · Pull Request #1618 · Altinity/ClickHouse

arthurpassos · 2026-04-06T13:49:44Z

Export partition mechanics changes:

ping restarting thread in case of zookeeper session failure
adds a few failpoints to make testing better
makes export_merge_tree_partition_system_table_prefer_remote_information false by default (I am considering to remove it completely)
adds commit retry count / max retries to prevent a task from living forever when commit is failing. Fail the entire task if commit retries exceeds max retries.
fixes race condition in ExportPartitionManifestUpdatingTask by draining the status queue while only holding the status lock instead of holding the status lock and the export partition lock
abstract away common functions like getContextCopyWithTaskSettings to avoid code duplication
Add task timeout. If the task exceeds the timeout, it is killed with reason: timeout exceeded. This helps with apache iceberg idempotency vs old manifest cleanup, tasks stuck in pending state forever due to missing parts OR no destination table;
rename enable_experimental_export_merge_tree_partition_feature to allow_experimental_export_merge_tree_partition
throw on exports if allow_experimental_insert_into_iceberg not enabled

Apache Iceberg specifics:

Store apache iceberg metadata json in zookeeper task
Derive destination partition values from source merge tree part (no recalculation)
preserve write_full_path_in_iceberg_metadata in zookeeper task
now exportparttask has a commit step that is only executed in case it is not export partition - this is because we need to commit even a single part - maybe I should re-think this architecture.
some vibe coded structures for iceberg stats
write f_clickhouse_export_partition_transaction_id to apache iceberg manifest so we can check it before comitting twice
copy and paste and adapt the icebegergstoragesink commmit phase to icebergmetadata so we can commit export partition operations
create sidecar file to persist file level statistics so they can be used at commit time - those are downloaded / read at commit time
create a simple icebergimportsink
add per fil stats to multifilewriter

Changelog category (leave one):

Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Make changes to the export partition background engine and support experimental exports to apache iceberg

Documentation entry for user-facing changes

...

CI/CD Options

Exclude tests:

Regression jobs to run:

github-actions · 2026-04-06T13:50:51Z

Workflow [PR], commit [dbf7866]

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5b0e833565

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-04-06T14:05:02Z

+        {
+            auto * object_storage = dynamic_cast<StorageObjectStorageCluster *>(dest_storage.get());
+
+            auto * iceberg_metadata = dynamic_cast<IcebergMetadata *>(object_storage->getExternalMetadata(query_context));


Guard IcebergMetadata casts with USE_AVRO

IcebergMetadata is defined only under #if USE_AVRO in IcebergMetadata.h, but this new dynamic_cast<IcebergMetadata *> is compiled unconditionally. In non-AVRO builds (USE_AVRO=0), this translation unit (and the same pattern added in StorageReplicatedMergeTree.cpp) cannot compile, so the change breaks optional-AVRO build configurations.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-04-06T14:05:02Z

+        const String sidecar_path = replaceFileExtensionWithAvro(
+            filename_generator.convertMetadataPathToStoragePath(path));


Use storage paths directly when reading export sidecars

The export path list is populated from filename.path_in_storage, but commit now treats each entry as a metadata path and calls convertMetadataPathToStoragePath before reading sidecars. With write_full_path_in_iceberg_metadata=1, table_dir is an URI prefix (for example s3://...) while these entries are plain storage paths (/...), so the conversion throws and EXPORT PARTITION cannot commit.

Useful? React with 👍 / 👎.

arthurpassos · 2026-04-07T12:41:56Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6c6194814d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

DimensionWieldr · 2026-04-21T14:25:09Z

+        context_copy->makeQueryContextForExportPart();
+        context_copy->setCurrentQueryId(manifest.query_id);
+        context_copy->setSetting("output_format_parallel_formatting", manifest.parallel_formatting);
+        context_copy->setSetting("output_format_parquet_parallel_encoding", manifest.parquet_parallel_encoding);


Got a failing test trying to use output_format_parquet_compression_method ZSTD and SNAPPY. Looks like this setting isn't propagated here?

Yeah.. this is the reason, but only for partition export. For part export that should be working.

I still need to find the best approach for persisting these settings.

mkmkme

There's no way I could finish this review properly in time. From reading the code diagonally, it looks good. Discussed some of the AI findings in the call. Probably worth addressing in the future PRs related to that feature.

Approving.

mkmkme · 2026-04-22T12:29:41Z

 - **Type**: `Bool`
 - **Default**: `false`
- **Description**: Ignore existing partition export and overwrite the ZooKeeper entry. Allows re-exporting a partition to the same destination before the manifest expires.
+- **Description**: Ignore existing partition export and overwrite the ZooKeeper entry. Allows re-exporting a partition to the same destination before the manifest expires. **IMPORTANT:** this is dangerou because it can lead to duplicated data, use it with caution.


"dangerou" :)

arthurpassos · 2026-04-23T18:28:18Z

todo arthur check key value pair string because maybe it will be duplicated

DimensionWieldr · 2026-04-23T19:02:24Z

@arthurpassos For anyone upgrading CH from a version that didn't have export to one that does, existing/old tables' ZooKeeper trees are missing the exports folder.

I have a test for this failing when it tries to run EXPORT PARTITION on a table whose ZooKeeper tree doesn't have the exports/ folder yet. CH tries to create a file inside that folder but ZooKeeper refuses because the parent folder doesn't exist. The EXPORT command fails with a "no such node" error.

arthurpassos · 2026-04-23T19:44:10Z

@arthurpassos For anyone upgrading CH from a version that didn't have export to one that does, existing/old tables' ZooKeeper trees are missing the exports folder.

I have a test for this failing when it tries to run EXPORT PARTITION on a table whose ZooKeeper tree doesn't have the exports/ folder yet. CH tries to create a file inside that folder but ZooKeeper refuses because the parent folder doesn't exist. The EXPORT command fails with a "no such node" error.

This is weird, I should have it covered. Let's discuss this internally

DimensionWieldr · 2026-04-23T20:10:47Z

@arthurpassos Truncate and export seem to disagree on REST catalog.

Basically when I EXPORT PARTITION A, CH writes the manifest-list path as a bucket-relative path: /data/iceberg_.../metadata/snap-797704761-...avro.
Then TRUNCATE the destination, this time CH writes the manifest-list path as a full S3 URI, something like s3://warehouse/data/iceberg_.../metadata/snap-1022122687-...avro.
Then I EXPORT PARTITION B and IcebergWrites.cpp inspects the existing files, sees both s3://... and /data/... paths coexisting, and refuses to commit with Code: 36. DB::Exception: Paths in Iceberg must use a consistent format.

This looks like kinda the same issue I mentioned before, which should be fixed by ClickHouse#100420. So I guess we leave it for now?

arthurpassos · 2026-04-23T20:12:17Z

@arthurpassos Truncate and export seem to disagree on REST catalog.

Basically when I EXPORT PARTITION A, CH writes the manifest-list path as a bucket-relative path: /data/iceberg_.../metadata/snap-797704761-...avro. Then TRUNCATE the destination, this time CH writes the manifest-list path as a full S3 URI, something like s3://warehouse/data/iceberg_.../metadata/snap-1022122687-...avro. Then I EXPORT PARTITION B and IcebergWrites.cpp inspects the existing files, sees both s3://... and /data/... paths coexisting, and refuses to commit with Code: 36. DB::Exception: Paths in Iceberg must use a consistent format.

This looks like kinda the same issue I mentioned before, which should be fixed by ClickHouse#100420. So I guess we leave it for now?

How does regular insert behave in this case? 'Who' does it agree with?

arthurpassos · 2026-04-23T20:14:42Z

@arthurpassos For anyone upgrading CH from a version that didn't have export to one that does, existing/old tables' ZooKeeper trees are missing the exports folder.
I have a test for this failing when it tries to run EXPORT PARTITION on a table whose ZooKeeper tree doesn't have the exports/ folder yet. CH tries to create a file inside that folder but ZooKeeper refuses because the parent folder doesn't exist. The EXPORT command fails with a "no such node" error.

This is weird, I should have it covered. Let's discuss this internally

Discussed this internally and it was an err on test side. A re-start is required as the nodes are created upon initialization

DimensionWieldr · 2026-04-23T20:19:38Z

@arthurpassos Truncate and export seem to disagree on REST catalog.
Basically when I EXPORT PARTITION A, CH writes the manifest-list path as a bucket-relative path: /data/iceberg_.../metadata/snap-797704761-...avro. Then TRUNCATE the destination, this time CH writes the manifest-list path as a full S3 URI, something like s3://warehouse/data/iceberg_.../metadata/snap-1022122687-...avro. Then I EXPORT PARTITION B and IcebergWrites.cpp inspects the existing files, sees both s3://... and /data/... paths coexisting, and refuses to commit with Code: 36. DB::Exception: Paths in Iceberg must use a consistent format.
This looks like kinda the same issue I mentioned before, which should be fixed by ClickHouse#100420. So I guess we leave it for now?

How does regular insert behave in this case? 'Who' does it agree with?

Insert is bucket-relative too, so same as export. Looks like truncate is the outlier.

arthurpassos · 2026-04-23T20:24:01Z

@arthurpassos Truncate and export seem to disagree on REST catalog.
Basically when I EXPORT PARTITION A, CH writes the manifest-list path as a bucket-relative path: /data/iceberg_.../metadata/snap-797704761-...avro. Then TRUNCATE the destination, this time CH writes the manifest-list path as a full S3 URI, something like s3://warehouse/data/iceberg_.../metadata/snap-1022122687-...avro. Then I EXPORT PARTITION B and IcebergWrites.cpp inspects the existing files, sees both s3://... and /data/... paths coexisting, and refuses to commit with Code: 36. DB::Exception: Paths in Iceberg must use a consistent format.
This looks like kinda the same issue I mentioned before, which should be fixed by ClickHouse#100420. So I guess we leave it for now?

How does regular insert behave in this case? 'Who' does it agree with?

Insert is bucket-relative too, so same as export. Looks like truncate is the outlier.

So.. here's the deal: I trusted the current Iceberg writes upstream implementation. It will write full paths for the manifest files (not data files) IF, and only if, write_full_path_in_iceberg_metadata is true. The data files are not respecting it iirc, and that PR should fix it (maybe it does more than that).

On the other hand, the truncate implementation by @il9ue will write full paths if write_full_path_in_iceberg_metadata is true OR the catalog is transactional (this is the piece regular writes and exports are not following at this moment).

This is where the inconsistency lives. The question is: who is right?

DimensionWieldr · 2026-04-23T21:29:00Z

Tests are pretty much done and passing (with the exception of some issues discussed above with iceberg writes) and cover rest and glue catalogs.

Did one more pass of AI audit and it came back with one issue.

Medium: Commit precondition no-op leaves export `PENDING` without commit retry budget
  • Impact: After all parts are in processed/, commit() can return without calling Iceberg commit or updating ZK status, and without calling handleCommitFailure. The task can stay
    `PENDING` indefinitely (until manual kill, TTL on a later terminal state, or optional task timeout), with `commit_attempts` stuck at 0.
  • Anchor: ExportPartitionUtils::commit (ExportPartitionUtils.cpp, early returns after getExportedPaths); callers ExportPartitionTaskScheduler::handlePartExportSuccess and
    ExportPartitionManifestUpdatingTask.cpp (tryCleanup commit fix-up).
  • Trigger: getExportedPaths returns empty or fewer paths than manifest.parts.size() while ZK already shows no processing/* children (e.g. ZK read inconsistency, partial processed
    data, or path accounting bug).
  • Why this is a defect: The state model explicitly adds commit_attempts / FAILED for exceptional commit failures, but this branch is a silent failure — no terminal transition and
    no budget consumption; `tryCleanup` still returns `true` after a no-op `commit()`, which mislabels cleanup success relative to the documented tryCleanup contract.
  • Fix direction (short): Treat empty / incomplete exported paths as an error: throw (so existing handleCommitFailure runs) or call handleCommitFailure / a dedicated counter before
    return; make tryCleanup return `false` unless ZK status is actually terminalized.
  • Regression test direction (short): Integration or unit test: empty processed/ listing at commit time → assert commit_attempts increases and eventual FAILED (or explicit throw),
    not infinite PENDING.

@arthurpassos Thoughts?

arthurpassos · 2026-04-23T22:01:31Z

Tests are pretty much done and passing (with the exception of some issues discussed above with iceberg writes) and cover rest and glue catalogs.

Did one more pass of AI audit and it came back with one issue.

Medium: Commit precondition no-op leaves export `PENDING` without commit retry budget
  • Impact: After all parts are in processed/, commit() can return without calling Iceberg commit or updating ZK status, and without calling handleCommitFailure. The task can stay
    `PENDING` indefinitely (until manual kill, TTL on a later terminal state, or optional task timeout), with `commit_attempts` stuck at 0.
  • Anchor: ExportPartitionUtils::commit (ExportPartitionUtils.cpp, early returns after getExportedPaths); callers ExportPartitionTaskScheduler::handlePartExportSuccess and
    ExportPartitionManifestUpdatingTask.cpp (tryCleanup commit fix-up).
  • Trigger: getExportedPaths returns empty or fewer paths than manifest.parts.size() while ZK already shows no processing/* children (e.g. ZK read inconsistency, partial processed
    data, or path accounting bug).
  • Why this is a defect: The state model explicitly adds commit_attempts / FAILED for exceptional commit failures, but this branch is a silent failure — no terminal transition and
    no budget consumption; `tryCleanup` still returns `true` after a no-op `commit()`, which mislabels cleanup success relative to the documented tryCleanup contract.
  • Fix direction (short): Treat empty / incomplete exported paths as an error: throw (so existing handleCommitFailure runs) or call handleCommitFailure / a dedicated counter before
    return; make tryCleanup return `false` unless ZK status is actually terminalized.
  • Regression test direction (short): Integration or unit test: empty processed/ listing at commit time → assert commit_attempts increases and eventual FAILED (or explicit throw),
    not infinite PENDING.

@arthurpassos Thoughts?

This is VERY unlikely, it can only happen if the zookeeper data got corrupted in between calls. And for these cases, the task timeout should do its job.

Regardless, the idea of treating those cases as errors and bump the commit_retry_attempts is a good idea. I will implement it.

arthurpassos · 2026-04-23T22:08:46Z

Tests are pretty much done and passing (with the exception of some issues discussed above with iceberg writes) and cover rest and glue catalogs.
Did one more pass of AI audit and it came back with one issue.

Medium: Commit precondition no-op leaves export `PENDING` without commit retry budget
  • Impact: After all parts are in processed/, commit() can return without calling Iceberg commit or updating ZK status, and without calling handleCommitFailure. The task can stay
    `PENDING` indefinitely (until manual kill, TTL on a later terminal state, or optional task timeout), with `commit_attempts` stuck at 0.
  • Anchor: ExportPartitionUtils::commit (ExportPartitionUtils.cpp, early returns after getExportedPaths); callers ExportPartitionTaskScheduler::handlePartExportSuccess and
    ExportPartitionManifestUpdatingTask.cpp (tryCleanup commit fix-up).
  • Trigger: getExportedPaths returns empty or fewer paths than manifest.parts.size() while ZK already shows no processing/* children (e.g. ZK read inconsistency, partial processed
    data, or path accounting bug).
  • Why this is a defect: The state model explicitly adds commit_attempts / FAILED for exceptional commit failures, but this branch is a silent failure — no terminal transition and
    no budget consumption; `tryCleanup` still returns `true` after a no-op `commit()`, which mislabels cleanup success relative to the documented tryCleanup contract.
  • Fix direction (short): Treat empty / incomplete exported paths as an error: throw (so existing handleCommitFailure runs) or call handleCommitFailure / a dedicated counter before
    return; make tryCleanup return `false` unless ZK status is actually terminalized.
  • Regression test direction (short): Integration or unit test: empty processed/ listing at commit time → assert commit_attempts increases and eventual FAILED (or explicit throw),
    not infinite PENDING.

@arthurpassos Thoughts?

This is VERY unlikely, it can only happen if the zookeeper data got corrupted in between calls. And for these cases, the task timeout should do its job.

Regardless, the idea of treating those cases as errors and bump the commit_retry_attempts is a good idea. I will implement it.

Should be fixed by af3dda1

DimensionWieldr · 2026-04-24T01:05:47Z

We don't have time to iron out everything, but I think we've covered the main user scenarios. Since we're looking to release very soon, I'll put down a few more of AI's nitpicks that we can address if there is time. Otherwise, we can leave them for the next release.

Zero-row export/import path can dereference null writer internals
    • Impact: export-to-Iceberg can crash in finalization when no data file was opened.
    • Anchor: src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergWrites.cpp (IcebergImportSink::finalizeBuffers), src/Storages/ObjectStorage/DataLakes/Iceberg/MultipleFileWriter.cpp
       (MultipleFileWriter::finalize).
    • Trigger: export pipeline produces no chunks (e.g., part rows fully masked by delete mask).
    • Why defect: finalizeBuffers() always calls writer->finalize(), but finalize() unconditionally dereferences output_format/buffer, which are initialized only after startNewFile()
       from consume().
    • Fix direction (short): guard finalize()/finalizeBuffers() for "no file opened" state.

Replicated export tasks can fail despite query-level Iceberg enablement
    • Impact: EXPORT PARTITION ... TO Iceberg can fail in background with SUPPORT_IS_DISABLED even when initiating query enabled Iceberg writes.
    • Anchor: src/Storages/MergeTree/ExportPartitionUtils.cpp (getContextCopyWithTaskSettings), src/Storages/MergeTree/MergeTreeData.cpp (exportPartToTable).
    • Trigger: global/profile allow_experimental_insert_into_iceberg=0, session enables it only for the export query (default scheduler mode).
    • Why defect: background task context reconstructs selected settings but does not propagate allow_experimental_insert_into_iceberg; part export re-checks it and throws.
    • Fix direction (short): persist and replay allow_experimental_insert_into_iceberg in manifest/task context.

Transactional truncate constructs malformed catalog metadata location
    • Impact: truncate on transactional catalogs can publish an invalid metadata URI (double-prefixed location), risking failed or broken catalog updates.
    • Anchor: src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergMetadata.cpp (truncate), plus src/Storages/ObjectStorage/DataLakes/Iceberg/FileNamesGenerator.cpp
      (generateMetadataName).
    • Trigger: catalog->isTransactional() true in truncate flow.
    • Why defect: generator is initialized with table_dir=location, so metadata_name is already location-prefixed; code then prepends location again.
    • Fix direction (short): use metadata_name directly when already absolute/prefixed.

KILLED can still be overwritten to COMPLETED by racing commit
    • Impact: user-visible state machine is non-monotonic (PENDING -> KILLED -> COMPLETED) under kill/commit race.
    • Anchor: src/Storages/MergeTree/ExportPartitionUtils.cpp (commit), src/Storages/StorageReplicatedMergeTree.cpp (killExportPartition).
    • Trigger: KILL EXPORT PARTITION while commit already passed data commit and is updating status.
    • Why defect: kill uses version-checked CAS from PENDING; commit uses unconditional trySet(..., -1) to COMPLETED.
    • Fix direction (short): make final COMPLETED transition conditional on expected pending state/version.

arthurpassos · 2026-04-24T01:24:27Z

We don't have time to iron out everything, but I think we've covered the main user scenarios. Since we're looking to release very soon, I'll put down a few more nitpicks that we can address if there is time. Otherwise, we can leave them for the next release.


Zero-row export/import path can dereference null writer internals

    • Impact: export-to-Iceberg can crash in finalization when no data file was opened.

    • Anchor: src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergWrites.cpp (IcebergImportSink::finalizeBuffers), src/Storages/ObjectStorage/DataLakes/Iceberg/MultipleFileWriter.cpp

       (MultipleFileWriter::finalize).

    • Trigger: export pipeline produces no chunks (e.g., part rows fully masked by delete mask).

    • Why defect: finalizeBuffers() always calls writer->finalize(), but finalize() unconditionally dereferences output_format/buffer, which are initialized only after startNewFile()

       from consume().

    • Fix direction (short): guard finalize()/finalizeBuffers() for "no file opened" state.



Replicated export tasks can fail despite query-level Iceberg enablement

    • Impact: EXPORT PARTITION ... TO Iceberg can fail in background with SUPPORT_IS_DISABLED even when initiating query enabled Iceberg writes.

    • Anchor: src/Storages/MergeTree/ExportPartitionUtils.cpp (getContextCopyWithTaskSettings), src/Storages/MergeTree/MergeTreeData.cpp (exportPartToTable).

    • Trigger: global/profile allow_experimental_insert_into_iceberg=0, session enables it only for the export query (default scheduler mode).

    • Why defect: background task context reconstructs selected settings but does not propagate allow_experimental_insert_into_iceberg; part export re-checks it and throws.

    • Fix direction (short): persist and replay allow_experimental_insert_into_iceberg in manifest/task context.



Transactional truncate constructs malformed catalog metadata location

    • Impact: truncate on transactional catalogs can publish an invalid metadata URI (double-prefixed location), risking failed or broken catalog updates.

    • Anchor: src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergMetadata.cpp (truncate), plus src/Storages/ObjectStorage/DataLakes/Iceberg/FileNamesGenerator.cpp

      (generateMetadataName).

    • Trigger: catalog->isTransactional() true in truncate flow.

    • Why defect: generator is initialized with table_dir=location, so metadata_name is already location-prefixed; code then prepends location again.

    • Fix direction (short): use metadata_name directly when already absolute/prefixed.



KILLED can still be overwritten to COMPLETED by racing commit

    • Impact: user-visible state machine is non-monotonic (PENDING -> KILLED -> COMPLETED) under kill/commit race.

    • Anchor: src/Storages/MergeTree/ExportPartitionUtils.cpp (commit), src/Storages/StorageReplicatedMergeTree.cpp (killExportPartition).

    • Trigger: KILL EXPORT PARTITION while commit already passed data commit and is updating status.

    • Why defect: kill uses version-checked CAS from PENDING; commit uses unconditional trySet(..., -1) to COMPLETED.

    • Fix direction (short): make final COMPLETED transition conditional on expected pending state/version.

It can't happen, there is no way to construct a zero row part afaik
True, but not sure what's the right course of action here
Truncate related
We have already discussed this 100 times, not an issue

arthurpassos added 19 commits March 19, 2026 09:02

prior to llm build fix

be1fd1d

making progress

b82ce39

fix version

2687a20

export part working now

4175e33

progress before refactor

d9a27c7

checkpoint, commented out partitioning..

fdf6bf7

compatibility check seems ok

bca4aeb

simplify code

843961c

fix tst that was.. not working for some super odd reason

80753da

fix compatibility check for year vs years

fc93f76

fix

abbc77d

progress

57f3a6a

do not even recompute partition value, just use it from the source

1ee1b34

somehow fix the concurrency problem

52a812a

make it actually transactional

da5b6be

Merge branch 'antalya-26.1' into export_partition_iceberg

3664749

add test for crash during 2phase commit

a878a1e

not quite good

b5bd0eb

put writefullpath in zk and add some comments

5b0e833

try to fix fast_test

998a992

chatgpt-codex-connector Bot reviewed Apr 6, 2026

View reviewed changes

arthurpassos added 5 commits April 6, 2026 11:14

again

1aa1b31

again

f21d66a

again

26827e2

partially fix path bug

6c61948

some more improvements

99ce30f

chatgpt-codex-connector Bot reviewed Apr 7, 2026

View reviewed changes

Comment thread src/Storages/MergeTree/MergeTreeData.cpp

Comment thread src/Storages/StorageReplicatedMergeTree.cpp Outdated

vibe coded ffix for catalog concurrent writes

4259ec3

arthurpassos added 2 commits April 21, 2026 09:39

also check compatibility for iceberg spec on simple part export

30b598a

change exception code

4351238

DimensionWieldr reviewed Apr 21, 2026

View reviewed changes

arthurpassos added 2 commits April 21, 2026 11:37

simplify stats and add tests that verify them

de8a5df

include missing files

ae11e30

mkmkme previously approved these changes Apr 22, 2026

View reviewed changes

throw on iceberg_exprimental_writes not enabled

ed75fa2

arthurpassos dismissed mkmkme’s stale review via ed75fa2 April 23, 2026 16:57

preserve max rows and max bytes for iceberg exports

2e54f55

svb-alt added the antalya-26.1.11.20001 label Apr 23, 2026

throw on commit errs

af3dda1

DimensionWieldr added the verified Approved for release label Apr 24, 2026

arthurpassos added 5 commits April 24, 2026 08:44

enable iceberg writes in tests

01e51f7

rename setting

be18239

improve docs

debc24a

fix pending test

4a6cb1a

forgot about standalone export part tests

dbf7866

		const String sidecar_path = replaceFileExtensionWithAvro(
		filename_generator.convertMetadataPathToStoragePath(path));

Conversation

arthurpassos commented Apr 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Documentation entry for user-facing changes

CI/CD Options

Exclude tests:

Regression jobs to run:

Uh oh!

github-actions Bot commented Apr 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Apr 6, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Apr 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

arthurpassos commented Apr 7, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

DimensionWieldr Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

arthurpassos Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

mkmkme left a comment

Choose a reason for hiding this comment

Uh oh!

mkmkme Apr 22, 2026

Choose a reason for hiding this comment

Uh oh!

arthurpassos commented Apr 23, 2026

Uh oh!

DimensionWieldr commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

arthurpassos commented Apr 23, 2026

Uh oh!

DimensionWieldr commented Apr 23, 2026

Uh oh!

arthurpassos commented Apr 23, 2026

Uh oh!

arthurpassos commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DimensionWieldr commented Apr 23, 2026

Uh oh!

arthurpassos commented Apr 23, 2026

Uh oh!

DimensionWieldr commented Apr 23, 2026

Uh oh!

arthurpassos commented Apr 23, 2026

Uh oh!

arthurpassos commented Apr 23, 2026

Uh oh!

DimensionWieldr commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

arthurpassos commented Apr 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

arthurpassos commented Apr 6, 2026 •

edited

Loading

github-actions Bot commented Apr 6, 2026 •

edited

Loading

DimensionWieldr commented Apr 23, 2026 •

edited

Loading

arthurpassos commented Apr 23, 2026 •

edited

Loading

DimensionWieldr commented Apr 24, 2026 •

edited

Loading