Skip to content

Fix chunk formatting warning and fix consolidated metadata not implemented warning - #1765

Open
ctuguinay wants to merge 5 commits into
echostack-org:mainfrom
ctuguinay:fix_chunk_formatting_warning
Open

ctuguinay wants to merge 5 commits into
echostack-org:mainfrom
ctuguinay:fix_chunk_formatting_warning

Conversation

@ctuguinay

@ctuguinay ctuguinay commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Problem 1:

Dask needs dictionary chunk encoding and Zarr requires list-like chunk encoding.

I fix this by separating Dask and Zarr encodings and rechunking Dask with it's dictionary encoding and removing that encoding once the encoding dictionary is passed into to_zarr.

Problem 2:

Xarray describes to_zarr's consolidated parameter behavior as:

If True, apply zarr’s consolidate_metadata function to the store after writing metadata and read existing stores with consolidated metadata; if False, do not. The default (consolidated=None) means write consolidated metadata and attempt to read consolidated metadata for existing stores (falling back to non-consolidated).

But Zarr v3 doesn't support consolidated metadata just yet, so we will always get this falling back to non-consolidated warning whenever we call .to_zarr on an Echodata object serialized using Zarr v3. Fixing this will handle one of the problems listed here: echostack-org/echopype-examples#119.

I fix this by explicitly calling consolidated=False.

@ctuguinay ctuguinay self-assigned this Aug 10, 2026
@ctuguinay ctuguinay added the bug Something isn't working label Aug 10, 2026
@ctuguinay ctuguinay linked an issue Aug 10, 2026 that may be closed by this pull request
@ctuguinay ctuguinay closed this Aug 10, 2026
@ctuguinay
ctuguinay force-pushed the fix_chunk_formatting_warning branch from 2ad2d4f to 8ca408d Compare August 10, 2026 17:36
@ctuguinay ctuguinay reopened this Aug 10, 2026
@ctuguinay

Copy link
Copy Markdown
Collaborator Author

Weird...not quite sure why it auto-closed itself

@ctuguinay

Copy link
Copy Markdown
Collaborator Author

This should be ready for review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Chunk formatting warning reappeared

1 participant