Skip to content

Fix z_l->lev rename for OM4 snapshot sources - #756

Merged
alxmrs merged 1 commit into
mainfrom
u/alxmrs/snapshots
Aug 19, 2026
Merged

Fix z_l->lev rename for OM4 snapshot sources#756
alxmrs merged 1 commit into
mainfrom
u/alxmrs/snapshots

Conversation

@alxmrs

@alxmrs alxmrs commented Jun 4, 2026

Copy link
Copy Markdown
Member

The 5-daily snapshot source (1958-2022.OM4p25_5daily_snapshots) exposes the depth-center coordinate as z_l but has no z_i. The old rename gated the z_l->lev rename on z_i being present, so z_l was left intact and downstream rechunking on "lev" failed with a ValueError (issue #450).

Extract normalize_vertical_coords() to rename z_l->lev and z_i->ilev independently, and key the interface-info branch on ilev rather than z_i. Behavior is unchanged for the averaged sources (which carry z_i).

🤖

@alxmrs

alxmrs commented Jun 4, 2026

Copy link
Copy Markdown
Member Author

I ran this script on with the small data setting and wrote the results to my machine. Here is the dataset it produced:

>>> ds
<xarray.Dataset> Size: 211MB
Dimensions:    (time: 10, y: 180, x: 360)
Coordinates:
  * time       (time) object 80B 1958-01-06 00:00:00 ... 1958-02-20 00:00:00
  * x          (x) float64 3kB 0.5 1.5 2.5 3.5 4.5 ... 356.5 357.5 358.5 359.5
  * y          (y) float64 1kB -89.24 -88.25 -87.25 -86.26 ... 87.25 88.25 89.24
Data variables: (12/100)
    hfds       (time, y, x) float32 3MB dask.array<chunksize=(1, 180, 360), meta=np.ndarray>
    mask_0     (y, x) bool 65kB dask.array<chunksize=(180, 360), meta=np.ndarray>
    mask_1     (y, x) bool 65kB dask.array<chunksize=(180, 360), meta=np.ndarray>
    mask_10    (y, x) bool 65kB dask.array<chunksize=(180, 360), meta=np.ndarray>
    mask_11    (y, x) bool 65kB dask.array<chunksize=(180, 360), meta=np.ndarray>
    mask_12    (y, x) bool 65kB dask.array<chunksize=(180, 360), meta=np.ndarray>
    ...         ...
    vo_6       (time, y, x) float32 3MB dask.array<chunksize=(1, 180, 360), meta=np.ndarray>
    vo_7       (time, y, x) float32 3MB dask.array<chunksize=(1, 180, 360), meta=np.ndarray>
    vo_8       (time, y, x) float32 3MB dask.array<chunksize=(1, 180, 360), meta=np.ndarray>
    vo_9       (time, y, x) float32 3MB dask.array<chunksize=(1, 180, 360), meta=np.ndarray>
    wfo        (time, y, x) float32 3MB dask.array<chunksize=(1, 180, 360), meta=np.ndarray>
    zos        (time, y, x) float32 3MB dask.array<chunksize=(1, 180, 360), meta=np.ndarray>
Attributes: (12/13)
    hfds:                              {'cell_measures': 'area: areacello', '...
    m2lines/cli_args:                  /Users/alxmrs/git/Ocean_Emulator/data/...
    m2lines/date_created:              2026-06-04T14:11:01.542698
    m2lines/ocean_emulators_git_hash:  https://github.com/Open-Athena/Ocean_E...
    regrid_method:                     conservative
    so:                                {'cell_measures': 'area: areacello', '...
    ...                                ...
    tauvo:                             {'cell_methods': 'yq:point xh:mean tim...
    thetao:                            {'cell_measures': 'area: areacello', '...
    uo:                                {'cell_methods': 'z_l:mean yh:mean xq:...
    vo:                                {'cell_methods': 'z_l:mean yq:point xh...
    wfo:                               {'cell_measures': 'area: areacello', '...
    zos:                               {'cell_measures': 'area: areacello', '...
>>> ds.time
<xarray.DataArray 'time' (time: 10)> Size: 80B
array([cftime.DatetimeJulian(1958, 1, 6, 0, 0, 0, 0, has_year_zero=False),
       cftime.DatetimeJulian(1958, 1, 11, 0, 0, 0, 0, has_year_zero=False),
       cftime.DatetimeJulian(1958, 1, 16, 0, 0, 0, 0, has_year_zero=False),
       cftime.DatetimeJulian(1958, 1, 21, 0, 0, 0, 0, has_year_zero=False),
       cftime.DatetimeJulian(1958, 1, 26, 0, 0, 0, 0, has_year_zero=False),
       cftime.DatetimeJulian(1958, 1, 31, 0, 0, 0, 0, has_year_zero=False),
       cftime.DatetimeJulian(1958, 2, 5, 0, 0, 0, 0, has_year_zero=False),
       cftime.DatetimeJulian(1958, 2, 10, 0, 0, 0, 0, has_year_zero=False),
       cftime.DatetimeJulian(1958, 2, 15, 0, 0, 0, 0, has_year_zero=False),
       cftime.DatetimeJulian(1958, 2, 20, 0, 0, 0, 0, has_year_zero=False)],
      dtype=object)
Coordinates:
  * time     (time) object 80B 1958-01-06 00:00:00 ... 1958-02-20 00:00:00

Here is the command I used:

>>> ds.attrs["m2lines/cli_args"]
'/Users/alxmrs/git/Ocean_Emulator/data/ocean_preprocessing/__main__.py om4 s3://emulators/wg2437/1958-2022.OM4p25_5daily_snapshots s3://m2lines-pubs/FOMO/raw/ocean_static_no_mask_table.zarr s3://m2lines-pubs/FOMO/raw/grids/ocean_hgrid.zarr s3://m2lines-pubs/FOMO/raw/grids/gaussian_grid_180_by_360.zarr --output_path=/tmp/om4_onedeg_snap_test.zarr --skip_spatial_filtering --skip_validation --small_run --cluster=local --n_workers=4'

@alxmrs

alxmrs commented Jun 5, 2026

Copy link
Copy Markdown
Member Author

@codex may I have your review?

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@alxmrs

alxmrs commented Aug 18, 2026

Copy link
Copy Markdown
Member Author

@fomo-bot will you review this PR? If you think it's ready to merge, will you approve this?

@fomo-bot

Copy link
Copy Markdown
Collaborator

No code findings in the current diff at dbedd352.

I did not approve because the PR is still a draft and GitHub reports mergeStateStatus=BLOCKED with reviewDecision=REVIEW_REQUIRED. The branch is also 14 commits behind current origin/main, so I would mark it ready/update it and let CI rerun before merge.

What I checked:

  • Full PR diff and live PR metadata.
  • No reviews, inline comments, or review threads are currently recorded.
  • git diff --check origin/main...HEAD passed.
  • cd data && uv run --extra test pytest tests/test_simulation_preprocessing.py -q passed: 3 passed, 2 warnings.
  • Existing GitHub checks on the current head are success/skipped, from July 23, 2026.
Debug info

@alxmrs
alxmrs marked this pull request as ready for review August 18, 2026 21:21
@alxmrs

alxmrs commented Aug 18, 2026

Copy link
Copy Markdown
Member Author

@fomo-bot will you review for approval now?

@alxmrs

alxmrs commented Aug 18, 2026

Copy link
Copy Markdown
Member Author

@fomo-bot will you also rebase this change with main?

The 5-daily snapshot source (1958-2022.OM4p25_5daily_snapshots) exposes the
depth-center coordinate as z_l but has no z_i. The old rename gated the
z_l->lev rename on z_i being present, so z_l was left intact and downstream
rechunking on "lev" failed with a ValueError (issue #450).

Extract normalize_vertical_coords() to rename z_l->lev and z_i->ilev
independently, and key the interface-info branch on ilev rather than z_i.
Behavior is unchanged for the averaged sources (which carry z_i).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@fomo-bot
fomo-bot force-pushed the u/alxmrs/snapshots branch from dbedd35 to e9f987e Compare August 18, 2026 21:22

@fomo-bot fomo-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed after rebasing onto current main. The diff is narrowly scoped, focused local validation passed, and the full GitHub check suite is green on e9f987e.

@fomo-bot

Copy link
Copy Markdown
Collaborator

Rebased PR #756 onto current origin/main and force-pushed the existing branch. New head is e9f987ec.



I re-reviewed the rebased diff and approved it. Final live state is reviewDecision=APPROVED, mergeable=MERGEABLE, and mergeStateStatus=CLEAN. GitHub CI is green on the rebased head, and local focused validation also passed with cd data && uv run --extra test pytest tests/test_simulation_preprocessing.py -q (3 passed, 2 warnings).

Debug info

@alxmrs
alxmrs added this pull request to the merge queue Aug 18, 2026
Merged via the queue into main with commit 018a6a7 Aug 19, 2026
20 checks passed
@alxmrs
alxmrs deleted the u/alxmrs/snapshots branch August 19, 2026 00:12
@github-project-automation github-project-automation Bot moved this from Backlog to Done in Samudra Planning Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants