Skip to content

Use PETSc-native XDMF visualization topology#218

Open
gthyagi wants to merge 7 commits into
underworldcode:developmentfrom
gthyagi:bugfix/petsc-hdf5-viz-topology
Open

Use PETSc-native XDMF visualization topology#218
gthyagi wants to merge 7 commits into
underworldcode:developmentfrom
gthyagi:bugfix/petsc-hdf5-viz-topology

Conversation

@gthyagi
Copy link
Copy Markdown
Contributor

@gthyagi gthyagi commented May 29, 2026

Summary

This PR revives the PETSc-native XDMF topology path from feature/annulus-spherical-benchmark-debug on top of current development, then extends the output API so timestep files can optionally carry PETSc reload metadata as well.

The branch currently contains:

  • 22dd9ab2 / 90774778: add petsc_format control to Mesh.write() and make XDMF use PETSc's direct /viz/topology/cells connectivity rather than internal DMPlex /topology/cells.
  • 9f41035e / 47719277: keep restart-style topology in write_timestep mesh files while still pointing XDMF to direct visualization topology.
  • 5cd8a1f8: remove the manual _write_mesh_viz_groups() path added in PR Fix XDMF output to explicitly write cell-to-vertex topology and ensure strict ParaView compatibility #205 and rely on PETSc-native visualization topology.
  • 4bd05789: add unified timestep/reload output support with write_timestep(..., petsc_reload=True) and write_checkpoint(..., create_xdmf=True).

Closes #217.

XDMF / visualization topology change

On current development, PR #205 added a manual Python _write_mesh_viz_groups() path to assemble /viz/geometry and /viz/topology. This PR removes that manual gather/deduplication path and lets PETSc write the visualization topology natively.

The resulting mesh output keeps the restart/topology information needed by UW3 while writing direct visualization connectivity at:

/viz/topology/cells

and geometry at:

/geometry/vertices

The XDMF writer validates and references the PETSc-written /viz/topology/cells dataset rather than raw DMPlex /topology/cells.

Unified timestep / PETSc reload output

This PR also adds an explicit output-payload switch:

mesh.write_timestep(
    "output",
    index=0,
    outputPath=str(output_dir),
    meshVars=[velocity, pressure],
    create_xdmf=True,
    petsc_reload=True,
)

With create_xdmf=True, timestep variable files contain the existing XDMF/remap payloads:

/fields/<variable>
/fields/coordinates
/vertex_fields/... or /cell_fields/...

With petsc_reload=True, the same per-variable HDF5 files also contain PETSc DMPlex reload payloads:

/topologies/uw_mesh/dms/<variable>/section
/topologies/uw_mesh/dms/<variable>/vecs/<variable>

That means a single output.mesh.<variable>.00000.h5 file can be used by both:

var.read_timestep(...)
var.read_checkpoint(...)

write_checkpoint(..., create_xdmf=True) now routes through the same unified writer. The default write_checkpoint() path remains restart-only and backward-compatible.

Why this is needed

The PETSc-native topology path avoids maintaining UW3-side MPI topology assembly, vertex deduplication, and cell reordering code. The unified output option also addresses the earlier architectural concern that visualization/remap output and PETSc-native reload output were locked into separate file families.

Users can now choose explicitly:

# Visualization/remap only
mesh.write_timestep(..., create_xdmf=True, petsc_reload=False)

# PETSc-native reload only
mesh.write_checkpoint(...)

# Unified visualization/remap + PETSc reload output
mesh.write_timestep(..., create_xdmf=True, petsc_reload=True)

Scope note

This PR addresses the PETSc-native /viz/topology/cells replacement requested in #217 for the current linear visualization topology workflow. It does not attempt to implement curved/isoparametric geometry support; that is a larger infrastructure change and should be tracked separately if needed.

Tests

Core UW3 tests run locally in amr-dev:

  • ./uw build
  • .pixi/envs/amr-dev/bin/python -m pytest -q tests/test_0003_save_load.py tests/test_0005_xdmf_compat.py
    • 16 passed
  • .pixi/envs/amr-dev/bin/mpirun -n 2 .pixi/envs/amr-dev/bin/python -m pytest -q --with-mpi tests/test_0003_save_load.py::test_timestep_with_petsc_reload_roundtrip
    • passed on both ranks
  • .pixi/envs/amr-dev/bin/mpirun -n 2 .pixi/envs/amr-dev/bin/python -m pytest -q --with-mpi tests/parallel/test_0005_xdmf_viz_topology_mpi.py
    • passed on both ranks in earlier validation for this branch

Benchmark validation:

  • Ran the spherical Thieulot benchmark at cellsize=1/8, P2/P1, m=-1, 8 MPI ranks using the unified writer path.
  • Confirmed unified files contain both XDMF/remap and PETSc reload payloads.
  • Confirmed the checkpoint-only metrics stage completes from the same unified files using read_checkpoint().
  • Serial reload comparison from the 8-rank output gave:
velocity max abs diff 0.0
pressure max abs diff 0.0

gthyagi added 3 commits May 30, 2026 04:23
Write visualization timesteps with PETSc HDF5_VIZ so generated XDMF references /viz/topology/cells rather than checkpoint-style DMPlex topology. Keep write_checkpoint on HDF5_PETSC for restart files.

Harden checkpoint_xdmf against invalid topology datasets, use integer topology precision from the HDF5 dataset, and write mesh variable DataItems with their direct dataset dimensions instead of HyperSlab wrappers.

Add regression tests covering write_timestep visualization topology, write_checkpoint restart topology, and 3D XDMF references.
Restore write_timestep mesh files to PETSc's default HDF5 layout instead of forcing HDF5_VIZ. This preserves labels and internal DMPlex restart datasets such as /topology/cells, /topology/cones, /topology/order, and /topology/orientation in output.mesh.00000.h5.

Keep the XDMF writer guarded so visualization still references /viz/topology/cells and never the internal PETSc /topology/cells connectivity. This keeps ParaView-compatible direct cell connectivity while retaining restart-like mesh contents.

Update the XDMF compatibility regression test to require both restart-style topology datasets and the /viz/topology/cells dataset from write_timestep output.
@gthyagi
Copy link
Copy Markdown
Contributor Author

gthyagi commented May 29, 2026

Additional validation after opening the PR:

  • Confirmed the checkout is on bugfix/petsc-hdf5-viz-topology.
  • Ran ./uw build in the configured amr-dev environment; build completed successfully.
  • Re-ran .pixi/envs/amr-dev/bin/python -m pytest -q tests/test_0005_xdmf_compat.py after the build: 10 passed.
  • Re-ran .pixi/envs/amr-dev/bin/mpirun -n 2 .pixi/envs/amr-dev/bin/python -m pytest -q --with-mpi tests/parallel/test_0005_xdmf_viz_topology_mpi.py after the build: 1 passed on each rank.

gthyagi added 2 commits May 30, 2026 05:14
Add a petsc_reload option to Mesh.write_timestep so timestep variable files can include PETSc DMPlex section/vector metadata alongside the existing XDMF, /fields, and vertex/cell compatibility payloads. This lets the same per-variable HDF5 files support both read_timestep remapping and read_checkpoint reloads.

Refactor PETSc reload writing into shared helpers and add create_xdmf support to write_checkpoint by routing through the unified timestep writer. Invalid option combinations for the timestep layout now fail explicitly.

Update checkpoint/reload developer documentation and add roundtrip tests covering unified write_timestep output and write_checkpoint(create_xdmf=True).
Emit a visible FutureWarning from Mesh.write_checkpoint() so new code is directed to the unified write_timestep(..., petsc_reload=True) API.

Update checkpoint/reload documentation to present write_timestep() as the standard output method and write_checkpoint() as a legacy compatibility wrapper. Adjust tests to assert the warning on compatibility calls.
@gthyagi
Copy link
Copy Markdown
Contributor Author

gthyagi commented May 30, 2026

Follow-up pushed in 5ab5cb40:

  • Mesh.write_checkpoint() now emits a visible FutureWarning and is documented as a compatibility wrapper.
  • New code is directed to the unified standard API: write_timestep(..., petsc_reload=True).
  • Developer docs now present write_timestep() as the single recommended output method, with create_xdmf and petsc_reload selecting the payloads.
  • Tests that intentionally exercise write_checkpoint() now assert the compatibility warning.

Validation after the change:

  • ./uw build
  • .pixi/envs/amr-dev/bin/python -m pytest -q tests/test_0003_save_load.py tests/test_0005_xdmf_compat.py16 passed
  • .pixi/envs/amr-dev/bin/mpirun -n 2 .pixi/envs/amr-dev/bin/python -m pytest -q --with-mpi tests/test_0003_save_load.py::test_timestep_with_petsc_reload_roundtrip → passed on both ranks

@gthyagi gthyagi marked this pull request as ready for review May 30, 2026 10:59
@gthyagi gthyagi requested a review from lmoresi as a code owner May 30, 2026 10:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant