Skip to content

The ARTS 2D path has no working automated coverage — three independent failures #124

Description

@joglekara

The 2D/ARTS integration tests do not test anything, and the CI check that appears to cover them is green for the wrong reason. Three separate problems stack up; fixing any one alone leaves the path still uncovered.

Surfaced while working on #120 / #122, which changes 2D numerical output and therefore wanted this safety net. Not caused by that PR.

1. gpu_runner exposes no GPU to JAX

.github/workflows/test.yaml defines test-tsadar-gpu with runs-on: "gpu_runner" and installs -e ".[gpu,test,hdf]", so the intent is clearly there. But both jobs on a recent run report identical results:

job result
test-tsadar-cpu 24 passed, 5 skipped (172s)
test-tsadar-gpu 24 passed, 5 skipped (225s)

And the GPU job's own log shows the angular tests skipping:

tests/test_forward/test_angular_2v.py s
tests/test_inverse/test_angular_2v.py ss

Those skip on if not any(["gpu" == device.platform for device in devices()]), so JAX saw no GPU. Either the runner has no device attached or the [gpu] extra isn't bringing up CUDA. Worth a python -c "import jax; print(jax.devices())" step in the job so this is visible instead of silent.

2. The config files those tests open were deleted

Even with a working GPU, both tests raise FileNotFoundError:

test opens status
tests/test_forward/test_angular_2v.py:39 tests/configs/arts2v_test_defaults.yaml exists
tests/test_forward/test_angular_2v.py:42 tests/configs/arts2v_test_inputs.yaml missing
tests/test_inverse/test_angular_2v.py:134 tests/configs/arts2d_test_defaults.yaml missing
tests/test_inverse/test_angular_2v.py:137 tests/configs/arts2d_test_inputs.yaml exists

Both missing files were removed in b75dec9 (#95). Each test lost exactly one of its two config files, which is why this went unnoticed — the tests never got far enough to complain, because they were already skipping.

3. The surviving config has rotted independently

tests/configs/arts2d_test_inputs.yaml:81-84 sets:

flm_type: mora-yahi
LTx: 22500.
LTy: 40000.

but spherical_harmonics.py:229 reads dist_cfg["params"]["dtx"] on the mora-yahi branch. Restoring the deleted defaults file would surface a KeyError next.

Net effect

There is no automated check on the 2D/ARTS path at all — not on CPU (skipped), not on GPU (skipped, and would crash). #122 adds tests/test_forward/test_sinogram.py::test_calc_in_2D_matches_exact_rotation, which drives calc_in_2D end-to-end on CPU in a few seconds and gives that path some floor. But it is a numerical-equivalence check against the pre-existing implementation, not a physics regression test, and it does not replace the integration tests.

Suggested order

  1. Add a device-report step to the GPU job so ported some code to jax #1 is never silent again.
  2. Decide whether gpu_runner is expected to have a GPU. If not, the angular tests are dead weight and the job name is actively misleading.
  3. Restore or regenerate the two missing config files, and fix arts2d_test_inputs.yaml's LTx/LTy -> dtx.
  4. Consider making the ATS tests fail rather than skip when their fixtures are missing — a skip is indistinguishable from "no GPU here" and is what hid this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions