Skip to content

test: Tier 0-2 scaffolding (shell syntax, imports, entrypoints, runner metadata)#708

Open
cailmdaley wants to merge 3 commits intodevelopfrom
chore/testing-scaffold
Open

test: Tier 0-2 scaffolding (shell syntax, imports, entrypoints, runner metadata)#708
cailmdaley wants to merge 3 commits intodevelopfrom
chore/testing-scaffold

Conversation

@cailmdaley
Copy link
Copy Markdown
Contributor

Summary

Scaffold a cheap, broad test suite that gates PRs to develop. The goal is to catch the kind of bugs we've hit repeatedly in recent reviews — bash syntax errors (#706 `init_run_v2.0.sh`), module-level import of optional deps that makes a `try/except` unreachable (#699 `coverage_map_builder.py`), broken `[project.scripts]` entry points, runners missing `@module_runner` metadata, docker image upstream drift (#707 skyproj/PROJ).

What's covered

File Catches
`tests/unit/test_shell_syntax.py` `bash -n` over every `scripts/sh/*.{sh,bash}` — unclosed quotes, unbalanced brackets, stray parens
`tests/unit/test_config_parse.py` configparser over every `example/**/*.ini` — malformed section headers, duplicate keys
`tests/unit/test_imports.py` Import every `shapepipe.*` submodule — syntax errors, typos, unreachable `except ImportError`
`tests/unit/test_entrypoints.py` `<script> -h` on every `[project.scripts]` entry — broken targets, args mishandled
`tests/unit/test_runner_metadata.py` Every `*_runner.py` exports a function decorated with `@module_runner`

Each test is parametrized one-case-per-file, so the CI output names exactly which script/module/runner is at fault.

Pre-existing failures

These surfaced when the suite first ran on `develop`. Tracked as `xfail(strict=True)` so the PR lands green and the failure re-surfaces as a loud `XPASSED` once fixed:

  • `stile v0.1` imports removed `treecorr.corr2` — breaks `mccd_plots_runner`, `random_cat_runner`, and their packages. Upstream issue; fixable by bumping stile or constraining treecorr.
  • `summary_run -h` treats `-h` as the `patch` positional and tries to `os.mkdir('-h')`. Argparse-adjustment needed in `scripts/python/summary_run.py`.

Plumbing fixes

  • `pyproject.toml` — `testpaths = ["shapepipe"]` was stale from before the `src` layout migration. Fixed to `["tests", "src/shapepipe/tests"]` so pytest actually discovers these.
  • New workflow `.github/workflows/ci-dev.yml` runs the unit suite on PRs to `develop` using the published `ghcr.io/cosmostat/shapepipe:develop` image. The existing `ci-release.yml` keeps gating main/master unchanged.

Local run

Inside the shapepipe container:
```
pytest tests/unit/ -v --no-cov -p no:warnings

196 passed, 3 skipped, 7 xfailed in 5.16s

```
(The 3 skips are `canfar_*` commands absent from an older local container; present in the CI image.)

Test plan

  • CI (develop) workflow runs green on this PR
  • Merge; verify next PR to develop triggers the workflow

🤖 Generated with Claude Code

cailmdaley and others added 3 commits April 21, 2026 09:07
…trypoints, runner metadata

- tests/unit/test_shell_syntax.py: bash -n over every scripts/sh/*.{sh,bash}
- tests/unit/test_config_parse.py: configparser over example/**/*.ini
- tests/unit/test_imports.py: import every shapepipe.* submodule
- tests/unit/test_entrypoints.py: invoke -h on every [project.scripts] entry
- tests/unit/test_runner_metadata.py: every *_runner.py exports a @module_runner

Pre-existing failures tracked as xfail so the suite lands green and the
issues remain discoverable (strict=True auto-notifies once fixed):
- stile v0.1 imports removed treecorr.corr2 → breaks mccd_plots_runner,
  random_cat_runner, plus their packages
- summary_run -h treats '-h' as the 'patch' positional and mkdirs it

Also:
- Fix stale testpaths (pyproject.toml) — was "shapepipe" from the old
  pre-src-layout era, so pytest never discovered anything new
- New CI workflow ci-dev.yml gates PRs to develop using the published
  ghcr.io/cosmostat/shapepipe:develop image (ci-release.yml stays as-is
  for main/master)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The fresh CI container reveals more pre-existing issues than our older
local container did. Mark them strict-xfail with reasons; GH issues
track the actual fixes.

New xfails:
- shapepipe.canfar.canfar_monitor + shapepipe.canfar_run: IndentationError
  in canfar_monitor.py:55 (docstring at 7 spaces, body at 8)
- mask + mask_runner: astroquery not in develop Docker image
- ngmix + ngmix_runner: numba not in develop Docker image
- split_exp + split_exp_runner: pkg_resources removed in newer setuptools
- uncompress_fits + uncompress_fits_runner: fitsio not installed
- canfar_submit_job, canfar_monitor, canfar_monitor_log entry points:
  transitive on the canfar_monitor.py IndentationError

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Committed inadvertently via `git add -A`. The file is a local
Claude-Code-specific instruction sheet, not meant for the repo.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cailmdaley
Copy link
Copy Markdown
Contributor Author

CI caught more failures than the local pre-#702 container did — all expected, all now xfailed. Filed issues for follow-up:

These are all pre-existing on develop — the scaffolding exposed them. Each xfail carries a `reason=` pointing at the issue so the linkage stays visible. `strict=True` auto-flips to XPASSED when any of these get fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant