Skip to content

[Bughunt Phase 1] CI cannot currently fail on a correctness regression: no assertions, no sanitizer job, no warning budget #291

Description

@grzanka

Follow-up from a full bughunt/architecture review. Every bug filed in the companion Phase 0 issue would have been caught automatically by the CI hardening described here — right now nothing in CI can detect a wrong numerical result, a memory leak, or a new compiler warning.

Current state, verified

  • test/C/AT_test.c is 78 lines of printf, zero assertions, and an unconditional return EXIT_SUCCESS. It exercises 4 of the library's 382 public functions.
  • There is no enable_testing() / add_test() anywhere in the tree, so ctest runs nothing even if invoked.
  • AT_run_CPPSC_method, AT_run_CPPSS_method, AT_run_GSM_method, AT_run_IGK_method — the library's four flagship algorithms — have zero test or example coverage. (SPISS's total non-functionality, filed separately, went undetected because of this.)
  • Building with -Wall -Wextra -Wshadow -Wuninitialized -Wformat=2 -Wpointer-arith -Wcast-qual -Wstrict-prototypes -Wmissing-prototypes -Wsign-compare -Wfloat-equal produces 268 warnings: 163 -Wstrict-prototypes, 59 -Wunused-parameter, 14 -Warray-parameter= (header/source array-bound mismatches, e.g. include/AT_MultipleCoulombScattering.h vs .c), 13 -Wmissing-prototypes, 11 -Wshadow, 4 -Wunused-variable, 3 -Wsign-compare (src/AT_SPC.c:348,356,363), 1 -Wmaybe-uninitialized (src/AT_StoppingPowerDataPSTAR.c:1323, E_MeV used uninitialized on an early-return path). None of this fails CI.
  • Building with -std=c2x/-std=c23 fails outright (the empty-parameter-list issue behind GCC 16 not compiling #287) — there's no CI leg that would catch a C23-toolchain regression before users hit it.
  • An ASan+UBSan build of the shipped example/demo reports a real leak (see Phase 0 issue, item 4) that a sanitizer CI job would have caught immediately.
  • example/basic_plots (RDD/ER/LET plots) is built but never run in CI at all.
  • Windows CI builds the examples but never runs them (Fix windows binaries, currently examples are not producing any output #258 — already open, and confirms this gap independently).

What's already working (don't regress this)

CPPSC, GSM and IGK ran clean under ASan/UBSan across the RDD×ER model matrix in my harness — the core numerics are in reasonable shape. The goal here is to make CI prove that, and keep proving it, not to imply the whole codebase is unsound.

Proposed work

  • Add enable_testing() to CMakeLists.txt and add_test() for the test binary; wire a ctest step into linux-reusable.yml
  • Adopt a real assertion library for test/C (Unity or Criterion — both available via the existing vcpkg/apt toolchain) and seed it with:
    • characterization tests pinning current output of AT_run_{CPPSC,GSM,IGK}_method for a fixed field, so future refactors (Phase 3) are safe
    • regression tests for each bug fixed under the Phase 0 issue
    • coverage of the ~30 most-used public functions, not an attempt at all 382
  • Add a Linux CI leg building with -fsanitize=address,undefined and running the (now real) test suite plus example/demo and example/basic_plots
  • Record 268 as the current warning baseline and fail CI on any increase (or, more simply, fix the 268 now — converting empty () parameter lists to (void) is mostly mechanical — and turn on -Werror for the warning classes above)
  • Add a -std=c23/-std=c2x CI leg so toolchain regressions like GCC 16 not compiling #287 are caught before users hit them, not after
  • Make the existing Windows CI job actually run amtrack_test/amtrack_demo (closes Fix windows binaries, currently examples are not producing any output #258)
  • Sweep example/basic_plots -t RDD -s <model> -y <model> across all RDD×ER combinations in CI (informational, not gating — several combinations are intentionally incompatible and already report that cleanly)

Acceptance criteria

  • ctest exercises a real assertion-based suite covering all four flagship algorithms
  • A sanitizer CI leg exists and is green
  • CI fails on new compiler warnings above the recorded baseline
  • A C23 CI leg exists and is green

Related: #287, #258

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions