Skip to content

ci: split AD test group into AD and ADBatch - #375

Merged
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:ci/split-ad-group
Oct 4, 2026
Merged

ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:ci/split-ad-group

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Sep 28, 2026 •

Copy link
Copy Markdown
Member

Summary

CI's AD group was taking ~76 minutes while other groups finish in 5–17 minutes. Local LTS timings showed derivative_tests.jl dominated (~105 min wall) with OOP+IIP (~53 min) and BatchIntegralFunction cases (~53 min) nearly equal, so this splits them into AD and ADBatch (both still versions = ["lts"]) without changing any assertions.

Shared setup lives in test/shared/ad_derivative_setup.jl (not a group folder). nested_ad_tests.jl stays in AD.

Verification

Per-file LTS timings (AD env, julia +lts):

  • nested_ad_tests.jl: WALL_SECONDS=199.26 (Pass 6)
  • derivative_tests.jl (pre-split): WALL_SECONDS=6325.04 (Pass 2005, Broken 1176)

Section timeline (UTC) for the pre-split file supported an OOP+IIP vs Batch split.

GROUP=AD julia +lts --project -e 'using Pkg; Pkg.test()'
# AD/derivative_tests.jl | Pass 1285 Broken 744 Total 2029  62m33.0s
# AD/nested_ad_tests.jl  | Pass 6 Total 6  2m31.6s
# Testing Integrals tests passed
# WALL_SECONDS=4375.77

GROUP=ADBatch julia +lts --project -e 'using Pkg; Pkg.test()'
# ADBatch/derivative_batch_tests.jl | Pass 720 Broken 432 Total 1152  46m02.3s
# Testing Integrals tests passed
# WALL_SECONDS=2843.28

Union check: SciMLTesting discovery runs AD/{derivative_tests,nested_ad_tests}.jl and ADBatch/derivative_batch_tests.jl; @info/@testset labels match master derivative_tests.jl; pass+broken counts 1285+720=2005 and 744+432=1176 match the pre-split file.

Not verified

  • Full CI matrix on GitHub Actions
  • GROUP=All / Core / QA after this change
  • Whether AD SIGSEGV on Julia 1.12+ still reproduces (reported separately in the agent REPORT, not this PR)

Please ignore this draft until reviewed by @ChrisRackauckas.

Risk assessment

  • Risk: Low — test layout/CI matrix only; no production code or assertion changes.
  • Blast radius: AD CI jobs only (one new matrix row ADBatch on LTS).
  • Evidence: Local GROUP=AD and GROUP=ADBatch Pkg.test() both passed; assertion counts union to the original AD file.
  • Independent review: pending
  • Merge: needs review

Harness: Cursor Agent CLI 2026.09.26-dd393fe; model: auto; transcript: /home/crackauc/sandbox/goals/qa-hygiene/jobs/integrals-ad-split/log.txt

Made with Cursor

Risk assessment

  • Risk: low
  • Blast radius: Test suite and CI matrix only. No src/ or Project.toml changes, so package users are unaffected. The only CI change is one new LTS-only matrix row, ADBatch. One small maintenance cost: test/ADBatch/Project.toml is a byte-for-byte copy of test/AD/Project.toml, so compat bumps now have to be made in both files.
  • Evidence: All 15 checks on head ef9a77ab pass (merge ref 576c36f against base 01dc4d50, which is still master HEAD). There are no failing checks, so nothing is a regression.
    • The last master CI run (2026-09-13) was also all green, with Tests - AD (Julia lts) taking 76 min.
    • On head, Tests - AD (Julia lts) passes in 23m29s with derivative_tests.jl at Pass 1285 / Broken 744 and nested_ad_tests.jl at Pass 6. Tests - ADBatch (Julia lts) passes in 18m38s with Pass 720 / Broken 432.
    • Those add up to 2005 pass and 1176 broken, the same as the original single file per the PR description.
    • I diffed the moved code mechanically. The removed setup block is identical to test/shared/ad_derivative_setup.jl, and the removed batch loops are identical to test/ADBatch/derivative_batch_tests.jl. No assertions, tolerances, or @test_broken lines changed.
    • The Core log shows test/shared/ is not picked up as a test file or group.
    • Core (julia pre) "passed" in 32s because the shared workflow skips the pre lane when the Julia build is not a release candidate (1.13.1 here). The PR has nothing to do with that skip.
  • Independent review: Devin CLI (Mac) / fusion-claude-opus-5-5-high-sidekick-swe-2-medium rated it low, high confidence: it is a verbatim, assertion-preserving split of one test file into two LTS-only groups. CI is fully green, test counts add up exactly, and no public API or dependency internals change (Mooncake.rdata_type and Integrals.ReCallVJP/MooncakeVJP were already used in the test code and were only moved).
  • Merge: auto-merge candidate. Caveats:
    • The PR is still marked draft and says "ignore until reviewed by @ChrisRackauckas".
    • The description does not mention the duplicated ADBatch/Project.toml.
    • Its "Not verified" list is out of date: CI has since run Core, QA and the full matrix green.
    • Its attribution footer gives the model only as "auto".

🤖 Risk assessment posted by an AI agent (fleet master) — harness: Devin CLI (Mac) / fusion-claude-opus-5-5-high-sidekick-swe-2-medium; dispatched by Devin CLI 3000.11.3 (Mac) fleet-master caretaker, model Fusion (claude-opus-5-5 medium + swe-2 medium)
Conversation: local transcript ~/.local/share/devin/cli/transcripts/speckle-whale.json (Chris's Mac)

The AD lane was ~76 minutes in CI; OOP+IIP and BatchIntegralFunction cases are each about half that, so move batch cases into ADBatch (same LTS-only matrix) to parallelize without changing assertions.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Cursor Agent <noreply@cursor.com>
Agent-Harness: Cursor Agent CLI 2026.09.26-dd393fe
Agent-Model: auto
Agent-Session: local session, transcript at /home/crackauc/sandbox/goals/qa-hygiene/jobs/integrals-ad-split/log.txt on amdci2
@ChrisRackauckas
ChrisRackauckas marked this pull request as ready for review October 4, 2026 14:20
@ChrisRackauckas
ChrisRackauckas merged commit 63b17e8 into SciML:master Oct 4, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants