Skip to content

Make @def and @init hygienic: modules in the generated code, not their names (#356) - #413

Open
ocots wants to merge 5 commits into
mainfrom
feat/hygienic-def-modules
Open

ocots wants to merge 5 commits into
mainfrom
feat/hygienic-def-modules

Conversation

@ocots

@ocots ocots commented Nov 3, 2026

Copy link
Copy Markdown
Member

Closes #356.

Summary

The code generated by @def and @init named CTModels, ExaModels and CTBase by symbol, so it only ran in a module that had bindings with these names: import OptimalControl or using OptimalControl: @def failed with UndefVarError: CTModels not defined. The module objects are now interpolated into the generated expressions, so they resolve where the macros are defined (same mechanism __wrap already used for CTBase.CTException, and the one applied to @Lie in CTLie.jl#43).

  • CTModels, ExaModels and LinearAlgebra are direct dependencies (CTModels = "0.19, 0.20, 0.21", the range used across the ecosystem).
  • The prefix machinery is replaced by three constants in src/defaults.jl: FUN = CTModels, EXA = ExaModels, ERR = CTBase.
  • ext/CTParserExaModels.jl is folded into src/exa_linalg.jl (methods unchanged; still temporary, Align CTParser with CTBase 0.29 / CTModels 0.18 / ExaModels 0.12 #325).

Decisions (ct-conventions v0.10.0)

  • init_prefix is removed too, beyond the literal list of the issue: @init had the identical defect (import OptimalControl + @init fails the same way) and nothing outside CTParser references it. Leaving it would only half-fix the bug.
  • Aqua now sees the ExaModels glue (TST8: an extension escaped it). The deliberate piracy is whitelisted with treat_as_own on the five ExaModels node types, not on the whole module, so a new pirated method elsewhere still fails. The ambiguity check follows the ecosystem pattern (ambiguities=false in test_all, then Aqua.test_ambiguities(CTParser; exclude=...)), excluding the stdlib functions the port extends; those ambiguities are upstream's, ported verbatim, and leave with the file.
  • Tests that can reproduce the bug (TST1, TST3): runtests.jl binds all three names in Main, so a flat test could not fail before the fix. test/test_hygiene.jl uses nested modules where the packages are loaded but the names are unbound (what import OptimalControl leaves), plus a module binding CTModels = 42, ExaModels = "nope", CTBase = nothing. It covers @def (:fun and :exa), @init, and the typed errors raised from generated code.
  • test_prefix.jl is deleted, test_prefix_bis.jl becomes test_backends.jl with a contract on FUN/EXA/ERR and an isdefined guard against the removed API coming back.

Breaking

The prefix API is removed (prefix_fun(!), prefix_exa(!), e_prefix(!), init_prefix(!), the __default_*prefix* functions and the PREFIX_* constants). Documented in CHANGELOG.md and BREAKING.md with a migration block (CHG3). The only external caller found is the archived ProjetLong26/CTTransfo. ExaModels is now loaded with CTParser, including for users of the functional backend only (it is already a dependency of OptimalControl).

Test plan

  • Pkg.test("CTParser"): 15/15 files, 2627 pass, 9 broken (the expected GPU @test_skip), 0 fail
  • test_hygiene.jl fails on main's generated code (module Bare raises UndefVarError), passes here
  • CI (including the GPU runners)
  • Downstream: OptimalControl no longer needs to re-export CTModels/ExaModels/CTBase for the generated code (separate PR there)

Conventions: ct-conventions v0.10.0, ct-agents v0.4.0.

Generated with Devin

ocots and others added 4 commits November 2, 2026 23:12
@def and @init generated code resolved CTModels, ExaModels and CTBase
by name in the caller's module, so `import OptimalControl` or
`using OptimalControl: @def` failed with `UndefVarError` (#356). The
three modules are now direct dependencies and are interpolated as
module objects into the emitted expressions — the same GlobalRef
mechanism __wrap already used for CTBase.CTException — so they resolve
where the macros are defined, never in the caller.

The configurable prefixes had no remaining purpose (only the package's
own tests set them): PREFIX_FUN, PREFIX_EXA, E_PREFIX, INIT_PREFIX,
their getters/setters and the __default_*_prefix functions are removed,
replaced by three constants FUN/EXA/ERR in defaults.jl. init_prefix is
removed too: @init had the same defect and nothing outside CTParser
references it.

ext/CTParserExaModels.jl is folded into src/exa_linalg.jl: ExaModels is
always loaded, so the extension mechanism no longer applies. The port
stays temporary until ExaModels wires its own extension (#325).

Qualified imports of the three modules in src/CTParser.jl follow MOD4
(using X: X); MOD6 already required their call sites qualified, the
change only moves the resolution point.

Refs #356.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
- test/test_prefix.jl is deleted and test_prefix_bis.jl renamed to
  test_backends.jl: the file keeps covering backend activation and the
  parser defaults, and gains the contract for the new FUN / EXA / ERR
  constants — module objects, not symbols — plus an isdefined guard so
  the removed API cannot quietly come back (TST7: specific assertions).
- test/runtests.jl drops the six prefix imports.
- test/test_initial_guess.jl drops the init_prefix setter testset (the
  setter is gone with INIT_PREFIX).
- test/test_aqua.jl now sees the code that lived in the extension:
  Aqua skips extensions (TST8), so neither the deliberate piracy nor
  the upstream ambiguities of the ExaModels linear-algebra port were
  ever checked. In src/ they are. The piracy is whitelisted precisely
  with treat_as_own on the ExaModels node types it ports to — not on
  the whole module — and the ambiguity check runs package-only with
  the stdlib functions the port extends excluded (upstream's own
  ambiguities, ported verbatim; they leave with the file, #325).
- test/Project.toml aligns the CTModels compat on src's.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
test/test_hygiene.jl reproduces the failure mode of #356 inside the
suite: nested modules are namespaces where the packages are loaded but
none of the names CTModels / ExaModels / CTBase is bound — exactly what
`import OptimalControl` leaves a caller with. A flat test file could
not reproduce it: runtests.jl binds all three names in Main.

- module Bare binds only the macros: @def (:fun and :exa builder),
  @init, and the typed errors thrown from generated code (the :exa
  dyn_check via @def_exa, __throw inside @init, the getter's val check)
  all resolve to CTBase.ParsingError / CTBase.IncorrectArgument.
- module Shadowed binds the same names to junk values (CTModels = 42):
  the interpolated module objects must win over any caller binding.

The fixture modules live at the test module's top level (TST1, TST3).

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
CHANGELOG.md records the removed prefix API with a migration block
(CHG3), the fix of #356, the fold of the extension into src/, the new
direct dependencies and the test changes; the compatibility footer now
says the release is breaking instead of claiming an unchanged API.
BREAKING.md mirrors the entry with the same migration block. The docs
index no longer documents prefix_fun/prefix_exa/init_prefix, and
AGENTS.md describes src/exa_linalg.jl instead of ext/ and states that
the generated code refers to the modules by object (FUN/EXA/ERR).

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@github-actions
github-actions Bot requested a review from jbcaillau November 3, 2026 08:09
@ocots ocots added run ci github-runner Trigger the CI workflow on GitHub-hosted runners run ci kkt-runner Trigger the CI workflow on the self-hosted kkt runner labels Nov 3, 2026
@ocots

ocots commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

Review

Reviewed at c5049f0 against ct-conventions v0.10.0 / ct-agents v0.4.0.

Verdict: good to merge once the remaining test-plan items are done. The fix is right: it uses the mechanism __wrap already used for CTBase.CTException, and it is applied everywhere. I grepped src/ and found no expression that still names CTModels, ExaModels or CTBase by symbol; every pref / e_pref is now FUN / EXA / ERR. Removing init_prefix too was the right call: leaving it would have kept the bug in @init.

What I checked

  • Generated code. def_fun, def_exa (builder, dyn_check, getter), __throw, _bound_length_check, _unknown_scheme_error, all p_*_fun! / p_*_exa! and init_fun / __gen_temporal_value interpolate the module objects. The macros still esc the whole expression, which is harmless now that the modules are objects and not names to resolve.
  • Tests. test_hygiene.jl is the test that was missing. It can fail (the Bare module reproduces import OptimalControl), and Shadowed covers the worse case where the caller binds those names to something else. The fixture modules sit at the top level of the test module (TST3).
  • CTModels lower bound. Everything the generated code calls (PreModel, definition!, time_dependence!, build(...; build_examodel=), time_name, build_initial_guess, validate_initial_guess, get_build_examodel) already exists at v0.19.0-beta, so 0.19, 0.20, 0.21 looks sound. CI only resolves the latest version, though, so the lower bound itself is not tested.
  • Aqua. treat_as_own on the five node types is tighter than whitelisting the whole ExaModels module, and it also covers the ExaModels._hdrpass_val method.
  • CI. CPU passes on 1.10 / 1.13 × 3 OS, and test-gpu-kkt passes. test-gpu-main / test-gpu-occidata were skipped, so the GPU box in the test plan is still open.

Minor points (none blocking)

  1. Stale docstring: src/initial_guess.jl:100 (__gen_spec_value) still says - `pref::Symbol`: backend module prefix. It should read pref::Module like the __gen_temporal_value docstring just above it.
  2. The Aqua ambiguity exclusions are broad. exclude=[Base.:*, Base.:+, Base.:-, Base.convert, Base.zero, ...] turns off the ambiguity check for those functions entirely, not just for the methods in exa_linalg.jl. That is fine while the port exists, but these exclusions must be removed together with the file. Could you add that explicitly to Align CTParser with CTBase 0.29 / CTModels 0.18 / ExaModels 0.12 #325 ("delete src/exa_linalg.jl and restore ambiguities=true / drop treat_as_own in test/test_aqua.jl") so that someone cleaning up later does not miss it?
  3. Optional nit: pref = FUN / e_pref = ERR are now plain aliases of constants and could be inlined ($FUN.PreModel()). Keeping them keeps the diff small, so it is up to you.
  4. Optional, test coverage: Shadowed does not exercise __throw (the ParsingError raised at expansion time by @init). Bare.broken_init covers it with the names unbound, but not with the names bound to junk values. A one-line Shadowed.broken_init would make the two modules cover the same cases.

Release note

This PR is breaking, so the next release must be a breaking bump (a minor bump in 0.x). Project.toml still reads 0.8.17-beta on purpose (it gets bumped in the release PR), while the latest tag is v0.9.5-beta, so the release PR should take care to pick the right version. The downstream OptimalControl PR (dropping the re-exports needed by the generated code) can be opened in parallel.

🤖 Generated with Claude Code

…le (#356)

- __gen_spec_value documented pref as a Symbol prefix; it is now a Module.
- test_hygiene: Shadowed gets the same broken_init as Bare, so the
  expansion-time ParsingError is checked with the names bound to junk too.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run ci github-runner Trigger the CI workflow on GitHub-hosted runners run ci kkt-runner Trigger the CI workflow on the self-hosted kkt runner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make @def hygienic: the generated code should contain the modules, not their names

2 participants