Skip to content

test(parser_app): guard the dev Makefile target against argument drift - #481

Open
prasanna-anchorage wants to merge 4 commits into
mainfrom
prasanna/makefile-dev-target-guard
Open

prasanna-anchorage wants to merge 4 commits into
mainfrom
prasanna/makefile-dev-target-guard

Conversation

@prasanna-anchorage

Copy link
Copy Markdown
Contributor

Why

Nothing in CI runs the dev make targets. The ubuntu job is make -C src generated, lint, test, narrow-build-check — so an argument parser_app's CLI stops accepting leaves make -C src parser_app broken with every check green.

That is not hypothetical. The target passed --usock for some time after 751118bd replaced it with --host-ip/--host-port, and panicked before reaching any flag the existing cli.rs tests cover:

thread 'main' panicked at parser/app/src/cli.rs:31:33:
Parser: invalid CLI args: found --usock, which was not an expected argument

It took someone running the target by hand to find (#454), and then a rebase reintroduced it on another branch (#450). Twice, silently, because no automated check looks.

What

One test, parser_app's cli.rs. It extracts the parser_app target's recipe arguments from src/Makefile, substitutes $(VAR) references, and feeds them to the real OptionsParser::<ParserParser> — so it catches drift from either side: a flag removed from the CLI, or a stale flag left in the recipe.

It then resolves the ABI trust posture too. Parsing alone would still pass a target that had lost --accept-unsigned-abis, which Cli::execute refuses to start without.

Two deliberate choices worth flagging for review:

  • include_str! rather than a runtime read, so the Makefile is a compile-time dependency. Moving or renaming it is a build failure, not a guard that quietly stops checking anything.
  • The structural assertions pin only "some arguments, at least one a flag" — not which flags. They exist because an unreadable recipe yields an empty argument list, which parses cleanly and proves nothing. Pinning specific flag names instead would make a legitimate rename fail with a misleading "the extractor broke" message before reaching the parse. I wrote it that way first and it misdiagnosed the --usock case, so the ordering here is deliberate: parse first, and let the failure name the offending argument.

Scope

parser_app only, deliberately. Of the five dev run targets, grpc-server and parser_gateway pass no arguments, parser_enclave passes positionals, and parser_host is qos_core's binary whose CLI is not ours to test. parser_app is the only one with flags this repo owns, and the only one that has actually rotted.

Verification

Mutated the recipe locally to confirm the guard fails for the right reason in each direction, rather than only that it passes as written:

Recipe state Result
as written passes
--usock restored fails: UnexpectedInput("--usock"), naming the argument
--accept-unsigned-abis dropped fails on the posture check
target renamed out from under it fails structurally, not vacuously

make -C src fmt, lint (including -p parser_app --no-default-features, so narrow-build-check is unaffected) and test (39 suites, 1674 tests, 0 failures) all green.

Base

Stacked on #454 because that PR carries the --usock fix; on main this guard would correctly fail, since main's target still has the broken flag. GitHub will retarget this to main when #454 merges. #454's own approval is untouched — this is a child branch, not a push to prs-556-parser-app.

Nothing in CI runs the dev `make` targets -- `make {generated,lint,test,
narrow-build-check}` is the whole of the ubuntu job -- so an argument
`parser_app`'s CLI stops accepting leaves `make -C src parser_app` broken with
every check green.

That is not hypothetical. The target passed `--usock` for some time after
751118b replaced it with `--host-ip`/`--host-port`, and panicked before
reaching any flag the existing tests cover:

    thread 'main' panicked at parser/app/src/cli.rs:31:33:
    Parser: invalid CLI args: found --usock, which was not an expected argument

It took someone running the target by hand to find, and a rebase to
reintroduce it.

This feeds the Makefile's own arguments to the real `OptionsParser`, so it
catches drift from either side: a flag removed from the CLI, or a stale flag
left in the recipe. It then resolves the ABI trust posture as well, since
parsing alone would still pass a target that had lost the posture flag
`Cli::execute` refuses to start without.

`include_str!` makes the Makefile a compile-time dependency, so moving or
renaming it is a build failure rather than a guard that quietly stops
checking anything. The structural assertions exist for the same reason: an
unreadable recipe yields an empty argument list, which would parse cleanly
and prove nothing. They deliberately pin only "some arguments, at least one
a flag" and not *which* flags, so a legitimate rename fails on the parse --
naming the offending argument -- rather than on a misleading "the extractor
broke" message.

Verified all four directions by mutating the recipe locally: passes as
written; fails with `UnexpectedInput("--usock")` when the old flag is
restored; fails on the posture check when `--accept-unsigned-abis` is
dropped; fails structurally when the target is renamed out from under it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, focused addition — single file, a new test plus two small private helpers that guard make parser_app's dev target against argument drift. I traced the Makefile-parsing logic against the actual parser_app target in src/Makefile line-by-line and it correctly extracts the argv; clippy is clean and the test passes. Good use of a real regression case (the --usock/--host-ip/--host-port flag rename) to justify why this guard is worth having, and it fills a real gap — nothing else in CI exercises this target's actual argv.

One non-blocking note for follow-up, not this PR:

The Makefile-recipe parsing (makefile_target_args/substitute_make_vars) is a bespoke text scraper — it doesn't handle conditionals, $(shell ...), or more than one -- separator. If the parser_app recipe ever grows any of those, this helper would likely mis-extract tokens silently rather than fail loudly, which undermines the drift-detection this test is meant to provide. Not asking for a rewrite here, just flagging it in case the recipe gets more complex later — a shared PARSER_APP_ARGS-style variable referenced by both the recipe and the test would make this airtight.

@prasanna-anchorage
prasanna-anchorage enabled auto-merge (squash) September 13, 2026 21:14
Copilot AI lite review requested due to automatic review settings September 13, 2026 22:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Two moderate issues remain in argument parsing and host validation coverage.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds a regression test ensuring the parser_app Makefile target remains compatible with its CLI and ABI trust requirements.

Changes:

  • Extracts and substitutes Makefile recipe arguments.
  • Validates them with the real CLI parser.
  • Adds structural and ABI posture checks.
File summaries
File Description
src/parser/app/src/cli.rs Adds Makefile target extraction and CLI compatibility testing.
Review details

Suppressed comments (1)

src/parser/app/src/cli.rs:328

  • When an unbalanced $(...) reference occurs after a prefix, out already contains the prefix, but this break leaves rest unchanged and line 337 appends the entire token again (foo$(VAR becomes foofoo$(VAR). That contradicts the helper's stated 'leave it verbatim' behavior and can make the parser validate different arguments from the recipe; append only &rest[start..] and return, or otherwise advance rest before breaking.
            let Some(end) = after.find(')') else {
                // Unbalanced `$(`: leave it verbatim rather than guessing, so the
                // caller's parse fails loudly instead of on a silently mangled arg.
                break;
  • Files reviewed: 1/1 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/parser/app/src/cli.rs Outdated
abi_trust() never touches --host-ip/--host-port, so a malformed host or
port in the parser_app recipe would still pass this guard even though
Cli::execute's own host_addr() call would panic on it at runtime -- the
same bug class --usock was before it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@prasanna-anchorage

Copy link
Copy Markdown
Contributor Author

Addressed Copilot's finding in a352b462: the guard now calls host_addr() after abi_trust(), so a malformed --host-ip/--host-port in the recipe is caught the same way a missing posture flag already was.

Verified by mutation, not just that it compiles:

  • As-is: passes.
  • Mutated the recipe's --host-ip $(PARSER_HOST) to a literal --host-ip localhost: now fails with could not parse ip to IP v4: AddrParseError(Ipv4) at cli.rs:39, confirming the new call actually catches it. Reverted after.

cargo test -p parser_app --lib 42/42, cargo clippy -p parser_app --all-targets -- -D warnings clean, cargo fmt -p parser_app -- --check clean.

No new test needed — this extends the existing guard's own assertion rather than adding a parallel one, since the thing being verified (the target's actual startup path) is the same.

@prasanna-anchorage

Copy link
Copy Markdown
Contributor Author

@shahan-khatchadourian-anchorage — your approval got dismissed by a352b462 (the Copilot-thread fix above). Re-approval needed to unblock; the only change since your last review is that one-line host_addr() addition, mutation-verified in the comment above. Everything else you already reviewed is untouched.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @prasanna-anchorage

prasanna-anchorage added a commit that referenced this pull request Sep 15, 2026
* ci: cover the tvc-deploy workspace on pull requests

`tools/tvc-deploy` is its own cargo workspace, so `src/Cargo.toml`'s
workspace lints and `make -C src lint|test` cannot reach it, and no
pull_request-triggered workflow built it. Its tests ran only inside
`tvc-deploy.yml`'s deploy job, which is gated on `workflow_dispatch` or the
`tvc-deploy-test` label -- so a PR that changed the deploy helper had no CI
coverage of it at all.

This is not hypothetical: **56 tests already on `main` never ran in PR CI**,
and #455 took the crate to 65 while adding 258 lines of posture logic (the
XOR, on-curve pubkey validation, pivotArgs composition) that CI never
exercised. It is the same class as the two gaps already paid for -- the
parser_cli narrow build (#480, which surfaced a fixture bug latent for five
days) and `make parser_app` argument drift (#481, after that target broke
silently twice).

Adds a lint+test step to `main.yml`'s `ubuntu` job. `--locked` for the same
reason the narrow-build checks use it: the crate keeps its own Cargo.lock, so
without it a stale lock is silently updated in CI instead of failing. Verified
`--locked` passes both before and after #455, so the step is safe regardless of
merge order. Cheap: the crate builds in seconds and the tests run in 0.01s.

Also makes the deploy workflow's posture resolution explicit on every path.
#455 added a fail-closed guard for `workflow_dispatch`, but on a
`pull_request` labeled `tvc-deploy-test` the `abi_signer_pubkey` input does not
exist, so the guard (ANDed with the event name) did not fire and a blank value
fell through to `--accept-unsigned-abis` with no signal. Bounded -- `APP_ID`
falls back to `vars.TVC_TEST_APP_ID`, the dev test app, so it was never a
production exposure -- but it meant the only end-to-end deploy validation
always exercised the permissive posture, never require-signed: the one where a
wrong key or bad pivotArgs makes parser_app refuse to start, i.e. the one worth
validating. It is also the exact behaviour `abi_signer_pubkey`'s own
description gives as the reason it has no default.

Now: an explicit input still wins; a configured `TVC_TEST_ABI_SIGNER` repo
variable makes the label path validate require-signed; and with neither, the
permissive fallback still works but announces itself via `::notice::`. The
dispatch guard stays first on purpose -- `app_id` is free text there, so a
blank posture must not resolve to anything, not even the test signer, which
would push a test key at whatever app was named.

Verified by driving the extracted shell through all eight input x event
combinations, including that dispatch-with-blank still exits 1 when the
variable *is* set. Triggers and the job's `if` are unchanged, both workflows
re-parse as YAML, and the new step's own commands pass (clippy exit 0, 65
tests).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* ci: derive the validation deploy's signer instead of configuring it

Replaces the `TVC_TEST_ABI_SIGNER` repo variable this PR originally leaned on.
A variable someone has to set is a weak dependency: until an admin sets it the
label-triggered deploy keeps validating only the permissive posture, and
nothing fails to say so.

The repo already has a deterministic answer. `visualsign-ethereum`'s
`CLI_DEV_SIGNING_KEY_SEED` is a committed constant and `parser_cli` already
signs its own ABI files with that key, so its public half is derivable with no
secret, no variable and no admin action. Derived it, checked it through
`tvc-deploy`'s own `validate_signer_pubkey` (accepted: 65 bytes, SEC1 `04`,
on-curve), and pinned it as `DEV_ABI_SIGNER_PUBKEY` in the workflow. So the
label path now always exercises *require-signed* rather than falling back to
permissive, and the `::notice::` apologising for the fallback is gone with it.

A workflow cannot import a Rust constant, so the hex is duplicated there.
`dev_abi_signer_pubkey_is_pinned` is what stops that copy going stale: rotate
the seed and it fails, printing the replacement value to paste in. Without it
the drift would be silent *and* green -- the post-deploy smoke is a Solana
V0+ALT transaction, which no Ethereum ABI posture can affect, so CI would keep
passing while deploying a posture that allowlists a key nothing signs with.
Mutation-checked: changing the seed to `[0x43; 32]` fails the test with the
correct new value in the message.

Scope stays deliberate. Only the label path takes the derived key, and only
because it deploys to the dev TEST app (`vars.TVC_TEST_APP_ID`); the comment
says not to point a real app at a well-known dev key. `workflow_dispatch` is
untouched and still fail-closed first -- a blank posture there resolves to
nothing, not even to this key, because `app_id` is free text. `TVC_TEST_ABI_SIGNER`
survives as an override for anyone who wants a different signer.

Re-verified all eight input x event combinations after the change, including
that dispatch-with-blank still exits 1 when the override variable is set.
`make lint` 0, `make test` 0 (44 suites / 2223 tests), tvc-deploy clippy 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants