test(parser_app): guard the dev Makefile target against argument drift - #481
prasanna-anchorage wants to merge 4 commits into
Conversation
Nothing in CI runs the dev `make` targets -- `make {generated,lint,test,
narrow-build-check}` is the whole of the ubuntu job -- so an argument
`parser_app`'s CLI stops accepting leaves `make -C src parser_app` broken with
every check green.
That is not hypothetical. The target passed `--usock` for some time after
751118b replaced it with `--host-ip`/`--host-port`, and panicked before
reaching any flag the existing tests cover:
thread 'main' panicked at parser/app/src/cli.rs:31:33:
Parser: invalid CLI args: found --usock, which was not an expected argument
It took someone running the target by hand to find, and a rebase to
reintroduce it.
This feeds the Makefile's own arguments to the real `OptionsParser`, so it
catches drift from either side: a flag removed from the CLI, or a stale flag
left in the recipe. It then resolves the ABI trust posture as well, since
parsing alone would still pass a target that had lost the posture flag
`Cli::execute` refuses to start without.
`include_str!` makes the Makefile a compile-time dependency, so moving or
renaming it is a build failure rather than a guard that quietly stops
checking anything. The structural assertions exist for the same reason: an
unreadable recipe yields an empty argument list, which would parse cleanly
and prove nothing. They deliberately pin only "some arguments, at least one
a flag" and not *which* flags, so a legitimate rename fails on the parse --
naming the offending argument -- rather than on a misleading "the extractor
broke" message.
Verified all four directions by mutating the recipe locally: passes as
written; fails with `UnexpectedInput("--usock")` when the old flag is
restored; fails on the posture check when `--accept-unsigned-abis` is
dropped; fails structurally when the target is renamed out from under it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
shahan-khatchadourian-anchorage
left a comment
There was a problem hiding this comment.
Nice, focused addition — single file, a new test plus two small private helpers that guard make parser_app's dev target against argument drift. I traced the Makefile-parsing logic against the actual parser_app target in src/Makefile line-by-line and it correctly extracts the argv; clippy is clean and the test passes. Good use of a real regression case (the --usock/--host-ip/--host-port flag rename) to justify why this guard is worth having, and it fills a real gap — nothing else in CI exercises this target's actual argv.
One non-blocking note for follow-up, not this PR:
The Makefile-recipe parsing (makefile_target_args/substitute_make_vars) is a bespoke text scraper — it doesn't handle conditionals, $(shell ...), or more than one -- separator. If the parser_app recipe ever grows any of those, this helper would likely mis-extract tokens silently rather than fail loudly, which undermines the drift-detection this test is meant to provide. Not asking for a rewrite here, just flagging it in case the recipe gets more complex later — a shared PARSER_APP_ARGS-style variable referenced by both the recipe and the test would make this airtight.
There was a problem hiding this comment.
🟡 Changes recommended
Two moderate issues remain in argument parsing and host validation coverage.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Adds a regression test ensuring the parser_app Makefile target remains compatible with its CLI and ABI trust requirements.
Changes:
- Extracts and substitutes Makefile recipe arguments.
- Validates them with the real CLI parser.
- Adds structural and ABI posture checks.
File summaries
| File | Description |
|---|---|
src/parser/app/src/cli.rs |
Adds Makefile target extraction and CLI compatibility testing. |
Review details
Suppressed comments (1)
src/parser/app/src/cli.rs:328
- When an unbalanced
$(...)reference occurs after a prefix,outalready contains the prefix, but thisbreakleavesrestunchanged and line 337 appends the entire token again (foo$(VARbecomesfoofoo$(VAR). That contradicts the helper's stated 'leave it verbatim' behavior and can make the parser validate different arguments from the recipe; append only&rest[start..]and return, or otherwise advancerestbefore breaking.
let Some(end) = after.find(')') else {
// Unbalanced `$(`: leave it verbatim rather than guessing, so the
// caller's parse fails loudly instead of on a silently mangled arg.
break;
- Files reviewed: 1/1 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
abi_trust() never touches --host-ip/--host-port, so a malformed host or port in the parser_app recipe would still pass this guard even though Cli::execute's own host_addr() call would panic on it at runtime -- the same bug class --usock was before it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Addressed Copilot's finding in Verified by mutation, not just that it compiles:
No new test needed — this extends the existing guard's own assertion rather than adding a parallel one, since the thing being verified (the target's actual startup path) is the same. |
|
@shahan-khatchadourian-anchorage — your approval got dismissed by |
shahan-khatchadourian-anchorage
left a comment
There was a problem hiding this comment.
LGTM. Thanks @prasanna-anchorage
* ci: cover the tvc-deploy workspace on pull requests `tools/tvc-deploy` is its own cargo workspace, so `src/Cargo.toml`'s workspace lints and `make -C src lint|test` cannot reach it, and no pull_request-triggered workflow built it. Its tests ran only inside `tvc-deploy.yml`'s deploy job, which is gated on `workflow_dispatch` or the `tvc-deploy-test` label -- so a PR that changed the deploy helper had no CI coverage of it at all. This is not hypothetical: **56 tests already on `main` never ran in PR CI**, and #455 took the crate to 65 while adding 258 lines of posture logic (the XOR, on-curve pubkey validation, pivotArgs composition) that CI never exercised. It is the same class as the two gaps already paid for -- the parser_cli narrow build (#480, which surfaced a fixture bug latent for five days) and `make parser_app` argument drift (#481, after that target broke silently twice). Adds a lint+test step to `main.yml`'s `ubuntu` job. `--locked` for the same reason the narrow-build checks use it: the crate keeps its own Cargo.lock, so without it a stale lock is silently updated in CI instead of failing. Verified `--locked` passes both before and after #455, so the step is safe regardless of merge order. Cheap: the crate builds in seconds and the tests run in 0.01s. Also makes the deploy workflow's posture resolution explicit on every path. #455 added a fail-closed guard for `workflow_dispatch`, but on a `pull_request` labeled `tvc-deploy-test` the `abi_signer_pubkey` input does not exist, so the guard (ANDed with the event name) did not fire and a blank value fell through to `--accept-unsigned-abis` with no signal. Bounded -- `APP_ID` falls back to `vars.TVC_TEST_APP_ID`, the dev test app, so it was never a production exposure -- but it meant the only end-to-end deploy validation always exercised the permissive posture, never require-signed: the one where a wrong key or bad pivotArgs makes parser_app refuse to start, i.e. the one worth validating. It is also the exact behaviour `abi_signer_pubkey`'s own description gives as the reason it has no default. Now: an explicit input still wins; a configured `TVC_TEST_ABI_SIGNER` repo variable makes the label path validate require-signed; and with neither, the permissive fallback still works but announces itself via `::notice::`. The dispatch guard stays first on purpose -- `app_id` is free text there, so a blank posture must not resolve to anything, not even the test signer, which would push a test key at whatever app was named. Verified by driving the extracted shell through all eight input x event combinations, including that dispatch-with-blank still exits 1 when the variable *is* set. Triggers and the job's `if` are unchanged, both workflows re-parse as YAML, and the new step's own commands pass (clippy exit 0, 65 tests). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ci: derive the validation deploy's signer instead of configuring it Replaces the `TVC_TEST_ABI_SIGNER` repo variable this PR originally leaned on. A variable someone has to set is a weak dependency: until an admin sets it the label-triggered deploy keeps validating only the permissive posture, and nothing fails to say so. The repo already has a deterministic answer. `visualsign-ethereum`'s `CLI_DEV_SIGNING_KEY_SEED` is a committed constant and `parser_cli` already signs its own ABI files with that key, so its public half is derivable with no secret, no variable and no admin action. Derived it, checked it through `tvc-deploy`'s own `validate_signer_pubkey` (accepted: 65 bytes, SEC1 `04`, on-curve), and pinned it as `DEV_ABI_SIGNER_PUBKEY` in the workflow. So the label path now always exercises *require-signed* rather than falling back to permissive, and the `::notice::` apologising for the fallback is gone with it. A workflow cannot import a Rust constant, so the hex is duplicated there. `dev_abi_signer_pubkey_is_pinned` is what stops that copy going stale: rotate the seed and it fails, printing the replacement value to paste in. Without it the drift would be silent *and* green -- the post-deploy smoke is a Solana V0+ALT transaction, which no Ethereum ABI posture can affect, so CI would keep passing while deploying a posture that allowlists a key nothing signs with. Mutation-checked: changing the seed to `[0x43; 32]` fails the test with the correct new value in the message. Scope stays deliberate. Only the label path takes the derived key, and only because it deploys to the dev TEST app (`vars.TVC_TEST_APP_ID`); the comment says not to point a real app at a well-known dev key. `workflow_dispatch` is untouched and still fail-closed first -- a blank posture there resolves to nothing, not even to this key, because `app_id` is free text. `TVC_TEST_ABI_SIGNER` survives as an override for anyone who wants a different signer. Re-verified all eight input x event combinations after the change, including that dispatch-with-blank still exits 1 when the override variable is set. `make lint` 0, `make test` 0 (44 suites / 2223 tests), tvc-deploy clippy 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Why
Nothing in CI runs the dev
maketargets. Theubuntujob ismake -C src generated,lint,test,narrow-build-check— so an argumentparser_app's CLI stops accepting leavesmake -C src parser_appbroken with every check green.That is not hypothetical. The target passed
--usockfor some time after751118bdreplaced it with--host-ip/--host-port, and panicked before reaching any flag the existingcli.rstests cover:It took someone running the target by hand to find (#454), and then a rebase reintroduced it on another branch (#450). Twice, silently, because no automated check looks.
What
One test,
parser_app'scli.rs. It extracts theparser_apptarget's recipe arguments fromsrc/Makefile, substitutes$(VAR)references, and feeds them to the realOptionsParser::<ParserParser>— so it catches drift from either side: a flag removed from the CLI, or a stale flag left in the recipe.It then resolves the ABI trust posture too. Parsing alone would still pass a target that had lost
--accept-unsigned-abis, whichCli::executerefuses to start without.Two deliberate choices worth flagging for review:
include_str!rather than a runtime read, so the Makefile is a compile-time dependency. Moving or renaming it is a build failure, not a guard that quietly stops checking anything.--usockcase, so the ordering here is deliberate: parse first, and let the failure name the offending argument.Scope
parser_apponly, deliberately. Of the five dev run targets,grpc-serverandparser_gatewaypass no arguments,parser_enclavepasses positionals, andparser_hostis qos_core's binary whose CLI is not ours to test.parser_appis the only one with flags this repo owns, and the only one that has actually rotted.Verification
Mutated the recipe locally to confirm the guard fails for the right reason in each direction, rather than only that it passes as written:
--usockrestoredUnexpectedInput("--usock"), naming the argument--accept-unsigned-abisdroppedmake -C src fmt,lint(including-p parser_app --no-default-features, sonarrow-build-checkis unaffected) andtest(39 suites, 1674 tests, 0 failures) all green.Base
Stacked on #454 because that PR carries the
--usockfix; onmainthis guard would correctly fail, sincemain's target still has the broken flag. GitHub will retarget this tomainwhen #454 merges. #454's own approval is untouched — this is a child branch, not a push toprs-556-parser-app.