Skip to content

scripts/assert-file — file assertions for checks, without the nested-quoting minefield - #131

Open
bluemihai wants to merge 3 commits into
NateBJones-Projects:mainfrom
bluemihai:assert-file-helper
Open

bluemihai wants to merge 3 commits into
NateBJones-Projects:mainfrom
bluemihai:assert-file-helper

Conversation

@bluemihai

Copy link
Copy Markdown

Why

A Ringer check lives inside a JSON string, inside a shell command. Every hand-written ruby -e '...' or python3 -c '...' assertion therefore passes through JSON escaping, then shell quoting, then the target language's string parser, before its regex is even compiled.

On one day I lost four consecutive checks to that stack — and in every case the check was wrong, not the work:

  • one aborted because the banned string it grepped for appeared in the explanatory comment the brief had asked the worker to write;
  • two carried \" sequences that reached Ruby as a syntax error, or as a literal that could never match;
  • one asserted on a notes.md belonging to a different task entirely.

A wall of red from broken checks reads as a broken system rather than a careful one, and it teaches workers to bend correct code until the grep goes quiet.

What this adds

scripts/assert-file, a small stdlib-only helper so that class of bug cannot be written again — no nested quoting, comments stripped by file type, and a non-zero exit that says which assertion failed and what it actually saw.

assert-file PATH --absent PATTERN [...]     pattern must NOT appear
assert-file PATH --contains PATTERN [...]   pattern MUST appear
assert-file PATH --count PATTERN=N          pattern appears exactly N times
assert-file PATH --before A B               A must appear before B

--strip-comments is on by default (--raw opts out), which is the fix for the first failure above: a brief that asks a worker to justify itself in a comment shouldn't hand the check a false positive. Patterns are literal by default; --regex when you mean it. --label prefixes the failure message so a multi-assertion check says which one broke.

Tests

tests/test_assert_file.py, six tests, each running the real script as a subprocess — the thing under test is the command-line contract, and asserting on an imported function would prove something no check exercises.

Covers contains/absent on one file, comment-stripping plus its --raw opt-out, exact counts, ordering (--before catches order, not mere presence), a failure message that names the pattern it could not find, and — the one that matters most for check honesty — a missing file failing rather than passing vacuously.

Full suite green on this branch: 261 passed.

🤖 Generated with Claude Code

bluemihai and others added 3 commits September 10, 2026 04:22
… minefield

On 2026-09-09 four consecutive Ringer checks failed CORRECT worker output in
one evening. Two never executed at all: a \" sequence reached Ruby as
"unterminated string; expected a closing delimiter" after passing through JSON
escaping, shell quoting and Ruby string parsing. One aborted on a banned string
that appeared in the explanatory comment the brief had *demanded* the worker
write. One asserted on a notes.md belonging to a different task.

The check is the product — the skill says so — and it is the one artifact in
the loop that nothing checks. This removes the class: comments stripped by file
type (ERB <%# %> first, the exact shape that failed), literal matching so
nothing needs escaping, and failures that report the real count and line number
instead of a bare abort.

Red/green tested against all four real cases, including proving --raw still
fails the file that comment-stripping passes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
First-write bug, caught by running the tool rather than trusting it. DOTALL
was applied to every comment pattern, so the LINE form ^\s*#.*$ swallowed the
whole file from its first comment: a 5979-byte Ruby controller stripped to 175
bytes, and every --contains after that reported a false absence.

Only ERB was exercised before shipping, and ERB uses a block comment, so the
broken path never ran. Comment patterns now carry their own spans-newlines
flag; tested across .rb, .erb and .js, including that a comment-only string is
still stripped and that --raw still fails the file comment-stripping passes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Six subprocess tests, because the thing under test IS the CLI contract — a
check invokes it from a shell and reads its exit code. Covers contains/absent,
comment-stripping (the originating bug: a worker's own explanatory comment
tripping a banned-string grep) and its --raw opt-out, exact counts, ordering,
a failure message that names the pattern, and a missing file failing rather
than passing vacuously.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant