Redact credentials before captured worker output is persisted or served

[bluemihai]

The problem

A live Resend API key was found sitting in plaintext in ~/.ringer/runs/*.json on my machine.

The mechanism is general, and not specific to any one language or service: when a test suite's HTTP mock blocks an outbound call, its failure message prints the blocked request's full headers, including Authorization: Bearer <token>. That failure message is the check command's stdout — and Ringer stores check stdout verbatim in the run record and serves worker log tails to the dashboard. So any credential the worker's environment happens to hold can land on disk in cleartext.

Distinct from the existing redact_spec flag, which hides the request packet you sent from the dashboard. This is about credentials appearing in output coming back.

What this adds

redact_secrets(text) replaces credential values with [REDACTED] while keeping key names and surrounding structure, so the text stays diagnostic — it is injected into the retry prompt, so making it unreadable would cost more than it saves. Covers bearer tokens in any quoting style, known credential prefixes (sk-ant-, sk-, re_, ghp_/gho_/ghu_/ghs_, github_pat_, xox*, glpat-, AWS AKIA…, Stripe sk_live_/rk_test_…), and generic api_key= / token: / secret => assignments.

It is idempotent, byte-identical on clean input, and fails closed: if redaction itself raises, the input is dropped rather than returned. Returning it would hand back the exact cleartext the function exists to remove, silently, on the one path nobody watches.

Wired at the three funnels where captured output escapes into something persisted or served:

• the Verifier's check excerpt (which feeds check_output_tail, check_excerpt, the notes and the retry prompt)
• tail_lines (feeds log_tail / log_tail_full in the run record)
• tail_file_text (served to the dashboard by the log endpoint)

Redacting at the funnels rather than at each consumer means new consumers are covered by default.

scripts/scrub-run-logs.py

Cleans what is already on disk: runs/*.json, the runs.jsonl model log, artifacts/**/*.html, and worker logs. Dry-run by default; only writes with --write; --state-dir for pointing at a fixture.

Two details worth flagging, both found the hard way:

1. JSON and JSON Lines are redacted through their parsed structure, never as raw text. Run records embed worker specs as JSON strings whose inner quotes are backslash-escaped; a text-level regex rewrites across that escaping and leaves a file that no longer parses. This corrupted two real run records in a dry-run rehearsal before it was caught, and there is a regression test for it.
2. After rewriting runs.jsonl it resets the scoreboard's cached read cursor. The read model reads that log incrementally from a stored byte offset, and rewriting the file invalidates it — the observed symptom was the scoreboard reporting "894 rows, 1 skipped lines" against a file whose 895 records all parsed cleanly. The existing log_size < offset guard only catches the log getting shorter; redaction can leave it longer.

Swept my own machine with it: 214 files, 1752 replacements, no secrets left and no record lost.

Tests

tests/test_redact_secrets.py, 25 tests. Pattern coverage, idempotency, the byte-identical no-op, and the ordinary-identifier case (require_authentication must not match re_). Executed assertions through tail_lines, tail_file_text and the real check-output path rather than greps over source, plus the fail-closed path, the JSON-escaping regression, the JSONL round-trip and the cursor reset.

Full suite green on this branch: 280 passed.

🤖 Generated with Claude Code