Skip to content

Add assert --format json|toon (typed AssertView) - #213

Merged
jantimon merged 1 commit into
mainfrom
feat/assert-json
Aug 2, 2026
Merged

Add assert --format json|toon (typed AssertView)#213
jantimon merged 1 commit into
mainfrom
feat/assert-json

Conversation

@jantimon

@jantimon jantimon commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Problem

assert emitted only a human ASCII table plus an exit code. diff and cpu-diff both have --format json, but a CI PR-comment script gating on assert had to scrape the ASCII table to render "layout slice 6 ms > 4 ms budget" — the one place in a gate that consumed text instead of structured output.

Change

assert gains --format json|toon, emitting a typed AssertView (additive, off the stored types, the same pattern as the other views):

  • thresholds[]: one AssertThresholdRow per gate — target (run / step / group), metric, axis (count | timing | slice), budget, value (the measured number or null when not measured), verdict (pass | fail | n/a-fail), and the routed member on a run-group.
  • passed, violations[], notes[] (group disclosures), and a kind discriminator (recording | group).

Example (--target node, a not-measured forced-count on a node recording is a loud n/a-fail):

{
  "target": ".../rec.json", "kind": "recording",
  "thresholds": [
    { "target": "run", "metric": "forced layout/style", "axis": "count",
      "budget": 0, "value": null, "verdict": "n/a-fail", "reason": "not measured" },
    { "target": "run", "metric": "wall ms", "axis": "timing",
      "budget": 100000, "value": 22.565, "verdict": "pass" }
  ],
  "passed": false,
  "violations": ["run: forced layout/style was not measured; cannot satisfy max 0"],
  "notes": []
}

Invariants held

  • Exit code unchanged: passed is true exactly when the process exits 0. A --format json gate fails the build exactly as the human report does (unit + e2e assert passed↔status).
  • Human output byte-identical: the table and its wording are untouched; JSON is an early branch before the table code.
  • The n/a-fail contract: a not-measured metric is value: null / verdict: "n/a-fail" / exit 1, never a fabricated 0 that silently passes.

Scope

  • src/model/query.ts: AssertView, AssertThresholdRow, AssertVerdict, AssertAxis.
  • src/commands/assert.ts: build the view in both the plain and run-group paths; a shared finishAssert emits structured or prints the human report and sets the exit code.
  • src/cli.ts: --format + hidden --json on assert.
  • src/index.ts + README type table + public-types walker: AssertView/AssertThresholdRow exported and enumerated.
  • New test/unit/assert-format.test.mjs (7 tests: shape, the n/a-fail exit-code guarantee, the slice axis, TOON, exit-code parity, and the run-group member routing).

Gates: lint, format:check, build, unit (745), the 3 Chrome-driven assert e2e tests all green.

assert emitted only a human ASCII table plus an exit code, so a CI PR-comment
script had to scrape the table -- the one place a gate consumed text instead of
JSON. It now emits a typed AssertView under --format json|toon, mirroring the
query/diff verbs: a row per threshold (axis count/timing/slice, budget, measured
value or null, verdict pass/fail/n-a-fail, plus the routed member on a run-group),
the overall passed flag, the violation lines, and any group notes.

The exit code is unchanged (0 = passed, 1 = any failed), so a --format json gate
fails the build exactly as the human report does; the human table and its wording
are byte-identical. AssertView/AssertThresholdRow are exported from the package
root and covered by the public-types walker.
@jantimon
jantimon merged commit d62f653 into main Aug 2, 2026
4 checks passed
@jantimon
jantimon deleted the feat/assert-json branch August 2, 2026 15:19
@github-actions github-actions Bot mentioned this pull request Aug 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant