feat(antigravity): read token usage incl. cached from conversation DB

[eugeneng04]

Problem

The agy (Antigravity CLI) harness reported {"input":0,"output":0,"total":0,"cached":0} for every run: it parsed transcript.jsonl for a per-record tokens field that real transcripts never carry (the shape existed only in test fixtures), so every bucket silently defaulted to 0.

Fix

Token usage is persisted in the conversation DB (conversations/<uuid>.db, gen_metadata protobuf blobs) as a per-turn usage record at wire path .1.4:

f2=input (non-cached), f5=cached, f9=reasoning, f10=output, f3=f9+f10 (integrity guard).

• Decode that as the primary token source → {input, cached, cache_write: None, reasoning, output, total}, summed per turn; poll for agy's async DB flush.
• Guarded: the f3 == f9 + f10 invariant rejects non-matching records; undecodable rows (schema drift) stop the poll early; anything unrecoverable yields all-None, never a fabricated 0.
• Fall back to transcript-aggregated counts for old agy formats that did carry per-record tokens.
• metadata.token_source records which source fed the run (db / transcript / unavailable).
• Trajectory/output extraction is unchanged (still from the transcript).

Verification

• Field mapping verified empirically on agy 1.1.3: controlled-output runs (5-word reply → output=5; 7-word → output=7) and a 9-turn warm-cache session (input=48324, cached=121878, reasoning=1214, output=493) matching per-turn hand decodes exactly.
• Proto3 zero-omission edge cases covered: fully-cached turns (f2 omitted) and thinking-only turns (f10 omitted) are counted; config-shaped noise records are rejected.
• 26 unit tests for the decoder, state machine, poll, and fallback paths; full agents/results suites green on main; ruff clean.
• Reviewed via an 8-angle review pass + adversarial verification against real conversation DBs (two top-ranked candidate bugs were refuted empirically: WAL mode rules out the locked-reader race, and no false-positive protobuf matches exist in a 65-blob corpus).

Notes / known limitations

• The DB format is private and version-sensitive; documented in docs/appendix/known_issues.md with a removal condition (drop the decoder once agy exposes usage headlessly) and re-verify-on-upgrade guidance.
• Side-call usage (e.g. title generation) is not stored in gen_metadata; totals cover the main trajectory (~0.2% low observed).
• The row layer (normalize.py/ResultRow) does not yet carry cached/reasoning — deferred to the unified token-accounting work, along with moving the six-bucket schema to a shared module.
• Docs: adds the missing antigravity row to the supported-harnesses table and a Token accounting section covering the gemini/antigravity cached-input asymmetry.

[jessie1111101]

LGTM. might want to port over to sigs repo

[eugeneng04]

Migration note: ported to kubernetes-sigs

The antigravity agy harness is up as kubernetes-sigs/devops-bench#39 — the full package (agents/cli/antigravity/{__init__,agent,parsing}.py) plus tests/unit/agents/test_agents_cli_antigravity.py, since the module did not exist upstream yet. It self-registers via @base.AGENTS.register("antigravity") and depends only on modules already migrated.

Still to port (not carried in #39):

• docs/components/agents.md — the antigravity row in the supported-harnesses table + the Token accounting section.
• docs/appendix/known_issues.md — the DB-format removal-condition / re-verify-on-upgrade note.

Review findings fixed in the ported version (worth applying here at source too):

A 4-agent review panel (correctness / security / perf / Gemini) ran on the ported code; 5 findings were confirmed and fixed in #39. They exist in this PR too:

• [medium] agent.py — the copied antigravity-oauth-token is left in the run workdir, which the harness deliberately retains for artifact collection → live credential can be snapshotted. Fixed by unlinking the copy in a finally once agy exits.
• [medium] parsing.py::db_token_state — any sqlite3.Error was mapped to terminal absent, so a transient locked/half-written read during agy's async post-exit flush permanently dropped token accounting. Fixed by returning pending (retry) on OperationalError/DatabaseError.
• [low] transcript-path total excluded cached (DB path includes it) → understated total on transcript fallback. Fixed to input + output + cached in both transcript parsers.
• [low] format detection materialized the whole transcript just to read the first line. Fixed to a lazy first-line read.
• [low] db_token_state buffered every blob before decoding. Fixed to stream the cursor (peak memory = one blob).

Gemini's HIGH "protobuf field 0 → infinite loop/OOM" was adversarially refuted (the varint reader always advances; work is O(n) and the blob is agy's own local DB, not attacker input).