Skip to content

ci(review): two automatic reviews per pull request, on Sonnet - #24

Merged
bowenforsoup merged 1 commit into
mainfrom
ci/review-twice-on-sonnet
Oct 4, 2026
Merged

bowenforsoup merged 1 commit into
mainfrom
ci/review-twice-on-sonnet

Conversation

@bowenforsoup

@bowenforsoup bowenforsoup commented Oct 4, 2026 •

Copy link
Copy Markdown
Member

Why

Each Louise review is a Claude session that costs about $1.55 to $1.93 in API-equivalent usage, billed against the Claude subscription because CI signs in with CLAUDE_CODE_OAUTH_TOKEN. Every push started a new one: coracle.coffee ran 247 reviews on 100 pull requests in the first four days of October. About 60% of them re-reviewed a pull request that already had one. The review bots made about 8,300 of the roughly 9,600 Louise MCP calls in the last 30 days. Each review also bills 5 to 6 Actions minutes, and private repositories have nearly used the Team plan's 3,000 included minutes for October.

What changes

  • Two automatic reviews, then on request. A new gate job counts the earlier reviews before the review job starts. A push starts a review only while there have been fewer than two. Opening, reopening, marking ready for review, and the new louise:review label always start one.
    • The post job writes the count on the review comment's second line, <!-- louise-review-count: N -->, so the count survives the comment being replaced. A review posted before this change counts as one. The gate reads only bot comments.
    • The post job removes the louise:review label after posting, so adding it again asks for the next review.
  • Sonnet in CI. The session runs with --model sonnet. CLAUDE_CODE_SUBAGENT_MODEL=sonnet with CLAUDE_CODE_SUBAGENT_MODEL_FORCE=1 puts the louise:reviewer subagent on Sonnet over its model: opus. A local session keeps Opus. _FORCE needs Claude Code 2.1.257 or later; the action installed 2.1.289 on today's runs.
  • The README and the workflow's header describe both, and the caller example adds labeled to the event types.

Rollout

Merge this before any caller adds labeled. With the current main, a labeled event would start a review for any label. A caller without labeled still works: a push past the cap is skipped, and marking the pull request as a draft and then ready for review asks for another. Each repository that wants the label path adds labeled to its caller's types and runs gh label create louise:review once.

Testing

  • Ran the gate's jq on sample comments. An old review without a count reads as 1. A review with count 2 reads as 2. A human comment claiming count 99 is ignored. No comments gives 0.
  • Ran the post job's sed on review files with the marker line and without it.
  • actionlint 1.7.12, with shellcheck on the run: scripts: clean.
  • pnpm run lint:docs: 0 errors.
  • Not yet run end to end in Actions. The first pull request after merging will exercise it.

Generated with Claude Code

Each Louise review is a Claude session, and every push started a new one:
coracle.coffee ran 247 reviews on 100 pull requests in four days of October.

- A gate job counts the earlier reviews from a count the post job writes on
  the comment's second line. A push starts a review only while there have
  been fewer than two. Opening, reopening, marking ready, and the new
  louise:review label always start one; the post job removes the label.
- The session runs with --model sonnet, and CLAUDE_CODE_SUBAGENT_MODEL with
  _FORCE puts the louise:reviewer subagent on Sonnet too, over its
  model: opus, which a local session still uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bowenforsoup
bowenforsoup merged commit a5fd5f8 into main Oct 4, 2026
2 checks passed
@bowenforsoup
bowenforsoup deleted the ci/review-twice-on-sonnet branch October 4, 2026 22:38
bowenforsoup added a commit to bowenlabs/louise-toolkit that referenced this pull request Oct 4, 2026
## What changes

The Louise review caller adds `labeled` to its `pull_request` types, so
the `louise:review` label asks for another review. The label exists in
this repository now.

## Merge order

Merge bowenlabs/claude-plugins#24 first. That pull request limits a pull
request to two automatic reviews, moves the CI review to Sonnet, and
ignores labels other than `louise:review`. Until it's on `main`, a
`labeled` event here would start a review for any label.

This pull request is a draft so that opening it doesn't start a review
on the current workflow. Mark it ready after #24 merges; that review
then runs on Sonnet.

## Testing

- `actionlint` 1.7.12: clean.

Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
bowenforsoup added a commit to bowenlabs/astroidjs that referenced this pull request Oct 4, 2026
## What changes

The Louise review caller adds `labeled` to its `pull_request` types, so
the `louise:review` label asks for another review. The label exists in
this repository now.

## Merge order

Merge bowenlabs/claude-plugins#24 first. That pull request limits a pull
request to two automatic reviews, moves the CI review to Sonnet, and
ignores labels other than `louise:review`. Until it's on `main`, a
`labeled` event here would start a review for any label.

This pull request is a draft so that opening it doesn't start a review
on the current workflow. Mark it ready after #24 merges; that review
then runs on Sonnet.

## Testing

- `actionlint` 1.7.12: clean.

Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant