Repository navigation
Conversation
AIP-120 moves CI for external contributions into contributor forks so project runners serve committers, collaborators and exempted organisations, and so maintainers only review pull requests that already passed in the author's fork. The mechanics span the main test workflow, a scheduled reconciler for gated pull requests, a commit status that surfaces the fork CI state with a link to the fork run, a local pre-flight command and the contribution docs, so the design is recorded before implementation and the decisions that shape it are captured as ADR 0018. Generated-by: Claude Opus 5
The gate job in ci-amd.yml runs from the pull request's merge commit, so a gated external pull request can drop the gate or add a workflow with a pull_request trigger and get project CI anyway. The design accepted that as a one-run cost but left it unbounded per pull request, because every further push would get another run. The reconciler now detects this after the fact from what GitHub recorded rather than from the pull request tree: project CI ran for the head SHA and the pull request changes files under .github/workflows/. Both signals are required so that legitimate project CI runs (backlog reset, a removed "use project ci") and legitimate workflow edits that keep the gate intact are never closed. Closing stops further pull_request runs, so one attempt buys one run, and the closure is reported to Slack as an attention item rather than as a reconciler failure. ADR 0018 records the trade-off, including the external pull requests with wanted workflow changes that must keep "use project ci". Generated-by: Claude Opus 5
The dev list agreed to start with an opt-in pilot by a few volunteer contributors rather than enabling fork CI for everyone at once. The committer-only "use fork ci" label cannot serve that: it arrives after gpgsig -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJq0YVxCRC1aQ7uu5UhlAAAcYgQACYDDrIdXBZxVhqczF3TNrO7 qdkPgQy3NgLfVgmY0iy7ypzDRYY+glMD415Qc5OJy7n5M7XBqJ3ALQf04U/IGm0O bMXgq1+Z9B/sRz27VNQYJm9X30hL+pE4iYHwGAQHXsr/TIgl2WN+V4awsRYEVQXM x3Q/jwR12/YbiW2R3c61J+c7mrLdw4bZ6UtnDQqWUYkimgRGDLFvlZ/yMAj8kBuU UyXGeOtTaIJHlghczgxlTeBDNMjuE9o5S6xsW5n+eWJXy7E/883L6uMWR3OjLR01 l2gpSqrpi52xYMVd8Zik+gpuWjkQ45C9k+XgGQfLfhoelZJ8uZfToMqARne4xVlZ 4hFIHfZSGdcLCi+sbU6PKxrMdX7HF4vDAxP9Tv4IpIzANRxG9IH5uXpHwAAx24LF ogZQDyykjqTsadl7SqjKDw/Hybv4hklnnr8BFfvYd54iFkkP3fqSizlV4ecr1tyJ OBvI7DbNNNdySebmnMYQGBziLQF5uUseD8srhCiXG5XNaSqbOnNXc+FHZmn0oU3O EJ15O0QPi4rkRXQ5gYCjjb4tO99H3Mw6bMgiIWTjVa/1QPmOb9mtEbpRP/ESHuOe 1VgYd8tUczg6FB6Po5f/DXV2rZB36ij9Bo4nBmR7V1HApOx4E88lDIznGpqP/cWf MetUk2Nom7/sr+UaBu3i =wncY -----END PGP SIGNATURE----- the first pull_request run has already used project CI, and a maintainer would have to label every pull request the volunteer opens. A per-login join instant gates all their new pull requests from the first run while leaving older ones on project CI, so those can never be mistaken for gate tampering. The reconciler also has to run while the gate is disabled, which the earlier "exits early" wording contradicted for the label pilot as well. Generated-by: Claude Opus 5
Enabling ci-amd.yml in a fork also enables its cron, which would run the full canary matrix in every contributor fork twice a day. A join instant or enabled_since earlier than the merge that sets it gates pull requests that already ran project CI. Forks not named "airflow" would never show a fork run, and a pull request reopened after an inactivity close would be closed again at once. ADR 0018 was already taken. Generated-by: Claude Opus 5
A fork run executes the workflows, breeze and CI scripts of the pushed commit, so a pull request changing them decides its own fork CI result. Gate tampering was only checked on the head commit, so pushing a tampered commit and then a clean one hid the bypass, and the check could be dodged by renaming a job to the gate's name or by truncated check-suite listings. The gate also fell back to project CI when its label read failed, which an exhausted token quota could trigger for every pull request. Generated-by: Claude Opus 5
The gate job and the reconciler must agree on which pull requests use project CI and which are left to fork CI, so the decision lives in one standard-library-only function that the gate can run without installing breeze, driven by one reviewed configuration file. It ships disabled. Generated-by: Claude Opus 5
Contributors must get CI from a plain push to their fork, and project CI must not run for pull requests left to fork CI, without reacting to pull request events with write permissions. A gate job ahead of build-info decides from the base branch's configuration, and the push trigger widens to every branch while the gate keeps the old branch list in effect for apache/airflow. Skipping build-info also skips everything that depends on it, which the always() conditions of finalize-tests and notify-slack would otherwise treat as success. Pilot join instants are compared with pull request creation but only take effect on merge, so the config hook keeps new ones in the future. Generated-by: Claude Opus 5
A push in a contributor fork would otherwise be treated as a post-merge push and run the full matrix, about 220 minutes on the contributor's 20-runner cap, for every push. Computing the change set against apache/airflow gives the same selection a pull request would get, independent of how stale the fork's main is, and fork runs must never select self-hosted runners that forks do not have. Generated-by: Claude Opus 5
muditagrawal2007
left a comment
There was a problem hiding this comment.
Stacked-on-#74825 design is clean, and the "post the closing comment before the state comment records the new state" change is the right call — a failed post was silently lost before this PR. The gate-protection added layer is also a sensible guardrail for the same class of self-deciding risk that motivated the original gate. Read the full diff against fork-ci-5-docs; below is what stood out.
1. The gate-modified marker doesn't record gate_checked: true
The closure paths for gate_bypassed, closed (inactivity) and the regular state branches all now write Marker(..., gate_checked=gate_checked) — but the new gate_modified branch in release/classify writes Marker(ForkState.GATE_MODIFIED.value, pr.head_sha, now) without that flag:
_set_marker_comment(
plan,
pr,
state_comment(
Marker(ForkState.GATE_MODIFIED.value, pr.head_sha, now), fork=pr.head_repo, run_url=None
),
)The documentation states the opposite:
The state comment's marker records
"gate_checked": truewith the head commit only when this check ran for it
The check did run for this head commit — that's why we're in this branch. So the comment and the doc disagree.
The reason this matters in practice is maintainer-reopen-without-restore. Reading the docstring for reopened_since_closing, when marker.state == gate_modified and pr.last_reopened_at > marker.since, the "reopen since closing" path is skipped (correctly), and execution falls through to the gate-check block. With gate_checked: false on the marker, the check re-runs and closes the PR again — which matches the documented intent:
a maintainer's reopen without restoring the gate closes it again on the next run
So either:
- The code is right and the doc sentence above is overstated (e.g., add an exception for "and the close path retains
gate_checked: falseso a reopen re-runs the check"), or - The code should record
gate_checked: trueandalready_checkedshould additionally requirepr.last_reopened_at is None or pr.last_reopened_at <= marker.since. That keeps the once-per-head-commit guarantee and still re-runs after a maintainer reopen.
The second option is more consistent with the rest of the diff and with the rule "checked once per head commit". Right now the rule is held together by the same exception that makes the doc misleading.
already_checked = (
marker is not None
and marker.gate_checked
and marker.sha == pr.head_sha
and (pr.last_base_change_at is None or pr.last_base_change_at < marker.since)
and (pr.last_reopened_at is None or pr.last_reopened_at <= marker.since)
)Then record gate_checked=gate_checked on the gate-modified marker too, and add a test for the reopen-without-restore case (test_gate_change_reopens_closes_again or similar).
2. Gate-modified plans don't ask for the gate_checked sentinel that other closures carry
Same point as #1, expressed from the consumer side. test_a_change_to_the_gate_closes only asserts '"state": "gate_modified"' in plan.marker_comment, which passes whether or not the gate-checked flag is set. Tightening that to "gate_checked": true would lock in whichever resolution you pick and surface regressions in either direction. The other three tests for this branch (test_unfinished_gate_change_closure_is_retried, test_marker_records_whether_the_gate_was_checked, test_gate_change_is_checked_once_per_head_commit) sidestep the close path.
3. classify is now ~270 lines doing everything
Counting from the start of classify to its return plan, the function decides: which paths skip the gate (use project ci/use fork ci), gate tampering, gate-modified closure, "reopen since closing", fork state observation, green-trust decision (unverified vs verified), the shared-head-across-bases path, draft/labels, status, marker. None of these are individually complicated but reading them as one diff is. A small refactor — extracting _plan_closure(state, comment_factory, *, change=...) for the three close paths and _plan_record_state(state, since) for the record-only paths, with classify as a top-level orchestrator — would make the diff much easier to review and the existing tests can stay because they assert on Plan shape.
4. observe_fork_state doesn't know about gate_modified
def observe_fork_state(fork: ForkObservation, marker: Marker | None) -> tuple[ForkState | None, str | None]:
...
if fork.workflow_state is None:
if marker is not None and marker.state == ForkState.AWAITING_PUSH.value:
return ForkState.AWAITING_PUSH, None
return None, "no fork run and the fork workflow state was not read"For a re-opened gate_modified PR with no fork run, this returns (None, "no fork run...") rather than (GATE_MODIFIED, None). That's tolerable because the release/gate-check block runs before observe_fork_state and either closes again or falls through (depending on #1), but if you do end up extending the gate-check block to be re-entrant from observe_fork_state, the function needs an explicit gate_modified case, otherwise the post-#1 architecture risks an unknown line per run.
5. _other_pull_requests is re-issued per run, not cached per branch
BypassFinder._other_pull_requests calls rest_get(pulls?head=...) with pagination. The result depends only on (head_repo, head_ref, window_start) and find() is invoked once per branch by run_reconcile. Hoisting it into BypassFinder.__init__ (computed once with the same 6h window) would save one paginated call per PR per run. Trivial cost per call, but the call pattern repeats across every reconciles and shows up in the GitHub calls: {...} log line.
6. _set_status URL is hardcoded in the new branch
_set_status(
plan, pr, ForkState.GATE_MODIFIED.value, f"https://github.com/{repo}/pull/{pr.number}/files"
)vs. the helper used elsewhere — fork.run_url or fork_actions_url(pr.head_repo) and f"https://github.com/{repo}/pull/{pr.number}/checks". The hardcoded value is fine semantically (a gate_modified PR has no fork run to link to; linking the changed files is the right thing), but the inconsistency is worth a helper for consistency — something like gate_changed_files_url(pr.number) if you don't want to inline it.
7. _list_jobs raises and goes through the except GitHubError path
try:
jobs = self._list_jobs(run)
except GitHubError:
result = True
else:
result = ...That result = True on the right-hand side of the OR-test for "did work" reads the wrong way at first glance — "no jobs listed → counts as project CI" is exactly the intended semantic, but the previous shape made the same point by setting result = True before checking anything and the diff moved it under a guard. Worth a two-line comment at the assignment:
except GitHubError:
# A listing we can't make fails open: better to flag a run that turned out
# to be project CI than to miss one.
result = TrueTwo of the parametrized tests cover the False branch; there isn't a test that exercises rest_get raising on the jobs call. Adding one would lock in the "fail open on listing error" semantics.
8. base_at boundary semantics
for changed_at, previous, _ in reversed(pr.base_changes):
if when < changed_at:
base = previousFor a run whose created_at exactly equals a BaseRefChangedEvent.createdAt (when == changed_at), the new base applies. That's right for a run triggered by the retarget itself (pull_request with edited: change_base), which sees the new base. For a run whose created_at is one millisecond less than the retarget (when < changed_at), the loop sets base = previous (old base). That's also defensible — the run was triggered against the old base. The asymmetry around the boundary is intentional but worth a one-line comment; "runs at or after a retarget are considered to be against the new base".
9. Schema test for MONITORED_GATE_PROTECTED_PATHS?
GATE_PROTECTED_PATHS is a single frozenset at module scope; if a future PR renames a file or adds another gate piece (e.g., fork_ci_maintainer.py) the maintainer who lands it needs to remember to add it both here and to the four "this file is part of the gate" comments. A small assert that mirrors the comments — pulling the strings out of those four header comments and checking every protected path has a back-reference — would catch a forgotten update. Optional, but it's the kind of doc/code drift the existing comments are trying to prevent.
10. Smaller things
Marker(ForkState.GATE_MODIFIED.value, pr.head_sha, now)is built both inline (state_comment call) and via a future helper — at this PR's size, the duplication is small but a const for the per-state string would help.from dataclasses import replace—is_own_pilot_changeimports it but doesn't usereplacedirectly... wait, it does:return replace(head, pilot=base.pilot) == base and .... Fine.PILOT_BASE = "enabled = true\n[pilot]\nother = 2026-10-01T00:00:00Z\n"lives at module scope in the test file. Minor — fine for tests, but if anyone wants a second file to reference "what does an unchanged base look like" they'll copy-paste.- The "Gate protection" section in
08_fork_ci_for_external_contributors.mdis excellent and the table-row addition plus the**New comments go first.**rule are real, easily-understood improvements.
Overall the diff lines up with its stated design, the test density matches the risk of each new branch (≥6 cases for "checked once per head commit" alone), and the gate tampering and shared-head paths are clearly motivated by #74905 / #74906 / #74907. The one thing I'd want resolved before merge is the gate-modified marker flag conflict in #1/#2 — whichever way you settle it, it's worth a one-line comment where the marker is built explaining why gate_checked is on every other closure's marker and the exception.
Gated pull requests need labels, draft state, a state comment, a commit status that links to the fork run, and closing after inactivity or gate tampering, and none of that may be written in reaction to an untrusted pull_request event. A scheduled, idempotent reconciler that derives every change from what GitHub recorded is the only writer, so missed or repeated runs are harmless and nothing runs pull request code with write permissions. Whatever it cannot classify is reported to maintainers instead of being guessed. The monitor needs the breeze setup action, which frees disk space with sudo and docker, so it runs on ubuntu-latest rather than ubuntu-slim. Generated-by: Claude Opus 5
Fork CI only works when the contributor's remotes, fork and fork workflow are set up the way the gate and the reconciler expect, and committers must keep fork CI off so their pushes do not run twice. A read-only audit lets contributors verify all of that before pushing and offers the exact fix for anything missing. Remote parsing moves out of ci upgrade so both commands agree on what a correct setup is, and it now matches apache/airflow exactly instead of any URL containing it. Generated-by: Claude Opus 5
Contributors need to know how to set up CI in their fork, how to check that setup, and how to join or leave the pilot, and maintainers need to know which parts of a gated pull request's lifecycle the reconciler owns so triage does not fight it. The previous one-line advice to enable GitHub Actions in the fork did not say any of that. Generated-by: Claude Opus 5
The state comment is what the next reconciler run compares with. Recording awaiting_push there before the nudge was posted meant a failed post was never retried, and the contributor got no prompt before the inactivity closure. Generated-by: Claude Opus 5
…ering check A workflow or trigger the base branch does not have always counted as project CI, so a pull request adding a fork-only workflow that its jobs guard off in apache/airflow was closed for gate tampering although nothing ran. Generated-by: Claude Opus 5
…aintainer The green-trust decision is stored as a commit status, which every pull request with that head commit shares. Pull requests with different bases have different changed files, so a decision made for one could mark the other ready without checking it. Generated-by: Claude Opus 5
Runs from forks carry no pull request, so they were matched by fork branch alone. Legitimate project CI against a base branch without the gate, made before a retarget, or a run of another pull request from the same branch, could close a gated pull request for gate tampering. Generated-by: Claude Opus 5
A pull request whose CI runs in the contributor's fork could change the gate that sent it there: its code, its configuration, the gate job or the monitor. Such a change decides where the pull request's own CI runs, so the monitor now closes it and explains why. Contributors can still add, change or remove their own pilot entry, which is how they join and leave the pilot. Generated-by: Claude Opus 5
64545b5 to
fc720e5
Compare
5ffe91d to
8196f7a
Compare
e9d1bb4 to
87df430
Compare
Stacked on #74825. Part 6 of the AIP-120 implementation (design: #73378); this PR's base is the previous branch, so the diff shows only this layer.
Follow-ups to the reconciler in #74822, one commit each, plus protection of the gate itself:
awaiting_push, so a failed post is retried.green_unverified, since one commit status cannot hold a decision per base.[pilot]entry, thegatejob or its wiring inci-amd.yml, or the monitor) is closed with a comment explaining why. Each of those files says so. The state comment's marker recordsgate_checkedonly when this check ran for the head commit, so pull requests reconciled before it existed are checked once.closes: #74905
closes: #74906
closes: #74907
closes: #74908
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 5) following the guidelines
🤖 Generated with Claude Code