From 31f24b0642315c0d36c492b9c88969421a9ae5f0 Mon Sep 17 00:00:00 2001 From: Jarek Potiuk Date: Thu, 15 Oct 2026 23:05:03 +0100 Subject: [PATCH 1/4] Add breeze ci audit for the fork CI setup Fork CI only works when the contributor's remotes, fork and fork workflow are set up the way the gate and the reconciler expect, and committers must keep fork CI off so their pushes do not run twice. A read-only audit lets contributors verify all of that before pushing and offers the exact fix for anything missing. Remote parsing moves out of ci upgrade so both commands agree on what a correct setup is, and it now matches apache/airflow exactly instead of any URL containing it. Generated-by: Claude Opus 5 --- dev/breeze/doc/08_ci_tasks.rst | 18 + .../08_fork_ci_for_external_contributors.md | 11 +- dev/breeze/doc/images/output_ci.svg | 12 +- dev/breeze/doc/images/output_ci.txt | 2 +- dev/breeze/doc/images/output_ci_audit.svg | 116 ++++++ dev/breeze/doc/images/output_ci_audit.txt | 1 + ...utput_setup_check-all-params-in-groups.svg | 28 +- ...utput_setup_check-all-params-in-groups.txt | 2 +- ...output_setup_regenerate-command-images.svg | 2 +- ...output_setup_regenerate-command-images.txt | 2 +- .../airflow_breeze/commands/ci_commands.py | 93 +++-- .../commands/ci_commands_config.py | 9 + .../src/airflow_breeze/utils/ci_audit.py | 307 ++++++++++++++++ .../src/airflow_breeze/utils/git_remotes.py | 75 ++++ dev/breeze/tests/test_ci_audit.py | 334 ++++++++++++++++++ dev/breeze/tests/test_git_remotes.py | 83 +++++ 16 files changed, 1035 insertions(+), 60 deletions(-) create mode 100644 dev/breeze/doc/images/output_ci_audit.svg create mode 100644 dev/breeze/doc/images/output_ci_audit.txt create mode 100644 dev/breeze/src/airflow_breeze/utils/ci_audit.py create mode 100644 dev/breeze/src/airflow_breeze/utils/git_remotes.py create mode 100644 dev/breeze/tests/test_ci_audit.py create mode 100644 dev/breeze/tests/test_git_remotes.py diff --git a/dev/breeze/doc/08_ci_tasks.rst b/dev/breeze/doc/08_ci_tasks.rst index 5c85fced90c1a..ca16b4483febc 100644 --- a/dev/breeze/doc/08_ci_tasks.rst +++ b/dev/breeze/doc/08_ci_tasks.rst @@ -161,6 +161,24 @@ If it cannot determine which milestone should be added, it also adds a comment t :width: 100% :alt: Milestone Tag Assistant +Checking your fork CI setup +--------------------------- + +Pull requests from external contributors can run their CI in the contributor's fork (AIP-120). +``breeze ci audit`` checks, read-only, whether your checkout and fork are set up for that: ``gh`` +installed and authenticated, the ``upstream`` remote pointing at ``apache/airflow`` and ``origin`` at your +fork, whether you are a committer, collaborator, member of an exempted organisation, pilot participant or +external contributor, whether the ``Tests (AMD)`` workflow in your fork is enabled (or disabled, for those +whose pull requests run project CI), how far your fork's ``main`` is behind, and the branch you are on. +Each failing check prints the command that fixes it; ``--fix`` runs those commands after asking. + +These are all available flags of ``audit`` command: + +.. image:: ./images/output_ci_audit.svg + :target: https://raw.githubusercontent.com/apache/airflow/main/dev/breeze/images/output_ci_audit.svg + :width: 100% + :alt: Breeze ci audit + Reconciling pull requests gated to fork CI ------------------------------------------ diff --git a/dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md b/dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md index a0ab1a81ada2c..99631eb64dfa3 100644 --- a/dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md +++ b/dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md @@ -774,8 +774,8 @@ prints one row: status, finding, and the command that fixes it. |---|---|---|---| | 1 | `gh` present and authenticated | `gh auth status` succeeds, version at least `MIN_GH_VERSION` | install / `gh auth login` | | 2 | Remote naming | a remote named `upstream` points at `apache/airflow` and `origin` does not | `git remote rename`/`add` per `contributing-docs/10_working_with_git.rst` | -| 3 | Fork exists | `origin` is a fork of `apache/airflow` owned by the authenticated user | `gh repo fork apache/airflow --remote=false` then add `origin` | -| 4 | Who you are | classifies the user: committer (`GET /user/memberships/orgs/apache` active), collaborator (`.asf.yaml`), exempted (`[allowlist]`), pilot participant (`[pilot]`), external | informational | +| 3 | Fork exists | `origin` is a fork of `apache/airflow` owned by the authenticated user | `gh repo fork apache/airflow --remote=false` then add `origin`, unless check 2's fix already adds it | +| 4 | Who you are | classifies the user in the gate's order: pilot participant (`[pilot]`, which gates whatever the role), committer (`GET /user/memberships/orgs/apache` active), collaborator (`.asf.yaml`), exempted (`[allowlist]`), external | informational | | 5 | Fork workflows | external or pilot participant: `ci-amd.yml` state on the fork is `active`; committer/collaborator/exempted: state is not `active` | `gh workflow enable ci-amd.yml -R ` or `gh workflow disable ...` | | 6 | Fork `main` freshness | fork `main` is not behind `upstream/main` by more than the configured threshold | `gh repo sync --branch main` | | 7 | Current branch | not `main`, has an upstream tracking `origin` | informational | @@ -786,7 +786,12 @@ confusing status. Forks of a public repository cannot be made private on GitHub, the only recommendation available. The remote-parsing block currently inside `breeze ci upgrade` is factored into -`utils/git_remotes.py` and reused, so both commands agree on what a correct setup is. +`utils/git_remotes.py` and reused, so both commands agree on what a correct setup is. A remote +is recognised as a GitHub repository in every form git accepts (`http` or `https` with optional +credentials, `www.` or a port, scp-like ssh, `ssh://` with an optional port, any case of the +host), and only when the repository is the whole path, so `apache/airflow-site` is never taken +for `apache/airflow`. A command the audit runs that is not installed, `gh` included, is reported +as a failed check rather than a crash. ## Documentation changes diff --git a/dev/breeze/doc/images/output_ci.svg b/dev/breeze/doc/images/output_ci.svg index bee7d61e07476..7158eac2e10d3 100644 --- a/dev/breeze/doc/images/output_ci.svg +++ b/dev/breeze/doc/images/output_ci.svg @@ -1,4 +1,4 @@ - +