Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions dev/breeze/doc/08_ci_tasks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,24 @@ If it cannot determine which milestone should be added, it also adds a comment t
:width: 100%
:alt: Milestone Tag Assistant

Checking your fork CI setup
---------------------------

Pull requests from external contributors can run their CI in the contributor's fork (AIP-120).
``breeze ci audit`` checks, read-only, whether your checkout and fork are set up for that: ``gh``
installed and authenticated, the ``upstream`` remote pointing at ``apache/airflow`` and ``origin`` at your
fork, whether you are a committer, collaborator, member of an exempted organisation, pilot participant or
external contributor, whether the ``Tests (AMD)`` workflow in your fork is enabled (or disabled, for those
whose pull requests run project CI), how far your fork's ``main`` is behind, and the branch you are on.
Each failing check prints the command that fixes it; ``--fix`` runs those commands after asking.

These are all available flags of ``audit`` command:

.. image:: ./images/output_ci_audit.svg
:target: https://raw.githubusercontent.com/apache/airflow/main/dev/breeze/images/output_ci_audit.svg
:width: 100%
:alt: Breeze ci audit

Reconciling pull requests gated to fork CI
------------------------------------------

Expand Down
42 changes: 30 additions & 12 deletions dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md
Original file line number Diff line number Diff line change
Expand Up @@ -772,21 +772,39 @@ prints one row: status, finding, and the command that fixes it.

| # | Check | Passing condition | Offered fix |
|---|---|---|---|
| 1 | `gh` present and authenticated | `gh auth status` succeeds, version at least `MIN_GH_VERSION` | install / `gh auth login` |
| 1 | `gh` present and authenticated | `gh api user` succeeds, version at least `MIN_GH_VERSION` | install / `gh auth login` |
| 2 | Remote naming | a remote named `upstream` points at `apache/airflow` and `origin` does not | `git remote rename`/`add` per `contributing-docs/10_working_with_git.rst` |
| 3 | Fork exists | `origin` is a fork of `apache/airflow` owned by the authenticated user | `gh repo fork apache/airflow --remote=false` then add `origin` |
| 4 | Who you are | classifies the user: committer (`GET /user/memberships/orgs/apache` active), collaborator (`.asf.yaml`), exempted (`[allowlist]`), pilot participant (`[pilot]`), external | informational |
| 5 | Fork workflows | external or pilot participant: `ci-amd.yml` state on the fork is `active`; committer/collaborator/exempted: state is not `active` | `gh workflow enable ci-amd.yml -R <fork>` or `gh workflow disable ...` |
| 6 | Fork `main` freshness | fork `main` is not behind `upstream/main` by more than the configured threshold | `gh repo sync <fork> --branch main` |
| 3 | Fork exists | `origin` is a fork of `apache/airflow` owned by the authenticated user | `gh repo fork apache/airflow --remote=false` then add `origin`, unless check 2's fix already adds or repoints it |
| 4 | Who you are | classifies the user in the gate's order: pilot participant (`[pilot]`, which gates whatever the role), committer (public `apache` membership, `GET /orgs/apache/public_members/<login>`, since the gate cannot see private membership), collaborator (`.asf.yaml`), exempted (`[allowlist]`), external; an external user whose `apache` membership is private is told to make it public or ask for `[allowlist]`; when the membership call fails for any reason but a 404, the role is reported as not checked and checks 2, 3 and 5 stay informational | informational |
| 5 | Fork workflows | external or pilot participant: `ci-amd.yml` state on the fork is `active`; committer/collaborator/exempted: informational, with a hint when it is `active` | `gh workflow enable ci-amd.yml -R <fork>`, or `gh workflow disable ...` shown but not run by `--fix` |
| 6 | Fork `main` freshness | fork `main` is not behind `upstream/main` by more than 200 commits (`MAX_FORK_MAIN_BEHIND`) | `gh repo sync <fork> --branch main` |
| 7 | Current branch | not `main`, has an upstream tracking `origin` | informational |

Committers and collaborators are told to keep fork workflows disabled because their pull
requests run on project CI; an enabled fork would run every push twice and attach a second,
confusing status. Forks of a public repository cannot be made private on GitHub, so that is
the only recommendation available.

The remote-parsing block currently inside `breeze ci upgrade` is factored into
`utils/git_remotes.py` and reused, so both commands agree on what a correct setup is.
The role is determined before checks 2 and 3, though printed as check 4. For a committer,
collaborator or exempted contributor, whose pull requests run project CI, checks 2 and 3 are
informational: working from an `apache/airflow` clone without a fork is fine for them, and
`--fix` only runs the fixes of failed checks. The same goes for check 5: an enabled fork
workflow runs every push twice and attaches a second status, so they are told they can disable
it, but it is not a failure, because a committer trying fork CI with the `use fork ci` label
needs it enabled. Forks of a public repository cannot be made private on GitHub, so disabling
the workflow is the only recommendation available.

A failed API call is reported as "could not check" rather than as a finding: a failed fork
lookup is not "not a fork of apache/airflow", and a failed workflow lookup is not "missing".
A 404 is a finding: the fork does not exist, or the workflow is missing. An `upstream` remote
that points somewhere else gets `git remote set-url upstream`, since `git remote add` would
fail on the existing name.

The remote parsing that `breeze ci upgrade` had inline is factored into
`utils/git_remotes.py` and reused, so both commands agree on what a correct setup is. A remote
is recognised as a GitHub repository in every form git accepts (`http` or `https` with optional
credentials, `www.` or a port, scp-like ssh, `ssh://` with an optional port, any case of the
host, and an ssh host alias from `~/.ssh/config` such as `git@github-personal:apache/airflow.git`,
counted when the alias names `github` and has no dot, or is `github.com` with a `-` suffix such
as `github.com-work`, since any other dotted host such as a GitHub Enterprise server is a
different GitHub), and only when the repository is the whole path, so `apache/airflow-site` is never taken
for `apache/airflow`. A command the audit runs that is not installed, `gh` included, is reported
as a failed check rather than a crash.

## Documentation changes

Expand Down
12 changes: 8 additions & 4 deletions dev/breeze/doc/images/output_ci.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion dev/breeze/doc/images/output_ci.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
037d52dccd1cf22e81b335f1319ecb9c
ceb5f9dfbcb79c72545427568d9b7dbc
Loading
Loading