Repository navigation
ci: add generate_workflows.py --sync-registry to adopt Dependabot action bumps - #4259
Conversation
…ion bumps Dependabot can bump a pinned action in the generated policy-engine-ci.yml but cannot edit .github/ci/actions.toml, so the generated file drifts from the registry and --check fails until a maintainer hand-syncs it. Add a --sync-registry mode that reads the bumped pin out of the managed files (the generated workflow outputs plus the composite actions), updates actions.toml, and regenerates. It fails closed on conflicting pins, is a no-op when already in sync, preserves the registry file layout, and validates that every path it reads or writes stays within the tree (refusing traversal and symlink escapes). Adds hermetic tests for all of these. Implements option 3 of microsoft#3638. Signed-off-by: AlgoVoi <chopmob@gmail.com>
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
PR Review Summary
Verdict: AI review comments are untrusted advisory output. The summary reports workflow-generated completion status only, not model-authored pass/fail claims. |
|
🔴 Contributor Check: HIGH
Automated check by AGT Contributor Check. |
github-actions
Bot
added
the
needs-review:HIGH
| if not match: | ||
| continue | ||
| ref = match.group("ref") | ||
| key = key_by_name.get(ref.split("@", 1)[0]) |
There was a problem hiding this comment.
scripts/ci/generate_workflows.py:317 The sync adopts any 40-hex SHA for a registered action with no check that it resolves to the version in the trailing comment: replacing actions/checkout's SHA with forty f's and # v9.9.9 in the managed files, --sync-registry wrote it into actions.toml and --check then passed. The registry exists to keep pins honest, and this turns adopting a pin into one command, so please verify before writing: git ls-remote https://github.com/<owner>/<repo> refs/tags/<comment> (and refs/tags/<comment>^{} for annotated tags) must equal the SHA, with a clear error otherwise. An explicit --no-verify for offline use is fine if the output says the pin was not verified.
| except (OSError, UnicodeError) as exc: | ||
| raise GenerationError(f"cannot read {rel}: {exc}") from exc | ||
| for line in text.splitlines(): | ||
| match = _PIN_LINE_RE.search(line) |
There was a problem hiding this comment.
scripts/ci/generate_workflows.py:313 _PIN_LINE_RE.search is unanchored, so a commented-out YAML line such as # - uses: actions/setup-python@<sha> # v9.0.0 counts as a pin source (it fails closed today only because the live pin in the same file conflicts). Please anchor it to a step line, for example ^\s*-?\s*uses:.
This branch has not been deployed
2 participants
Related Issue
Related to #3638. Implements option 3 (teach the generator to run the other direction: read the bumped pin out of the generated file and update the registry).
Problem & Solution
Problem: Dependabot can edit a pinned action in the generated
.github/workflows/policy-engine-ci.ymlbut cannot edit its source of truth.github/ci/actions.toml, so an actions major leaves the generated file drifting from the registry andgenerate_workflows.py --checkfails. Four such PRs (#3549, #3598, #3599, #3633) each needed a maintainer to hand-sync the registry and regenerate.Solution: add a
--sync-registrymode toscripts/ci/generate_workflows.py. It reads the bumped pin back out of the managed files (the generated workflow outputs plus the hand-authored composite actions under.github/actions, which by the registry's own contract reference every registered action), updatesactions.tomlto match, then regenerates. This is the deterministic equivalent of the manual sync, so the recurring step becomespython3 scripts/ci/generate_workflows.py --sync-registryon a Dependabot actions PR.It fails closed: if two managed files disagree on the pin for the same action it raises rather than guessing, and it is a no-op when the registry already matches. Only managed files drive the registry, so an unrelated hand-authored workflow cannot. The registry rewrite is line-based and leaves the file's header comments, blank lines, and other entries byte for byte intact.
Path safety: because this mode both reads the managed files and now writes
actions.toml, it validates that every path it touches stays inside the tree. Generated output paths must resolve beneath the repo root, composite action files must resolve beneath.github/actions, adopted version comments must look like a tag before they are quoted into TOML, and a symlinked registry file, composite directory, or composite action file is refused. The real tree has none of these, so normal--writeand--checkbehaviour is unchanged; the checks harden the file I/O the generator was already doing.This intentionally does not add a workflow that auto-commits onto Dependabot branches. That would need a
pull_request_target-class trigger with write access, which is a larger security surface and a separate decision. The deterministic script lands first; I am happy to follow up with the auto-commit workflow (option 2) on top of it if you want it.Impact on Your Work
Removes the manual registry-sync commit on every GitHub-Actions major. No change to
--writeor--checkbehaviour on a clean tree.Alternatives Considered
Type of Change
Package(s) Affected
Testing
Unit Testing
Hermetic tests added to
tests/ci/test_generate_workflows.pycovering: adopting a bumped pin, no-op when in sync, fail-closed on conflicting pins, registry-layout preservation, unsafe-comment rejection, output-path traversal, and symlinked registry / composite directory / composite files (escaping and in-tree).pytest tests/ci -qis green (119 tests).Manual Testing
On a clean checkout where
--checkpasses, simulated a Dependabot bump by changing theactions/checkoutSHA and# vcomment across every managed file, confirmed--checkfails on drift, ran--sync-registry, confirmedactions.tomladopted the new pin and--checkpassed again, then restored the tree.ruff checkpasses on both files.Checklist
Attribution & Prior Art
Prior art / related projects: none; this extends the repo's own
generate_workflows.py.AI Assistance
If AI tools materially shaped this change: drafted with AI assistance and independently validated by running the tests and the end-to-end sync on a Linux VM; every change was reviewed before submission.
IP, Patents, and Licensing