Skip to content

ci: isolate and attest immutable release publishing - #851

Merged
nekomoto911 merged 6 commits into
mainfrom
codex/issue165-release-hardening
Nov 4, 2026
Merged

nekomoto911 merged 6 commits into
mainfrom
codex/issue165-release-hardening

Conversation

@ByteYue

@ByteYue ByteYue commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Official node/CLI releases currently share a persistent runner with E2E, upload onto existing releases with replacement enabled, and provide no signed build provenance. This change builds in fresh GitHub-hosted VMs with read-only access, requires protected tag context and independent environment review, and separates short-lived policy-read, build and publishing identities.

Publication verifies GitHub attestations for node/CLI, an OCI digest and a signed source/version/run manifest, creates a new draft, uploads without overwrite, compares every fully downloaded asset, then publishes and confirms release immutability. Existing drafts/releases fail closed; image consumers use the signed OCI digest. A fail-closed policy checker and tag ruleset template specify immutable releases, non-bypass tag protection and independent release approval. Its currently requested read-only policy token cannot expose the bypass list; the authority blocker below must be resolved before activation.

The build also rejects node/CLI ELF requirements newer than GLIBC 2.35, preserving the Ubuntu 22.04 build ceiling; supported target OS smoke acceptance is still required. Missing environment/ruleset controls fail the policy gate before publication. Same-tag duplicate runs intentionally queue and then reject an existing release.

Validation: 24 behavioral tests passed (policy rejection matrix, complete bytes/inventory, upload failures, reruns, tampering, wrong origin, invalid proofs and prerelease classification and ABI acceptance/rejection); actionlint, diff checks and the required staged/history secret scans passed. Parent review found no blocker in policy, permissions and proof/source flow.

A newly validated P1 blocker remains: GitHub hides ruleset bypass_actors unless the policy reader has ruleset-write access. The current Administration-read App candidate therefore cannot complete its audit. Missing visibility now produces an explicit permission error and still fails closed. A sufficient isolated-reader design or independently verifiable alternative needs an owner decision; this change does not elevate App permissions.

Operational acceptance remains outstanding: administrators must configure the read-only GitHub App, immutable releases, tag protection and protected environment, prevent old-tag workflow execution on the historical shared runner while preserving incident evidence, confirm hosted build capacity/target OS compatibility, and run a controlled build/sign/publish/consumer-verification exercise. No native Rust build, real signing/release, settings mutation, runner cleanup or production deployment was performed. Existing unsigned versions require consumer-owned exact expiring migration pins; no generic bypass is introduced.

Related: Galxe/mono-grav#165, Galxe/mono-grav#167, Galxe/mono-grav#168, Galxe/mono-grav#169. These issues remain open pending operational acceptance.

Owner decision (2026-10-30, Asia/Shanghai): retain the read-only policy App and keep release publication blocked. Do not grant Administration:write or treat an omitted bypass list as empty. A separate complete-audit design remains follow-up work.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review 🔄 Running since 2026-10-29T16:57:06.264562Z 11749e7 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d8bb54a2e1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/release/release.py Resolved
@AshinGau

Copy link
Copy Markdown
Contributor

Review:三项声称基本落地,质量高于同类 PR;4 点需要处理

审查范围 codex/issue165-release-hardening,6 文件。对应 #167 / #168 / #169。

修得好的部分

  1. runner 隔离是真的,不是逻辑隔离。全流程 runs-on: ubuntu-22.04(GitHub 托管),并且有运行时断言:
RUNNER_KIND: ${{ runner.environment }}
...
test "$RUNNER_KIND" = github-hosted

配合 CARGO_HOME/RUSTUP_HOME 重定向到 $RUNNER_TEMP,确实离开了 ID 21 那台共享主机。#169 的核心矛盾(构建用户属于 docker + google-sudoers 且有 NOPASSWD)由此解除。

  1. 权限按 job 分离,不是一把 contents: write 全局开到底:
  • 顶层 permissions: contents: read(PR 验证与构建都拿不到写权)
  • 仅 publish job 提权到 contents: write / packages: write / id-token: write / attestations: write,且绑定 environment: release
  1. --clobber 已彻底移除,改为 assert-new 拒绝既有 release/draft,发布前校验完整下载的资产字节,最后确认 immutability。这与 [feat]: Return tx hash when submitting tx #167 要求的"重跑已有正式版本应安全拒绝"一致。

  2. attestation 覆盖二进制、OCI digest、manifest 三类,并且 --cert-identity 钉到 release.yml@refs/tags/$TAG,[feat] check account nonce in coordinator and skip the txn with old nonce #168 强调的"任意有效签名不能通过"被满足。

  3. 没有引入长期签名私钥 —— 用 actions/attest-build-provenance(OIDC + Sigstore),符合 [feat] check account nonce in coordinator and skip the txn with old nonce #168 明确要求。

需要处理的 4 点

P1 —— environment: release 的保护配置不在本 PR 内,且 ubuntu-22.04 需要实测兼容性

publish 依赖 environment: release 提供 required reviewers。但 environment 的 branch policy(限 v* tag)与 reviewer 名单是 GitHub 设置,不在代码里。PR 描述已把"protected environment"列为 operational acceptance —— 这点诚实。建议在 issue 里明确:在 environment 配好之前,该 PR 合入等于发布权限从"共享 runner + clobber"变成"任意 tag push 即可发布",风险方向变了但没消失。

另外 ubuntu-22.04 的 glibc 版本必须 ≤ 生产节点。原 workflow 有醒目注释说明这个约束("否则 GLIBC_X.YY not found"),新流程保留了 cat /etc/os-release 与 ldd --version 的输出步骤(107-108、121 行)—— 好。但托管 runner 的镜像会滚动更新,建议在步骤中断言 glibc 上限,而不只是打印。

P2 —— tag-ruleset.json 只是模板,不会被自动应用

scripts/release/tag-ruleset.json 是声明式配置,需要有权限的人执行 gh api 才能生效。建议在 issue 里标注该文件为"待应用",并附应用后的读回证据(gh api repos/Galxe/gravity-sdk/rulesets),否则容易误认为已生效。

P2 —— RELEASE_POLICY_APP_PRIVATE_KEY 又是一个长期 App 私钥

policy job 用 actions/create-github-app-token 换取只读 token,权限已最小化(administration: read + environments: read)—— 收窄得体。但这把私钥本身仍是长期凭据,且存放在 repo 级 secret(除非还配了 environment)。建议:确认它是否在 environment 内,或改用其他不需要私钥的机制。

P3 —— concurrency: group: release-${{ github.ref }} + cancel-in-progress: false

同一 tag 重跑会串行排队;配合 assert-new 拒绝已存在 release,行为正确。但 workflow_dispatch 与 push 对同一 tag 会产生相同 group,若有人手动 dispatch 一个已由 push 触发过的 tag,会排队等待而非快速失败。小问题,确认是否符合预期。

测试覆盖

PR 描述称 21 个 behavioral tests(策略拒绝矩阵、完整字节/清单、上传失败、重跑、篡改、错误来源、无效证明、prerelease 分类)。覆盖面看起来对得上 #168 第六条验收("合法通过、篡改/未签名/错误仓库或 workflow/错误版本均被拒绝")。建议在 PR 里贴一次 python3 -m unittest discover -s scripts/release -p 'test_*.py' -v 的完整输出,便于审阅者确认不是只跑 happy path。

边界

仅静态审查分支内容与 PR 描述。未执行任何构建、未触发 workflow、未验证 GitHub environment/ruleset 的实际配置状态,未读取 secret 值。glibc 兼容性与托管 runner 容量属于"需实测"项,本审查无法替代。

@ByteYue

ByteYue commented Oct 29, 2026

Copy link
Copy Markdown
Contributor Author

已逐项处理这轮 review,新增提交 44b54dd303:

  1. glibc 兼容上限:已修。 构建结束、产物上传之前用 readelf --version-info 检查 node/CLI 实际 ELF 需求;高于 GLIBC 2.35、无法识别或读取失败都会拒绝。2.35 仅保持已选 Ubuntu 22.04 构建边界,仍需真实目标机 smoke 验收。
  2. environment 未配置即任意 tag 可发布:这项推断不成立。 policy job 在 build/publish 前通过 audit-policy 检查必需 reviewers、禁止 self-review/admin bypass、只允许 v* tag、immutable 和 tag rules;缺失或 API 无权限均 fail closed。publish 批准后及创建 release 前还会重新读取策略。文档已明确说明。
  3. ruleset 模板、App key:建议合理,归入管理员验收。 文档明确模板待应用、需保留规则详情/环境的读回证据;App key 仍是长期只读策略凭据,实际 repo/environment secret 作用域和轮换需 owner 核实。普通 GITHUB_TOKEN 无所需 Administration-read 权限,不能把 OIDC 签名能力当成平台策略读取权限。本次未改线上设置或凭据。
  4. 同 tag push/dispatch 排队:符合预期。 保留同组串行,防止并发发布;重复任务随后由 assert-new 拒绝,不引入覆盖或快速失败竞态。

23 个 helper 测试、actionlint、实际本机 ELF 的 check-abi CLI、diff check 和历史/候选/PR 文本 secret scan 均通过。完整测试输出如下;Cargo/image 完整构建、真实签名/immutable release、目标机验证仍未执行。

python3 -m unittest discover -s scripts/release -p 'test_*.py' -v
test_accepts_complete_policy (test_release.PolicyTests.test_accepts_complete_policy) ... ok
test_audit_propagates_api_permission_failure (test_release.PolicyTests.test_audit_propagates_api_permission_failure) ... ok
test_audit_refuses_partial_policy_response (test_release.PolicyTests.test_audit_refuses_partial_policy_response) ... ok
test_built_binary_glibc_requirements_are_bounded (test_release.PolicyTests.test_built_binary_glibc_requirements_are_bounded) ... ok
test_each_missing_boundary_fails (test_release.PolicyTests.test_each_missing_boundary_fails) ... ok
test_full_tags_and_commits_only (test_release.PolicyTests.test_full_tags_and_commits_only) ... ok
test_unknown_binary_abi_fails_closed (test_release.PolicyTests.test_unknown_binary_abi_fails_closed) ... ok
test_changed_local_bytes_or_missing_proof_never_create_release (test_release.ReleaseTests.test_changed_local_bytes_or_missing_proof_never_create_release) ... ok
test_existing_draft_or_release_cannot_be_modified (test_release.ReleaseTests.test_existing_draft_or_release_cannot_be_modified) ... ok
test_failed_upload_cannot_publish_and_rerun_refuses_draft (test_release.ReleaseTests.test_failed_upload_cannot_publish_and_rerun_refuses_draft) ... ok
test_invalid_signature_never_creates_release (test_release.ReleaseTests.test_invalid_signature_never_creates_release) ... ok
test_linked_asset_rejected (test_release.ReleaseTests.test_linked_asset_rejected) ... ok
test_manifest_cli_runs_without_network (test_release.ReleaseTests.test_manifest_cli_runs_without_network) ... ok
test_manifest_hashes_actual_bytes_and_binds_release (test_release.ReleaseTests.test_manifest_hashes_actual_bytes_and_binds_release) ... ok
test_missing_live_policy_never_creates_release_and_restores_token (test_release.ReleaseTests.test_missing_live_policy_never_creates_release_and_restores_token) ... ok
test_nonimmutable_publication_is_reported_as_failure (test_release.ReleaseTests.test_nonimmutable_publication_is_reported_as_failure) ... ok
test_policy_reader_is_separate_from_publication_identity (test_release.ReleaseTests.test_policy_reader_is_separate_from_publication_identity) ... ok
test_prerelease_suffix_is_preserved_during_publication (test_release.ReleaseTests.test_prerelease_suffix_is_preserved_during_publication) ... ok
test_publishes_only_after_full_download_and_all_four_proofs (test_release.ReleaseTests.test_publishes_only_after_full_download_and_all_four_proofs) ... ok
test_rerun_cannot_rewrite_successful_release (test_release.ReleaseTests.test_rerun_cannot_rewrite_successful_release) ... ok
test_tampered_upload_stays_draft (test_release.ReleaseTests.test_tampered_upload_stays_draft) ... ok
test_unexpected_remote_asset_stays_draft (test_release.ReleaseTests.test_unexpected_remote_asset_stays_draft) ... ok
test_wrong_source_tag_or_workflow_never_create_release (test_release.ReleaseTests.test_wrong_source_tag_or_workflow_never_create_release) ... ok

----------------------------------------------------------------------
Ran 23 tests in 0.091s

OK

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 44b54dd303

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +68 to +69
permission-administration: read
permission-environments: read

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reconcile the policy token permissions with ruleset visibility

With the documented read-only App configuration, the policy job cannot pass its ruleset audit even when administrators apply the supplied ruleset correctly. GitHub's ruleset API documentation states that bypass_actors is returned only when the caller has write access to the ruleset, but this token requests Administration read only. Consequently, audit_policy() receives rules without that property, and check_policy() rejects them because it requires rule.get('bypass_actors') == [], blocking every release before the build. Reconcile the policy-reader design with this API restriction in both token-minting steps without treating an omitted bypass list as proof that no bypass exists.

Useful? React with 👍 / 👎.

@ByteYue ByteYue Oct 29, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

确认成立,已按 GitHub 官方文档核对:bypass_actors 对仅有 Administration-read 的调用者不可见,原有只读 policy App 不能完成完整策略验收。官方契约。

提交 11749e763a 先修复诊断和误导性 setup 文档:字段缺失时明确报告权限/可见性不足,仍然 fail closed;新增回归测试,24 个 helper tests、actionlint、diff/secret scans 均通过。没有将遗漏字段视为 [],也没有提高 App 权限。

完整修复需要 owner 明确批准足够可见性的隔离读取方案,或独立可验证的替代方案;这会改变外部权限边界,本轮没有擅自授予 Administration-write。该 P1 保持未解决,PR body 和 changelog 都已标为发布阻塞。

用户决策(2026-10-30,Asia/Shanghai):明确选择保持只读 App、保留发布阻塞。本轮不增加 Administration:write,不将缺失 bypass_actors 当作空列表;此 P1 保持未解决,后续另行设计能完整审计且满足权限隔离的方案。

@nekomoto911 nekomoto911 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

只读安全评审后 approve(基于 head 11749e76)。

验证到的:no --clobber、assert-new 拒重复 tag(含 draft)、发布后 immutable=true 复核、现场读 tag ruleset 审计、5 个三方 action 全钉官方 commit SHA、仅 publish job 获 id-token、build job 不碰任何 secret、所有 run 块 set -euo pipefail、无 || true/continue-on-error。第二轮新增 check-abi(readelf 拒 GLIBC>2.35、缺版本信息 fail-closed)与对隐藏 bypass_actors 的显式处理都为 fail-closed,无新旁路。

approve 不等于可直接上线,仍需 owner 侧(PR 自己已声明):

  • 打开 immutable releases + 建 tag ruleset + 配 release environment + 装 policy App。
  • P1 卡点:policy App 只有 Administration-read,GitHub 不向其返回 ruleset bypass_actors → CI 无法自证 tag ruleset 无人可绕;需 owner 批准一个有 ruleset-write 可见性的 reader 设计或等效可验证方案。
  • 历史 tag 仍存旧版 release.yml(带 --clobber、共享 runner)→ 须禁止对老 ref 发布并停用共享 runner 的发布能力。

@nekomoto911
nekomoto911 merged commit 005dae6 into main Nov 4, 2026
56 of 58 checks passed
@nekomoto911
nekomoto911 deleted the codex/issue165-release-hardening branch November 4, 2026 02:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants