Skip to content

ci: build kona-zkvm-canary in its own melange stack - #23288

Merged
geoknee merged 1 commit into
developfrom
geoknee/kona-zkvm-canary-melange-stack
Oct 12, 2026
Merged

geoknee merged 1 commit into
developfrom
geoknee/kona-zkvm-canary-melange-stack

Conversation

@geoknee

@geoknee geoknee commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Why

kona-zkvm-canary was a subpackage of melange/op-stack-rust.yaml. Its SP1 profiling feature has to stay out of the combined build, so it already ran its own cargo build with its own target tree (rust_kona_sp1_crates_zkvm-canary). It shared nothing with the main build, but melange runs subpackage pipelines after the main pipeline. So every op-stack-rust leg paid for two fat-LTO builds back to back.

Develop run 37809128424:

leg main 6-binary build canary build (after it) job total
aarch64 24m47s 22m42s 50m
x86_64 18m04s 14m09s 35m

Every image built from op-stack-rust waited for it, including kona-node/v*, kona-host/v* and kona-client/v* release tags.

Change

  • New melange/kona-zkvm-canary.yaml, the same build step moved verbatim: same build environment, install-dir, modroot, profile and strip.
  • The kona-zkvm-canary subpackage is removed from op-stack-rust.yaml.
  • In images.apko.json, a new kona-zkvm-canary melange stack (large runner, GHA cache keyed on rust/Cargo.lock), and the canary image now needs_melange: ["kona-zkvm-canary"].

The factory apko-plan script (pinned sha b71bfb2), run locally against this catalog, gives:

  • develop push: separate kona-zkvm-canary/{x86_64,aarch64} legs. kona-node, kona-host, kona-client and op-challenger still depend on op-stack-rust only.
  • kona-node/v1.8.0-rc.3 tag: melange builds only op-stack-rust, with no canary.
  • kona-zkvm-canary/v0.2.0 tag: only the canary stack, with build_version=v0.2.0.

Expected effect

  • Develop: about 50m → about 25m on aarch64 and 35m → 18m on x86 for the melange stage. The apko job still waits for the whole melange matrix, so the canary leg runs in parallel rather than vanishing.
  • op-stack-rust release tags skip the canary entirely.
  • Cost: two more large runner jobs per rust build. The canary's target tree moves out of the op-stack-rust cache into its own cache entry; CARGO_HOME is now duplicated across the two.

Caveats for review

  • The first develop run after merge is cold for the canary leg. It gets a new cache key, so expect one full maxperf build of kona-host + SP1. The op-stack-rust cache still restores, minus the canary tree it no longer needs.
  • This PR edits .github/images.apko.json, a shared_paths entry, so CI here plans the full catalog. That exercises the new stack end to end on fast-build.
  • Independent of fix: key melange cache by cargo profile factory#68 (cache key by cargo profile). Each helps on its own.

🤖 Generated with Claude Code

kona-zkvm-canary was an op-stack-rust subpackage, but its SP1 profiling
feature keeps it out of the combined cargo build, so it already ran a
second cargo invocation with its own target tree. That shared nothing with
the main build yet ran after it: a second fat-LTO link of kona-host + SP1
on every op-stack-rust leg (22m42s aarch64 / 14m09s x86_64 on develop
run 37809128424). Every image depending on op-stack-rust, including
kona-node, kona-host and kona-client releases, waited for it.

Give the canary its own melange stack with the same runner profile and
GHA cache settings, so it builds in parallel and op-stack-rust tags no
longer build it at all.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@geoknee
geoknee marked this pull request as ready for review October 9, 2026 09:54
@geoknee
geoknee requested a review from a team as a code owner October 9, 2026 09:54
@geoknee
geoknee added this pull request to the merge queue Oct 9, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to no response for status checks Oct 9, 2026
@geoknee
geoknee added this pull request to the merge queue Oct 9, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 9, 2026
@geoknee
geoknee added this pull request to the merge queue Oct 12, 2026
Merged via the queue into develop with commit e33c772 Oct 12, 2026
155 checks passed
@geoknee
geoknee deleted the geoknee/kona-zkvm-canary-melange-stack branch October 12, 2026 09:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants