Skip to content

ci: run the live suites from the test archive - #12973

Merged
cv merged 4 commits into
v1from
ci/live-from-archive
Oct 11, 2026
Merged

cv merged 4 commits into
v1from
ci/live-from-archive

Conversation

@cv

@cv cv commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator

Failure

Each live job compiles the workspace's test executables again, although Build and test already built and archived the same commit's tests for the lifecycle shards. Before compiling, each job also restores the Rust cache and compiles nemoclaw-build to decide whether the pull request can affect it. With #12970, live jobs set when the Linux Test / PLATFORM checks finish, so this repeated work delays required checks:

In the last warm v1 run (38070042632), each live job spent 107–128 s before its suite started:

  • Rust setup: 25–33 s, including restoring a 1 GB cache.
  • cargo ci build: 63–95 s.

Decision

Run the live suites the way the lifecycle shards already run:

  • Archive: on Linux, cargo ci archive also packs the test executables the live-docker and live-kind nextest profiles select. Selection still comes from the profiles, so there's no second list to maintain. Other platforms archive only the lifecycle suite.
  • Runner: ci live-docker and ci live-kind accept --archive-file, like ci lifecycle. --partition stays lifecycle-only.
  • Decision: Build and test runs nemoclaw-build changes live, since it already has the build tool compiled. Live jobs skip when the change can't affect them, so they no longer start a runner just to decide.
  • Live jobs: each waits for Build and test and Build bundle, then downloads both artifacts. It runs .build/ci/nemoclaw-build ci SUITE --archive-file, with no Rust setup and no compiling. It restores the downloads cache read-only, for Kind's pinned tools.

In the last four warm v1 runs, Linux Build and test finished 9–110 s before Build bundle. This change adds a few seconds to Build and test (a larger archive and the decision step), so occasionally the live jobs may start a few seconds later.

Validation

  • New tests: three, all failing before the change. On Linux, the archive holds the live suites; on other platforms, it holds only the lifecycle suite; the live steps accept an archive. The option-rejection test now covers live-docker --partition.
  • Real archive on Linux ARM64: listing from the archive gives exactly the compiled listing for all three profiles: 128 lifecycle tests, 10 Docker, and 5 Kind.
  • Archive size: 184 MB on v1, 293 MB now, mostly nemoclaw-build's and nemoclaw-runtime's test executables.
  • cargo ci passed in 523 s: 1,329 workspace tests and 128 lifecycle tests.
  • actionlint passes.
  • Hosted: a run-live/ push of this commit passed (38085576274). Setup before each suite dropped to 14–35 s, from 107–128 s, except one 117-second artifact download on ARM64 Kind. Whole live jobs vary more than that from run to run; see the comment below.

Base automatically changed from ci/require-live-suites to v1 October 10, 2026 20:55
@cv

cv commented Oct 10, 2026

Copy link
Copy Markdown
Collaborator Author

Hosted run on run-live/live-from-archive (same commit): 38085576274. Every job passed, including all four Test / PLATFORM gates, and macOS and Windows skipped their live jobs.

Job Last warm v1 run (38070042632) This change
linux_arm64 / Live / Kind 485 s 524 s
linux_arm64 / Live / Docker 561 s 474 s
linux_amd64 / Live / Kind 682 s 700 s
linux_amd64 / Live / Docker 803 s 708 s
  • No compiling on the runner: every Compiling line in the live logs comes from the image builds inside Docker.
  • Test counts match the archive listing: 10 Docker tests and 5 Kind tests on each Linux platform.
  • Cache state doesn't affect these numbers: the live jobs no longer restore Rust caches, so this cold branch run times them as a warm run would.
  • One slow download: the ARM64 Kind job took 117 s to download the test archive, while the Docker job downloaded the same artifact at the same moment in 6 s. I'm treating it as runner variance.

@cv

cv commented Oct 10, 2026

Copy link
Copy Markdown
Collaborator Author

Correction to the table above: whole-job times mix in run-to-run variance inside the suites. AMD64 Kind's image build inside Docker took 89 s here and 57 s before, for example. Measured from job start until the suite starts, setup dropped from 107–128 s to 14–35 s, a saving of 75–106 s per live job. The one exception is ARM64 Kind, at 152 s because of its 117-second download. I've updated the description to match.

# Conflicts:
#	.github/workflows/native-platform.yml
#	docs/contributing/testing.md
@github-actions

Copy link
Copy Markdown
Contributor

@cv
cv merged commit 7e76e80 into v1 Oct 11, 2026
43 checks passed
@cv
cv deleted the ci/live-from-archive branch October 11, 2026 10:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant