bootstrap: add bootstrap step to run stdarch-gen checks in CI

[xonx4l]

View all comments

This PR ports stdarch-gen checks into the bootstrap test runner as a step, so that we can run the stdarch-gen checks via x.py test.

Changes -:
-> Adds a new StdarchGenCheck bootstrap step.
-> Mirrors https://github.com/rust-lang/stdarch/blob/f71ac102d30d5d1957277cbffc6ba631b67bffd1/.github/workflows/main.yml#L309-L333 (check-stdarch-gen job)
-> stdarch-gen-arm uses a nightly only feature (#![feature(pattern)]) but bootstrap's cargo runs on the beta toolchain thus we set RUSTC_BOOTSTRAP=1 on the child process so it builds.
-> The generators call out to rustfmt to format their output so rustfmt needs to be on PATH. So added bootstrap's rustfmt to PATH before each run. If rustfmt isn't available, the step fails early and tells the user to set rustfmt = true in bootstrap.toml .

r? @Kobzol

[rustbot]

Kobzol is not on the review rotation at the moment.
They may take a while to respond.

[rustbot]

stdarch is developed in its own repository. If possible, consider making this change to rust-lang/stdarch instead.

cc @Amanieu, @folkertdev, @sayantn

[Kobzol]

Thank you! I left some comments, the most important one is the question about why things are copied over.

View changes since this review

[rustbot]

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

[Kobzol]

@bors try jobs=x86_64-gnu

[rust-bors]

💔 Test for bb3dc02 failed: CI. Failed job:

• try - x86_64-gnu (web logs, enhanced plaintext logs)

[JonathanBrouwer]

@bors p=6 scheduling

[rust-bors]

💔 Test for 850bd26 failed: CI. Failed job:

• auto - dist-powerpc-linux (web logs, enhanced plaintext logs)

[rust-log-analyzer]

A job failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

##[endgroup]
Image input checksum ba2e589c81df26c6140514775ac20a4c4656764d352845327fe1ea4b3f33521601b09fd4b0b4608a2083e211e333dc943883d277e972f6a3af3f7bde61ac0b14
##[group]Building docker image for dist-powerpc-linux
Docker version 28.0.4, build b8034c0
Error response from daemon: Get "https://ghcr.io/v2/": Get "https://ghcr.io/token?account=rust-lang&client_id=docker&offline_token=true&service=ghcr.io": net/http: request canceled (Client.Timeout exceeded while awaiting headers) (Client.Timeout exceeded while awaiting headers)
##[error]Process completed with exit code 1.
##[group]Run echo "disk usage:"
echo "disk usage:"
df -h
shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}

[Kobzol]

@bors retry Spurious

[rust-bors]

💔 Test for a4af1f8 failed: CI. Failed job:

• auto - dist-x86_64-linux (web logs, enhanced plaintext logs)

[JonathanBrouwer]

@bors retry

[JonathanBrouwer]

@bors p=0

[JonathanBrouwer]

@bors p=10 scheduling

[rust-log-analyzer]

The job x86_64-gnu-distcheck failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 21.62µs

##[endgroup]
 finished in 25.646 seconds
error: no matching package named `proc-macro2` found
location searched: directory source `/tmp/distcheck/distcheck-rustc-src/library/vendor` (which is replacing registry `crates-io`)
required by package `assert-instr-macro v0.1.0 (/tmp/distcheck/distcheck-rustc-src/library/stdarch/crates/assert-instr-macro)`
Command `/tmp/distcheck/distcheck-rustc-src/build/x86_64-unknown-linux-gnu/stage0/bin/cargo run --bin stdarch-gen-arm --release -- crates/stdarch-gen-arm/spec crates/core_arch/src [workdir=/tmp/distcheck/distcheck-rustc-src/library/stdarch]` failed with exit code 101
Created at: src/bootstrap/src/core/build_steps/test.rs:1275:27
Executed at: src/bootstrap/src/core/build_steps/test.rs:1284:17

Command has failed. Rerun with -v to see more details.
Build completed unsuccessfully in 1:19:32
make: *** [Makefile:49: check] Error 1
Command `make check [workdir=/tmp/distcheck/distcheck-rustc-src]` failed with exit code 2
Created at: src/bootstrap/src/core/build_steps/test.rs:3980:5
Executed at: src/bootstrap/src/core/build_steps/test.rs:3986:10

Command has failed. Rerun with -v to see more details.
Bootstrap failed while executing `test distcheck`
Currently active steps:
test::Distcheck {  } at src/bootstrap/src/core/build_steps/test.rs:3932
Build completed unsuccessfully in 1:27:19
  local time: Thu Aug 20 19:38:12 UTC 2026

[rust-bors]

💔 Test for 337f0e3 failed: CI. Failed job:

• auto - x86_64-gnu-distcheck (web logs, enhanced plaintext logs)

[Kobzol]

Looks like it doesn't work with vendoring. I think we encountered this in the past, but I don't remember what the exact cause/fix was. @bjorn3 if we run tests for a separate Cargo workspace, do we have to vendor it? I forgot where that is being done.

[bjorn3]

Everything we build has to be vendored. Vendoring is probably blocked on rust-lang/stdarch#2202 to get a determinstic lockfile in the first place. And even with that, the lockfile wouldn't be available in the vendor step unless we run stdarch-gen inside the vendor step. The list of vendored workspaces is at

rust/src/bootstrap/src/core/build_steps/vendor.rs

Lines 21 to 32 in 0cfa1c4

	("src/tools/cargo/Cargo.toml", vec!["src/tools/cargo"]),
	("src/tools/clippy/clippy_test_deps/Cargo.toml", vec![]),
	("src/tools/rust-analyzer/Cargo.toml", vec![]),
	("compiler/rustc_codegen_cranelift/Cargo.toml", vec![]),
	("compiler/rustc_codegen_gcc/Cargo.toml", vec![]),
	("library/Cargo.toml", vec![]),
	("library/stdarch/Cargo.toml", vec![]),
	("src/bootstrap/Cargo.toml", vec![]),
	("src/tools/rustbook/Cargo.toml", SUBMODULES_FOR_RUSTBOOK.into()),
	("src/tools/rustc-perf/Cargo.toml", vec!["src/tools/rustc-perf"]),
	("src/tools/opt-dist/Cargo.toml", vec![]),
	("src/doc/book/packages/trpl/Cargo.toml", vec![]),

This would need both the workspace containing stdarch-gen as well as the one generated by stdarch-gen.