feat: prototype compile-first Rust CPU/GPU backends (do not merge)

[ericksoa]

Outcome

DO NOT MERGE OR SHIP. This draft preserves Rust's semantic frontend and adds
compile-time-first CPU/GPU object backends and fast/balanced/release controls.
The final bounded test found zero eligible whole modules out of all 3,536
NemoClaw modules on both macOS and Linux. Park this GPU route and retain the
CPU controls and evidence. This result applies to the implemented leaf emitter;
it does not rule out other compiler architectures.

Implementation

• Capture exact frontend bitcode before object replacement, with exclusively
  owned manifests and raw emission receipts.
• Census every captured module, verifying bytes, hashes, source/argument identity,
  actual optimization level, target, costs and complete input multiplicities.
  Require both strict GPU export and the native object contract.
• Replay accepted captured modules through retained parallel CPU and physical
  GPU services, alternating corpus rounds and validating stable object bytes.
  Never enlarge the workload by copying functions. Zero eligibility stops before
  GPU initialization.
• Repeat the complete offline Linux capture/census on the trusted RTX PRO 6000
  runner. Retain bitcode, receipts, exact-head identity and reports. No package
  or image is published.

The continuation gate requires a repeatable 10% improvement in complete fast
builds against the strongest compatible CPU backend, preserved semantics, and
separate 1.5-times compiler/program size limits. The current GPU route fails the
available-work gate, so a persistent Cargo-wide GPU daemon and multi-GPU testing
are deferred. Whole-build GPU acceleration and hardware scaling remain unproved.

Verification

All three experimental jobs passed at
9228c79aed9a74780d07832c4966b7684651f0e1:
run 38111445606.

• Linux: a complete LLVM-bitcode NemoClaw build ran --version; all 3,536
  unique modules were captured and rejected by the current whole-module GPU
  contract. Most first failures were globals (2,617) or signatures (724).
• Mac: the same complete census found 0/3,536 eligible. All six modules initially
  rejected by the 2 MiB input cap also contain globals, verified separately with
  pinned LLVM tools; lifting only that cap adds no eligible work.
• A control Rust fixture had one eligible and seven rejected modules.
  Persistent CPU/Metal replay passed five rounds with identical objects.
  A separate two-module physical Metal test passed three alternating rounds.
  These fixtures validate tooling, not NemoClaw acceleration.
• CUDA full-Rust fixture passed physical GPU object emission, linking/execution,
  bitcode-only frontend/saved linking, and unchanged cache reuse: one GPU module
  and seven attributed CPU modules.
• Local repository-required cargo ci passed (601 seconds), including 1,290
  tests and 120 lifecycle tests. Independent compiler/backend tests, strict
  Clippy, formatting, whitespace and workflow actionlint passed.

Ordinary Linux PR checks fail on a Kubernetes test/API mismatch already present
in their merge base a49fc77ba43eba2c4fdbcecbf4c875f2f1ea80fb:
TofuWorkspace::new expects &Bundle, while the test supplies PathBuf.
That test does not exist on this experiment's exact branch head. The experimental
workflow checks out the verified head directly. No production tests were changed.

Measurements and limits

The same-head hosted native CPU comparison observed release Cargo time
280.693 seconds versus fast 172.246 seconds, 38.64% less.
Consistently stripped files measured 37,460,488 versus 44,006,976 bytes,
17.48% larger, within the program limit. This is a single sequential
release-then-fast comparison with separate target directories; source/file caches
and native build-script optimization can affect it. It is not randomized
qualification or GPU acceleration.

The Mac corpus audit's generated executable is 32.10% larger than its
declared reference and passes the program limit. Current Mac runtime compiler
assets are 235,153,385 bytes versus 222,966,640, 5.466% larger.
Accounting includes required adapters/kernels and non-system linked libraries
once, with shared LLVM reused. Validation tools, caches, development tools and
common Cargo/std/system assets are excluded. Linux/CUDA compiler footprint
remains unqualified.

Persistent CPU emission and whole Linux TPDE measurements remain research leads.
See FINAL_GPU_GATE.md, COMPILE_FIRST.md, POLICY.md and
fast-backend/NATIVE_PROOF.md under experiments/gpu-rust-compiler/.
Keep this PR open as a draft and excluded from releases.

---

Signed-off-by: Aaron Erickson aerickson@nvidia.com

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[github-actions]

v1 documentation preview: https://nvidia-preview-nemoclaw-v1-pr-12951.docs.buildwithfern.com/nemoclaw