ci: Windows CPU integration smoke job + OS label routing fix (RoFormer Windows plan, part 1)#294
Merged
beveradb merged 4 commits intoJul 20, 2026
Conversation
- New windows-cpu-integration job on windows-latest (Python 3.12, -E cpu): runs one model per DirectML-relevant architecture (RoFormer, MDX, VR) end-to-end via the existing parametrized CLI test, with actions/cache for model files. Reported in the gate job but non-blocking until stabilized (promotion to ruleset 529535 comes later). - Existing GPU jobs now use runs-on [self-hosted, linux, gpu]: GitHub matches any runner whose labels are a superset, so bare [self-hosted, gpu] jobs could be scheduled onto the upcoming Windows GPU runners. - Rewrite stale docs/CI-GPU-RUNNERS.md: it described the fixed VM pool deleted 2026-05-18; now documents the ephemeral JIT-runner architecture, label conventions, Windows coverage tiers, and current troubleshooting paths. - Add docs/archive plan for RoFormer Windows/DirectML support (#292). Part 1 of the RoFormer-on-Windows plan (issue #292). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… quality-floor) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 20, 2026
feat(runner-manager): Windows GPU ephemeral runner family (gpu-windows)
nomadkaraoke/karaoke-gen#881
Merged
…ndows CI On windows-latest, 'poetry run pytest' does not put the venv Scripts dir on the subprocess search PATH, so subprocess.run(["audio-separator", ...]) failed with WinError 2. Console scripts always sit next to the venv interpreter on both platforms, so resolve via shutil.which with a sys.executable-relative fallback. No behavior change on Linux (which() finds the same executable that bare invocation used). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Separator raises FileNotFoundError when AUDIO_SEPARATOR_MODEL_DIR points at a missing directory; hosted runners have no pre-baked model dir and an actions/cache miss leaves it absent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
beveradb
added a commit
that referenced
this pull request
Jul 20, 2026
…#292) (#295) * fix: load Roformer state dicts on CPU for DirectML devices (#292) torch-directml's torch.load deserialization hook expects integer device ids and raises TypeError ("'>=' not supported between instances of 'torch.device' and 'int'") when map_location targets a privateuseone device. This made the new Roformer implementation fail on every DirectML load and silently fall back to the legacy path (whose CPU map_location got further, then died on ComplexFloat at inference — separate fix). Load the state dict on CPU and move the model to the device afterwards, gated on DML so cuda/mps/cpu keep byte-identical behavior. Part 3 of the RoFormer-on-Windows plan (see #294 for the plan doc). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: route Roformer complex ops to CPU on DirectML devices (#292) torch-directml has no complex tensor support — torch.stft with return_complex, view_as_complex, complex multiplies and torch.istft all fail with 'Invalid or unsupported data type ComplexFloat'. Hop exactly those ops to CPU when the input lives on a privateuseone device, keeping the transformer stack (the heavy compute) on the DML device. This is the same pattern the MDX arch already uses in uvr_lib_v5/stft.py (which is why MDX works on DirectML today) and mirrors the existing MPS istft hop. All hops are gated on _is_dml_device(); cuda/mps/cpu behavior is unchanged — the MPS branches are untouched. The helper is module-level so tests can force the DML branches on CPU tensors: new equivalence tests assert the hopped path produces identical output to the normal path for both BSRoformer and MelBandRoformer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
beveradb
added a commit
to nomadkaraoke/karaoke-gen
that referenced
this pull request
Jul 20, 2026
…s) (#881) ## Summary Adds a Windows Server + T4 family to the ephemeral GHA runner fleet, following the existing pattern exactly (JIT-config single-use VMs from pre-baked image families). Built for python-audio-separator's upcoming `windows-directml` integration tests — part 2 of the RoFormer-on-Windows plan (nomadkaraoke/python-audio-separator#292; see plan in nomadkaraoke/python-audio-separator#294). ### Dispatcher (`ephemeral.py`) - `FamilySpec.os_label` (default `linux`) replaces the hardcoded `linux` advertised label - New `gpu-windows` family: n1-standard-4 + T4, `gha-runner-gpu-windows` image family, 100GB disk - `resolve_family` precedence: windows → gpu → build → general - Windows VMs get the JIT startup script via the `windows-startup-script-ps1` metadata key (`run.cmd --jitconfig` + guaranteed shutdown in `finally`); Linux families unchanged - +8 unit tests (41 pass): family routing, advertised labels, per-family startup metadata key, TERMINATE scheduling ### Image bake - `runner-image-provision.ps1`: NVIDIA **GRID** driver (WDDM mode — the datacenter driver runs T4 in TCC mode with no DirectX, which breaks DirectML), with TCC→WDDM switch + bounded reboot retries; Python 3.12 (torch-directml has no 3.13 wheels); Git, FFmpeg, system-wide Poetry, actions-runner win-x64; lean DirectML model set at `C:\audio-separator-models`. Idempotent via phase markers (startup scripts re-run every boot); emits the same READY/FAILED serial-console markers as the Linux provisioner so the existing wait step works unchanged. - `build-runner-images.yml`: `gpu-windows` variant (windows-cloud/`windows-2022` base, per-variant image + startup-metadata flags), included in monthly "all" builds - `docs/EPHEMERAL-GHA-RUNNERS.md`: fourth image family + label-routing rules ## Deploy sequence (operator) 1. **Before merge: `pulumi up` locally** (runner-manager function redeploys with the new family — my ADC is read-only so I could not run it) 2. Merge, then bake the first image: `gh workflow run build-runner-images.yml -f variants=gpu-windows` 3. First consumer job (`windows-directml`, `runs-on: [self-hosted, windows, gpu]`) lands in python-audio-separator next — it doubles as the verification job The companion label fix (`runs-on: [self-hosted, linux, gpu]` on existing jobs, so Windows runners can't steal Linux jobs) is already in nomadkaraoke/python-audio-separator#294. ## Testing - `test_ephemeral.py`: 41/41 pass - Workflow YAML validated; CodeRabbit local review run, findings applied (windows-2022 family name, bounded WDDM retry, reboot on installer exit 1, atomic downloads) - Real-world validation happens at first bake + first `windows-directml` job (next PR) @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part 1 of the plan to make RoFormer models work on Windows/DirectML (#292) and make Windows a genuinely supported platform. This PR is CI + docs only — no behavior changes.
windows-cpu-integrationjob on GitHub-hostedwindows-latest(free): end-to-end separation of one model per DirectML-relevant architecture (MelBandRoformer, MDX, VR) on the 20s fixture via the existing parametrized CLI test, withactions/cachefor model downloads. Python 3.12 to match the upcoming DirectML jobs (torch-directml has no 3.13 wheels). Reported in the gate job but non-blocking until stabilized.runs-on: [self-hosted, linux, gpu]. GitHub schedules a job onto any runner whose labels are a superset of the job's — a bare[self-hosted, gpu]job could otherwise get scheduled onto the upcoming Windows GPU runners. No-op today (Linux runners already advertiselinux), prerequisite for adding the Windows family.docs/CI-GPU-RUNNERS.md: it still described the fixed VM pool deleted 2026-05-18 (karaoke-gen #780); now documents the ephemeral JIT-runner architecture, label conventions, the Windows coverage tiers, and current troubleshooting paths (including the PAT-403 dispatch failure mode).docs/archive/2026-07-19-roformer-windows-directml-plan.md.Next steps (separate PRs)
gpu-windowsephemeral runner family +gha-runner-gpu-windowsimage bakemap_locationfix +windows-directmljob (continue-on-error)Testing
-kselection verified to collect exactly the 3 intended model paramswindows-cpu-integrationcheck result@coderabbitai ignore
🤖 Generated with Claude Code