feat(routing): build the routable model candidate pool - #8891
Conversation
Share provider-aware model policy checks and preserve native Copilot capabilities. Add frozen catalogue snapshots and deterministic endpoint-compatible candidate pools. Leave routing modules disconnected from the running proxy. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Documentation PreviewDocumentation has been built for this PR. To view locally:
Built from commit 4c62807 |
✅ Coverage Check PassedOverall Coverage
📁 Per-file Coverage Changes (1 files)
Coverage comparison generated by |
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The policy-bypassing resolver fallback and null-policy crash must be fixed before approval.
Get a fresh assessment by requesting another Copilot review.
Review effort: Balanced
Findings: 1
Open (1)
What changed in this PR
Adds an inert Copilot routing catalogue and deterministic candidate pool while centralizing provider-aware model-policy matching.
Changes:
- Adds frozen routing catalogues and filtered candidate generation.
- Preserves private Copilot routing metadata and native-token identity.
- Updates policy enforcement, tests, and documentation.
| File | Review |
|---|---|
docs/awf-config-spec.md |
Documents routing and policy behavior. |
containers/api-proxy/runtime-model-catalog.test.js |
Tests private routing metadata retention. |
containers/api-proxy/runtime-model-catalog.js |
Preserves copied routing metadata. |
containers/api-proxy/routing-catalogue.test.js |
Tests frozen catalogue snapshots. |
containers/api-proxy/routing-catalogue.js |
Builds authoritative Copilot catalogue records. |
containers/api-proxy/routing-candidates.test.js |
Tests candidate filtering, ordering, and protocols. |
containers/api-proxy/routing-candidates.js |
Moderate: A null policy causes dereference errors; normalize it to an empty unrestricted policy. |
containers/api-proxy/providers/copilot.js |
Exposes native Copilot routing identity. |
containers/api-proxy/provider-pricing-overlays.js |
Exports provider aliases. |
containers/api-proxy/model-resolver.test.js |
Tests provider-aware policy resolution. |
containers/api-proxy/model-resolver.js |
Critical: The no-alias middle-power fallback can select a disallowed model. Apply the policy predicate to fallback selection or disable fallback when policy is active. |
containers/api-proxy/guards/model-policy-guard.test.js |
Tests provider-aware guard behavior. |
containers/api-proxy/guards/model-policy-guard.js |
Implements shared provider-aware matching. |
containers/api-proxy/guards/common-guard-checks.js |
Passes provider context to policy guards. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| const familyCandidates = providerModels.filter(m => m.toLowerCase().startsWith(familyPrefix)); | ||
| const permittedCandidates = modelPolicyConfig | ||
| ? familyCandidates.filter(c => _isModelPermittedByPolicy(c, modelPolicyConfig)) | ||
| ? familyCandidates.filter(c => _isModelPermittedByPolicy(c, modelPolicyConfig, currentProvider)) |
Dismissed at the pull request owner’s request.
|
@copilot address review feedback |
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Fixed in d3458a5 (refined in a follow-up commit): |
|
✅ Copilot review passed with no inline comments. @lpcox Add the |
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤
|
|
🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅ Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
[!TIP] tools:
github:
mode: gh-proxySee GitHub Tools for more information on To allow these domains, add them to the network:
allowed:
- defaults
- "api.github.com"
- "example.com"See Network Configuration for more information.
|
|
❌ Security Guard failed. Please review the logs for details.
|
|
✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓
|
|
✅ Smoke Claude passed Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "api.anthropic.com"See Network Configuration for more information.
|
|
🌑 The shadows whisper... Smoke Codex failed. The oracle requires further meditation... Warning Firewall blocked 8 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "accounts.google.com"
- "clients2.google.com"
- "github.com"
- "github.githubassets.com"
- "msfeed25.pkgs.visualstudio.com"
- "update.googleapis.com"
- "www.google.com"
- "www.gstatic.com"See Network Configuration for more information.
|
|
🔌 Smoke Services — All services reachable! ✅
|
|
📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅ Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "o205451.ingest.us.sentry.io"See Network Configuration for more information.
|
|
❌ Smoke Gemini reports failed. Facets need polishing...
|
|
❌ Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed...
|
|
❌ Smoke Copilot BYOK AOAI (api-key) reports failed. AOAI BYOK (api-key) mode investigation needed...
|
|
Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.
|
|
Smoke Cloud Hypervisor completed. Cloud Hypervisor + Copilot passed. Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "example.com"
- "github.com"See Network Configuration for more information.
|
|
✅ Build Test Suite completed successfully! Warning Firewall blocked 8 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "api.nuget.org"
- "bun.sh"
- "dc.services.visualstudio.com"
- "deno.land"
- "dl.deno.land"
- "github.com"
- "releaseassets.githubusercontent.com"
- "repo.maven.apache.org"See Network Configuration for more information.
|
|
Smoke Test: Copilot BYOK (Direct) Mode ✅
Result: PASS
|
|
EGRESS_RESULT allow=pass deny=pass ✅ Allowed domain (api.github.com): reachable, HTTP 200 Overall status: PASS Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
[!TIP] tools:
github:
mode: gh-proxySee GitHub Tools for more information on To allow these domains, add them to the network:
allowed:
- defaults
- "api.github.com"
- "example.com"See Network Configuration for more information.
|
github-actions
Bot
added
the
smoke-copilot-network-isolation
Smoke Test: Cloud Hypervisor + Copilot
All checks passed. Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "example.com"
- "github.com"See Network Configuration for more information.
|
Smoke Test: Claude Engine Validation
Overall result: PASS Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "api.anthropic.com"See Network Configuration for more information.
|
|
Smoke Test: Copilot Engine
Overall: PASS cc @lpcox
|
|
Smoke Test: GitHub Actions Services Connectivity
Overall: PASS
|
📡 OTel Tracing Smoke Test Results
Overall: ✅ Pass — module loading, unit tests, env var wiring, and token-tracker hook all validated. Scenario 5 shows expected graceful degradation rather than a regression. Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "o205451.ingest.us.sentry.io"See Network Configuration for more information.
|
Chroot Version Comparison Results
Overall: FAILED — Node.js version differs between host and chroot environment (
|
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — PASS Notes:
Warning Firewall blocked 8 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "api.nuget.org"
- "bun.sh"
- "dc.services.visualstudio.com"
- "deno.land"
- "dl.deno.land"
- "github.com"
- "releaseassets.githubusercontent.com"
- "repo.maven.apache.org"See Network Configuration for more information.
|
Summary
Closes #8872. Builds on the routing configuration and contracts merged in #8853.
routing-catalogue.jsandrouting-candidates.js: frozen authoritative snapshots, deterministicchoice-0001identifiers, null-prototype reverse lookup, native wire names, and context limits. Offer effortless pairs only on chat completions and effort-bearing pairs only on responses; distinguish missing versus empty efforts and exclude unknown-only efforts.Scope and adaptations
Applied to current main (
8a3e2d9b), including the recently merged provider changes. Public APIs, pool shapes, ordering, error codes, and endpoint requirements follow the issue's reference. The reference's NUL pair-key separator is represented as a JavaScript\u0000escape.One additional correctness adaptation: the resolver's existing
autospecial pass-through also checks the shared provider-aware policy. Without that check,auto/copilot/autocould still bypass resolver filtering while the request guard rejected them. Tests cover denylist verification, explicit qualified opt-in, and deny precedence. The legacy no-provider matching path remains unchanged.Not included: runtime imports of the catalogue/candidate modules, routing activation, router execution-catalogue intersection, new policy syntax, fetching/caching changes, Compose/topology changes, or non-Copilot routing.
Validation
Executed with Node v24.8.0 on macOS; Python-dependent tests use installed Python 3.13.15.
npm run type-check: passed.npm run lint: passed with 2,238 existing warnings and zero errors.npm run build: passed.npx markdownlint-cli2 docs/awf-config-spec.md: passed.npx jest --runInBand: 384 suites passed; 6,127 tests passed, 4 existing skips.npm --prefix containers/api-proxy test -- --runInBand: 89 suites / 1,794 tests passed.npm --prefix containers/cli-proxy test -- --runInBand: 1 suite / 78 tests passed.git diff --cached --check: passed.The first root-suite run selected system Python 3.9 and failed Python imports; selecting Python 3.13 resolved those failures. A single enclave harness deadline failure on the targeted retry did not recur in the subsequent complete run. No tests were weakened or newly skipped.
Remaining validation: this host is macOS with UID 1000 and its Docker daemon is unavailable, so the issue's requested Linux/non-root/non-1000 environment was not available locally. Repository CI must pass before merge/issue closure.