Repository navigation
Conversation
|
NemoClaw-E2E-Growth: approve b56f349dea062d8a839d232d89919b1a120f9727e8404d61a1717a24befa3f3a |
…rity Signed-off-by: Deepak Jain <deepujain@gmail.com>
Signed-off-by: Deepak Jain <deepujain@gmail.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Code coverage overviewLanguages: TypeScript TypeScript / code-coverage/plugin
View coverage of the 10 most impacted files.
ℹ️ Calculations represent line coverage. Results may change as calculations are updated. Updated |
|
NemoClaw-E2E-Growth: approve 1360e1dc6a23f4601f8ab14eebfb1d4d9a54854de95623a6007a73f6bc622174 |
|
NemoClaw-E2E-Growth: approve 647470d10d59d3fe7716d3e96c48ca5253d411b68d6bcf6a11e25ca18e029dd5 |
Signed-off-by: Deepak Jain <deepujain@gmail.com>
|
🌿 Preview your docs: https://nvidia-preview-pr-12889.docs.buildwithfern.com/nemoclaw |
|
NemoClaw-E2E-Growth: approve 21603c26db9785e242ee55b99eb4b818b509c5e7666ccdf6116fe315f46e8b69 |
|
Qualification scope: deterministic selection, credential-reference, identity, and fixture behavior are covered locally. Live qualification is limited to the real OpenShell boundary: fresh native custom onboarding, a fresh agent request through the attached provider, restart authority, peer isolation, and owned-resource cleanup. The existing inference-routing target now includes native Anthropic Messages; host-local Anthropic passes do not establish that path. Optional Brev access was checked using the existing running instance. CLI login/listing succeeded, but SSH failed with public-key authentication after one supported refresh and retry. No remote test ran and no instance was created or changed. This external limitation does not stop the GitHub qualification path. Native Podman qualification separately encounters unavailable artifacts at its trusted toolchain pin; its earlier failure occurred before candidate execution. The onboarding repair and actual main-conflict integration are committed locally. The remaining Anthropic evidence repair is being validated as one batch before publication. Final-candidate runtime evidence and Pi image receipts remain pending; this PR is still draft. |
|
NemoClaw-E2E-Growth: approve 0f29b8054e04df3d5614865b4b363f43f78d209d0ca7d967f1779bb2c5db01e7 |
|
Native runtime qualification remains pending. I split the unchanged TC-INF-11 lifecycle into a bounded 250-minute target because the original file allowed 390 minutes of serial tests before cleanup. All cases and assertions are preserved; the retained routing job now has a 180-minute bound. The trusted main planner does not yet select the new native target, and still gives the retained routing job 75 minutes. I will avoid unsupported or knowingly undersized dispatches and continue applicable checks. This limitation does not establish native restart or credential-resolution success. Brev SSH and the pinned Podman toolchain limitations are already recorded above. No remote Brev test ran. Manual execution is justified only for real OpenShell attachment, credential resolution, agent requests, restart, isolation and cleanup; deterministic catalog/configuration behavior is covered locally. |
|
@coderabbitai full review |
|
| normalizedPinoRedactFixture | ||
| .replace(/(?:Bearer\s+)?openshell:resolve:env:[A-Za-z0-9_]+/giu, "unused") | ||
| .replace(/sk-OPENSHELL-RESOLVE-ENV-[A-Za-z0-9_-]+/gu, (candidate) => | ||
| isSafeCredentialPlaceholder(candidate) ? "unused" : candidate, | ||
| ) | ||
| .replace(/(?:xox[bx]|xapp)-OPENSHELL-RESOLVE-ENV-[A-Za-z0-9_-]+/gu, (candidate) => | ||
| isSafeCredentialPlaceholder(candidate) ? "unused" : candidate, | ||
| ) | ||
| // Generated bundles contain the accepted-placeholder matcher itself. | ||
| // Normalize that exact source fragment without stripping the reserved | ||
| // prefix from malformed placeholder-shaped credential values. | ||
| .replace(/(?:xox[bx]|xapp)-OPENSHELL-RESOLVE-ENV-\[A-Za-z0-9_\]\+/gu, "unused") | ||
| .replaceAll( | ||
| "/^sk-OPENSHELL-RESOLVE-ENV-(?:(?:v[0-9]{1,20}|s[a-f0-9]{64})_)?COMPATIBLE(?:_ANTHROPIC)?_API_KEY$/", | ||
| "unused", | ||
| ) |
| const result = spawnSync( | ||
| "bash", | ||
| [ | ||
| "-c", | ||
| 'source "$1"; assert_identity "$2" test-model initial "$3"', | ||
| "identity-check", | ||
| dcodeFreshReonboardCheck, | ||
| identity, | ||
| expected, | ||
| ], | ||
| { | ||
| encoding: "utf8", | ||
| env: { PATH: DEFAULT_TEST_PATH }, | ||
| }, | ||
| ); |
|
Advisor finding
The bounded review compared an older reviewed commit with this head and included merged upstream work. Please reassess the scope finding against the current base; if this evidence invalidates it, withdraw it, otherwise identify the candidate-owned Spark change. The associated Spark E2E recommendations have the same attribution issue. I am preserving the upstream workflow and the agreed Slice3 runtime scope. The Advisor gate remains failed; this response does not waive it or claim merge readiness. |
|
PR Review Advisor finished for commit Request review only when Require no Advisor blockers is green. |
|
NemoClaw-E2E-Growth: approve 92e6d8923acbda85e9d817f68145ccd260a5f7be0e33ec87af201079d1323c2f |
Signed-off-by: Deepak Jain <deepujain@gmail.com>
|
PR Review Advisor finished for commit Request review only when Require no Advisor blockers is green. |
|
PR Review Advisor finished for commit Request review only when Require no Advisor blockers is green. |
Signed-off-by: Deepak Jain <deepujain@gmail.com>
Signed-off-by: Deepak Jain <deepujain@gmail.com>
Signed-off-by: Deepak Jain <deepujain@gmail.com>
|
PR Review Advisor finished for commit Request review only when Require no Advisor blockers is green. |
|
Advisor follow-up for Delivery — F-delivery-flow-1aad0af551d6b58cafc2. The proposed overwrite mechanism conflates a GitHub Actions check run with a commit status. The installer workflow creates check run 116580802825; admission creates a separate commit status under the same name (run 38780980836, status ID 56453878665). Both records coexist on this head. GitHub documents that when a check and commit status have the same required name, both must pass: required-check documentation. Both workflow files are byte-identical to the current PR base. Please withdraw this overwrite finding, or identify a second commit-status producer or a concrete consumer that accepts admission success while the installer check fails. Renaming the admission status without checking its required-gate consumer could change the existing enforcement contract. Documentation — F-documentation-standard-work-e98b4c2d8962bc1f9b2c. The cited recovery command at line 468 immediately follows “If the restart fails, the route and configuration remain committed.” The same section then explicitly says to run the printed command if the gateway restart fails after configuration is committed. completeInferencePostCommit implements that failure-only recovery path. Please withdraw the P1 claim that this instructs an unconditional second restart, or identify the rendering/context that removes that condition. Repeating the condition in the next sentence may be a wording preference; the recorded failure condition already matches the behavior. No findings have been self-waived or marked resolved. The Advisor gate remains failed pending corrected review or disposition. The candidate is unchanged while existing image activation and selected staging E2E runs finish; the existing selected E2E plan is preserved. |
|
PR Review Advisor finished for commit Request review only when Require no Advisor blockers is green. |
|
Evidence-based challenge to the two P1 findings in Advisor run 38824308337, for head
The affected owners and callers are byte-identical across the previous head, this head, and its exact base. That comparison is attribution evidence, not by itself a security proof. These are disputed findings awaiting resolution; this response does not waive either gate or claim approval. |
Signed-off-by: Deepak Jain <deepujain@gmail.com>
Signed-off-by: Deepak Jain <deepujain@gmail.com>
…3201) ## Outcome Fresh custom hosted endpoint selections use an endpoint-specific native provider attached to the selected sandbox. Agent configuration, inference get, probes, and health retain the selected endpoint and API. Native failures do not fall back to the shared route. ## Reason This implements the bounded retry accepted in #13195, using the existing native-provider lifecycle and OpenShell 0.0.116. ### Related issues Fixes #13195. Part of #12558. Refs #12636 and #12889 as historical scope and review evidence; their closure is outside this PR. ## Changes - Bind a generated profile to the validated endpoint, API, credential source, and admitted IPs. Existing checked-in profiles supply the executable and credential boundary. Custom selections cannot safely repoint a public vendor profile. `custom.test.ts` protects profile collisions, SSRF admission, API paths, credential custody, and ambiguous creation. - Reuse verified provider authority without credential rotation. Missing authority and credentials stop before provider mutation. Preserve the existing Bedrock host adapter and legacy HTTPS-pin routes. - Connect custom receipts to existing onboarding, switching, registry ownership, rebuild preflight, configuration, health, and cleanup. Switching removes conflicting access before attachment and before rollback restoration. Tests cover selected-sandbox isolation and retained recovery authority. - Keep existing shared-route selections on their current path. Unavailable legacy routes receive conditional recreation guidance without automatic migration or an asserted cause. - Extend the existing full E2E fixtures to inspect custom provider identity and attachment and retain the native endpoint across restart. The real boundary remains OpenShell credential injection, sandbox networking, agent inference, and restart. Existing live outcome assertions remain; deterministic support tests own route selection and command construction. Bedrock retains its existing live owner. The existing Anthropic switch selector now exercises an authenticated native destination after its native authenticated baseline lifecycle. No new live target or workflow is added. - Extract the existing endpoint normalization functions unchanged so the new consumers fit the repository dependency budgets. Existing imports remain compatible. ## Verification ### Native export endpoint repair (`71a7901`) - Fix the reproduced export rejection from the review: managed native profiles validate the selected endpoint and runtime endpoint against their separate verified receipt fields. Anthropic OpenAI `/v1` translation and the Bedrock adapter remain valid; endpoint drift remains rejected. - All three added positive cases failed before the repair with `Registry and live inference endpoints differ.` Final validation passed 298 tests across `live-export-source.test.ts` and `verify-export-source.test.ts`, including selected/runtime/both-endpoint drift controls. Normal signed-commit hooks passed without increasing budgets. No secret values were read by the regression cases; the diff contains no secrets. - Prior candidate `afea1a7c05f3383bd20d5f79a477c0c8c0179423` completed its PR check rollup with 87 successful and 40 skipped checks, including managed images and GPU qualification. All nine [Advisor specialist reports](https://github.com/NVIDIA/NemoClaw/actions/runs/38858389798) were collected: no P0 and two P1 findings (Hermes native custom switch coverage and public route documentation). Their [recorded deferrals](#13201 (comment)) remain; the separate Advisor blocker gate failed. Prior CI evidence does not establish a pass for this repair commit. - Current-commit CI and scheduled automated reviews are pending. The earlier request-changes review remains for independent reassessment; no approval or merge readiness is claimed. ### Anthropic source selection and native-route merge repair - Previous candidate `afea1a7c05f3383bd20d5f79a477c0c8c0179423` contains selector repair `a472ac7` and one merge of canonical main `a304c5bd67fb0badff1fd5b256dc09d1a1268e1c`. The existing Anthropic switch target now selects changes to its native-provider, onboarding and native-selection owners. Five omitted paths failed before repair; the final boundary suite passed 14 tests. Existing planner coverage passed 136 tests before relocation; these results overlap the earlier combined 150-test run. - Resolve the required main conflicts while preserving custom-hosted endpoint/API/credential authority and main's native-local routing. Integration regressions exposed dropped custom binding and endpoint arguments, TLS readback, and native-local interpretation; these were repaired in their current owners. The final affected route/configuration/recovery batch passed 228 tests in six suites. The onboarding handler passed 38 tests after moving its unchanged native-route exclusion into the existing refresh predicate. Earlier failed integration runs and their diagnoses remain retained. These runs overlap; no summed test total is claimed. - The assertion census passed with 1,272 direct expectations across 77 live files. Strict Pi receipt/source parity passed; the existing paired receipts remain qualified. Original signed-commit and publication hooks passed, including final CLI type checks, without increasing architecture or complexity limits. The diff contains no secrets. - Prior candidate `53bc898` completed evaluation before replacement. Its [CI retry](https://github.com/NVIDIA/NemoClaw/actions/runs/38843616981/attempts/2) passed after one explicitly requested retry. The original 5,313 ms test timeout at an unchanged 5,000 ms deadline remains retained with undetermined cause; the passing retry does not establish infrastructure failure. [Managed images](https://github.com/NVIDIA/NemoClaw/actions/runs/38843616977) and [GPU qualification](https://github.com/NVIDIA/NemoClaw/actions/runs/38843618739/job/116799363127) passed with authenticated candidate and cleanup evidence. CodeRabbit and both scheduled sets of nine Advisor reports completed. These results are historical for the current candidate. - The source-selection finding is repaired here. The later [security review](https://github.com/NVIDIA/NemoClaw/actions/runs/38850077726) identified retained access after removing an admission allowlist entry. The accepted policy preserves verified durable private bindings: unsetting `NEMOCLAW_TRUSTED_PRIVATE_HOSTS` does not revoke an existing binding. Fresh or changed private endpoints still require admission; reuse retains exact endpoint/API/provider identity and admitted IPs. This policy disposition does not establish arbitrary receipt safety or live private-endpoint coverage. Public documentation remains with the accepted post-merge owner. - Current-candidate CI, CodeRabbit, all nine Advisor reports, images/GPU qualification and authorized manual live validation remain pending. The approved live scope is 21 existing manual jobs plus staging Launchable. Protected runtime remains held for the separate timeout follow-up and cleanup reconciliation; no retry allowance, timeout increase or all-pass waiver is introduced. - Three combinations remain documented follow-ups: Hermes native custom Anthropic-to-OpenAI with a real request, fresh public custom OpenAI onboarding, and Pi native credential rewrite followed by a real request. The existing Hermes switch, custom endpoint fixtures and Pi qualification are the nearest coverage, but do not prove those combinations. Extending their contracts is deferred to follow-up work; this PR does not claim that evidence. ### Historical resume credential handoff and merge resolution (`53bc898`) - Published candidate `53bc89879a90e533a191f8b8df2022dcad092ab6` integrates canonical main `b5efa876a775265b63ee963f0b363cc650bdd87e`. The sole conflict was the E2E assertion census. Resolution preserves this PR's existing probe increase and main's protected-runtime count reduction. The census check passed with 1,260 direct expectations across 77 live files. - Fix the single resume defect reported by the migration and operability Advisor reports. When the session has no credential name, resume obtains it from the validated custom attachment for the selected provider and normalized endpoint. Explicit selections take precedence. Actual setup still checks API and live provider identity before reusing authority. Reservation and legacy paths retain their existing behavior. - Two regression tests failed before the fix: the shim returned the default name instead of `TEAM_KEY`, and actual setup required a new credential. Final focused validation passed 58 tests across `resume-provider-shim.test.ts`, `resume-provider-recovery.test.ts`, and `native-provider/custom.test.ts`. Controls cover explicit selections, both custom provider kinds, endpoint/provider mismatch, invalid attachment, normalized endpoint and reuse without credential rotation. - Strict Pi receipt validation and source parity passed for this commit; the existing paired receipts remain valid. Original signed-commit hooks passed. A local dependency-limit failure was repaired by placing receipt lookup in the existing custom-provider owner, without changing budgets. The standalone compiler passed with the existing 8 GiB hook allowance after its default 4 GiB run exhausted the heap. Original publication validation and final CLI type checks passed. The diff contains no secrets. - Evaluation of `b8065df` completed before this repair. [Primary CI](https://github.com/NVIDIA/NemoClaw/actions/runs/38835151923) passed 23 jobs with two legitimate skips. [Managed images](https://github.com/NVIDIA/NemoClaw/actions/runs/38835151800) passed 14 jobs with four skips; AMD64, ARM64 and rootless activation each passed 13 phases and 13 cleanup actions. [GPU qualification](https://github.com/NVIDIA/NemoClaw/actions/runs/38835153672/job/116774805269) passed device approval, cron execution, the later agent turn, all six phases and all three cleanup actions. Podman CPU and portable rootless qualification passed. These results are historical for the new commit. - CodeRabbit and all nine [Advisor reports for `b8065df`](https://github.com/NVIDIA/NemoClaw/actions/runs/38836956611) completed. Two Advisor findings share the resume defect repaired here; public documentation retains its authorized post-merge owner. The CodeRabbit suggestion to split or remove Pi receipts is a false positive: the unchanged contributor contract requires this follow-up in the same PR, and strict source parity passed. The repeated Pi credential-rewrite request remains an unverified combination. - New-commit CI, automated review, image and GPU qualification remain pending. The 23 selected manual E2E jobs have no result for this commit. Retained Hermes, fresh public custom OpenAI and Pi combination gaps remain open. The protected-runtime timeout follow-up remains outside this PR, with NIM, rollback and final fixture cleanup unverified. That scheduled cycle completed before the selector repair. ### Historical Pi qualification receipt follow-up (`b8065df`) - Published candidate `b8065df5b1de956b533c278b77b8ba23a3efba5e` changes only the AMD64/ARM64 Pi qualification receipts and their matching authority hashes. The exact contract bytes came from [successful managed-image run 38826692761, attempt 1](https://github.com/NVIDIA/NemoClaw/actions/runs/38826692761), source `19f521416a5dbb62ac9fa169e5f03eb817c72297`, cohort `ghrun-38826692761-1`. Artifact identity, archive size and SHA-256, source, platform and common run/attempt were authenticated. The authority hashes are computed from the receipt bytes. - Strict Pi receipt validation and source parity passed. Qualified Pi image inputs are unchanged by this three-file update. Normal signed-commit and publication hooks passed with original hook execution inside the verified credential boundary. No new behavior or test was added; the strict receipt/source-parity check verifies this data update. The diff contains no secrets. - Evaluation of source `19f5214` completed: image publication passed 14 jobs with four legitimate skips; AMD64, ARM64 and rootless activation each passed 13 phases and 13 cleanup actions. [GPU qualification](https://github.com/NVIDIA/NemoClaw/actions/runs/38826694610/job/116748692376) passed explicit device approval, cron execution, a later agent turn, all six phases and all three fixture cleanup actions. The original device-denial cause remains unresolved. These are results for the qualified source, with independent new-commit checks still pending. - CodeRabbit completed review of `19f5214`, resolved both routing findings and reported no new actionable finding. All nine [Advisor reports](https://github.com/NVIDIA/NemoClaw/actions/runs/38829342333) completed. The aggregate retained the known Hermes combination gap and public-documentation findings, with prior source-backed and authorized post-merge dispositions. Primary CI failed solely the stale Pi receipt gate addressed here and its aggregate; all 12 CLI shards passed. - Evaluation of `b8065df` completed as summarized above. Applicable manual live E2E evidence for that commit remains unavailable. Historical ordinary E2E passes do not establish its result. Hermes native Anthropic-to-OpenAI, fresh public custom OpenAI onboarding and Pi credential-rewrite combination gaps remain recorded. The protected-runtime timeout stays outside this PR, with NIM, rollback and final fixture cleanup unverified. ### Historical routing API repair (`19f5214`) - Published candidate `19f521416a5dbb62ac9fa169e5f03eb817c72297` fixes the two reproduced custom Anthropic API-consistency findings. Native binding, persisted API selection and agent configuration share one resolved default; explicit native Responses keeps its request path and inference namespace. Fixed-provider identity checks and existing shared-route behavior remain covered. - Focused validation: `vitest run --project cli src/lib/inference/native-provider/custom.test.ts src/lib/onboard/setup-native-hosted.test.ts src/lib/sandbox/compose-sandbox-config-body.test.ts src/lib/inference/config.test.ts` passed 187 tests across four files. After consolidating the API resolver, its three affected suites passed 136 tests, overlapping that total. Normal signed-commit and publication hooks passed without increased complexity, architecture or growth limits. The diff contains no secrets. - Evaluation of `19f5214` completed as summarized above. The genuine source-matching paired Pi receipts are applied in the receipt-only follow-up. The earlier `94c8ec0` receipts remain historical and were not applied to changed routing source. - [GPU qualification on preceding candidate `94c8ec0`](https://github.com/NVIDIA/NemoClaw/actions/runs/38817587355/job/116720057849) passed explicit device approval, cron execution, a subsequent agent turn and all three fixture cleanup actions. The old denial did not recur; its precise cause remains unresolved. This result does not certify the new routing commit. - The protected-runtime timeout remains the recorded infrastructure fast-follow outside this PR. NIM, rollback and final fixture cleanup remain unverified; no all-pass result or broad waiver is claimed. ### Historical merge repair (`94c8ec0`) - Published candidate `94c8ec0b0dd0ada2161719bb082cab0bb693f739` merges canonical main `d1c1843147ba40a7d67c1a48affbba7c899010d9`. It preserves custom-native OpenAI/Anthropic behavior, fixed NVIDIA/OpenRouter identity and endpoint checks, and main's atomic credential-handle refresh. - The initial local repair passed 221 tests. Commit hooks then identified two dependency-limit violations; sharing the existing API-aware provider-key mapping removed both added imports without increasing limits. Four affected configuration and materialization suites passed 184 tests. After integrating the overlapping main fix, four startup/refresh suites passed 217 tests. These runs overlap and are not additive. - Mapping checks: `npx --no-install vitest run --project cli src/lib/sandbox/compose-sandbox-config-body.test.ts src/lib/inference/config.test.ts src/lib/inference/native-provider/custom.test.ts --project integration test/generation/generate-native-hosted-config.test.ts`. Startup checks: `npx --no-install vitest run --project integration test/agents/openclaw/runtime/provider-placeholder-refresh.test.ts test/agents/openclaw/runtime/nemoclaw-start.test.ts test/mcp/mcp-tool-discovery-image-contract.test.ts --project cli src/lib/sandbox/compose-sandbox-config-body.test.ts`. Source-architecture checks passed without increased budgets or new cycles. - Normal commit and publication hooks passed inside a verified credential-isolation boundary. Original hook scripts, arguments, input and failure status were preserved. Git transport authentication stayed outside the hook boundary. The diff contains no secrets. - Evaluation of `94c8ec0` completed. Images passed all 14 executed jobs, GPU qualification passed, and CodeRabbit plus all nine Advisor reports were collected. Primary CI failed the strict Pi receipt-refresh gate. CodeRabbit identified the two API-consistency defects repaired above. Advisor public-documentation and Pi combination-gap findings retain their recorded dispositions. These results are historical for the new routing commit. - Infrastructure fast-follow outside this PR: align the protected-runtime job, test and cleanup budgets and bound NIM acquisition. On `8e004`, GitHub's 75-minute job limit expired after all three agents passed Ollama/vLLM; NIM, rollback and final fixture cleanup remain unverified. Reconcile run-owned cleanup before replay. This remains incomplete qualification. ### Historical cycle 6 candidate - Integrated canonical main `f2ed1705ce2270efd4a3dda0b7ea09ea9efacd63` in `8fb87cba9a7864465c9deb9423318766af7f1239`, preserving the PR's source and test changes. Nine focused suites passed 263 tests. - Final candidate `8e004b2ecee0c34d8fafc9a41868fcadf693fb24` changes only the AMD64/ARM64 Pi qualification receipts and their matching authority digests. The trusted main collector authenticated both artifacts from [successful image publication 38770328038](https://github.com/NVIDIA/NemoClaw/actions/runs/38770328038), including digest, size, source revision, architecture and common run/attempt. That publication passed all-agent activation on AMD64, ARM64 and rootless Podman. - Strict Pi source parity reports `qualified`. Focused receipt/source-parity and candidate-authority suites passed 34 tests. Normal commit and publication hooks passed; no hook bypass or force push. The receipt follow-up is the final authorized publication in cycle 6. - Integrated-candidate CodeRabbit and all nine [Advisor attempt 2 specialist reports](https://github.com/NVIDIA/NemoClaw/actions/runs/38773784513/attempts/2) are complete. The one infrastructure retry recovered the missing migration review. Advisor's aggregate still reports the previously dispositioned Hermes combination gap and public-documentation drift; these retain the [existing dispositions](#13201 (comment)). The reported missing native Anthropic selector resolves to the existing `openclaw-inference-switch` job, whose Anthropic shard selects native custom mode. - The fresh public custom OpenAI onboarding combination remains explicitly unverified in live E2E, alongside the existing Hermes and Pi combination gaps. Retained deterministic and shell-request evidence covers constituent behavior; it does not establish those full combinations. No behavior failure was observed for these gaps, and no additional scenario is claimed as passed. - Evaluation of `8e004` completed: primary CI and images passed, CodeRabbit and all nine Advisor reports were collected and dispositioned, and all 26 ordinary runtime jobs acquired passing evidence across the retained original and diagnostic attempts. Twenty-two of 23 selected E2E selectors passed. Protected runtime timed out with NIM/rollback/final fixture cleanup unverified; separate GPU device approval failed. Original HTTP and dependency failures remain recorded, with their causes unresolved. These results do not establish the new merge commit's outcome. ### Cycle 5 - Candidate `9029658852ce5600bbfcfae943be13e265a482cf` repairs native provider handle materialization and refresh across the existing inference, OpenAI and Anthropic config fields. Raw, missing, unscoped and mismatched handles remain rejected or unchanged at their owning boundary. NVIDIA endpoint validation and the existing bounded pre-mutation wait remain intact. - Corrected Hermes export fixture category handling and custom HTTPS export mock parity. Refusal still requires both aliases to exit 2, matching diagnostics, no output files, and no credential leakage. Refreshed both Pi receipts and authority digests from [successful image run 38707069678](https://github.com/NVIDIA/NemoClaw/actions/runs/38707069678). Corrected the owning E2E README baseline description. - Local checks: native config command tests 29 passed; native export tests 8 passed; startup refresh and Hermes fixture tests 71 passed. CLI type checking with a 12 GiB heap and strict Pi receipt validation passed. Both original HTTP and HTTPS smoke reproductions now pass. Tests caught and verified the repair of an intermediate NVIDIA placeholder regression. - Completed the affected source and caller review, retaining the 563-row channel, 114-row native and 984-row messaging ledgers. No live target, retry, deadline or assertion-budget increase. The diff contains no secrets; test values are synthetic. - Cycle 4 [standard E2E 38712222640](https://github.com/NVIDIA/NemoClaw/actions/runs/38712222640) failed with the native and Hermes fixture defects repaired here. Its protected-runtime job failed with authenticated runner disk exhaustion; logs, runtime results and cleanup evidence are unavailable, so that result remains inconclusive. No unchanged workflow retry or base replay was dispatched. Separate [Launchable 38712268470](https://github.com/NVIDIA/NemoClaw/actions/runs/38712268470) passed with source/image identity and workspace absence verified. - Normal commit and publication hooks passed, including CLI types, repository/growth checks and secret scanning. GitHub verified the signed commit. Required CI, CodeRabbit, all nine Advisor reviews and exact-commit image publication completed. Twenty-two of 23 recommended E2E selectors passed. The protected-runtime selector failed during NIM model download because the upstream runner exhausted disk space; Rebecca accepted that specific upstream failure as nonblocking. Current-run cleanup is verified; NIM and subsequent rollback coverage remain incomplete. Cycle 5 evaluation is complete. Rebecca subsequently authorized cycle 6 and its mandatory Pi receipt-only follow-up; PR merge remains unauthorized. ### Cycle 4 - Candidate `92e4cc4cda6cbe72f712df4e62dbb080d96ce1de` repairs six causes observed on cycle 3: custom HTTPS profile TLS readback, an E2E fixture import requiring an unbuilt CLI catalogue, native receipt retention during rebuild, native resume credential recovery, verified credential migration accounting, and the native Anthropic baseline input. - Reuse still requires matching endpoint/API/credential key/gateway authority. Missing authority and credentials fail before mutation. Credential retirement applies only to the exact newly persisted value; reuse and Bedrock adapter tokens do not claim migration. Legacy routes retain their existing behavior. - Focused local evidence: profile/probe/fixture 135 tests passed; registry 62 passed; final recovery/migration sibling 117 passed; baseline 4 passed; final hook-repair shim/migration/fixture 35 passed. CLI type checking passed with a 12 GiB heap. Working-tree mock/live parity reported no errors. Normal commit and publication hooks passed; the signed commit is GitHub Verified. - Complete cycle 3 failed-job log review and affected assertion/caller audit are retained. The new channel ledger covers 563 predicates; corrected native Anthropic ledger covers 114. No new target, retry, deadline, assertion-budget or product-scope expansion. Commit-hook findings were repaired without raising architecture/growth limits. - Cycle3 [standard E2E 38690740007](https://github.com/NVIDIA/NemoClaw/actions/runs/38690740007) failed with the grouped causes above; protected runtime was skipped. Its separate [Launchable 38690773573](https://github.com/NVIDIA/NemoClaw/actions/runs/38690773573) passed with exact source/image evidence and confirmed workspace absence. These are historical results, not cycle 4 passes. - Cycle 4 CI and automated reviews completed. CI found the TLS mock and stale Pi receipts repaired in cycle 5. All nine Advisor artifacts were reviewed; public documentation remains with the authorized post-merge owner, and additional Hermes combination coverage was declined with its gap recorded. Image publication passed. Runtime results are summarized under cycle 5 above. ### Cycle 3 - Candidate `77b4c216412ddc16f2b7c559202cadcaa9eb9202` repairs native onboarding/rebuild smoke and Hermes managed-policy credential handling. Native checks verify the recorded attachment, endpoint, API, model and scoped credential reference before inference. Legacy paths remain explicit. - Local evidence: Hermes policy reader 74 tests passed; existing smoke 94 passed; native onboard/rebuild shell probes 2 passed. Route-preservation regressions passed after integrating main #13051. E2E support request/config/planner tests passed, including the final selector/planner/growth batch of 147 tests. - Cloud and Hermes runtime probes now use the recorded route. The existing OpenClaw Anthropic selector verifies native credential injection and a real agent request. Local support tests own deterministic request/config behavior. No additional E2E target, API matrix, retry allowance, or assertion-budget increase. - `npm run e2e:assertions:check`: passed, 1,257 direct expectations across 77 live files. The exact previously approved growth delta remains unchanged. Normal commit and publication hooks passed, including ShellCheck, repository checks, growth checks and final CLI/plugin type checks. GitHub verified the signed commit. Current-head CI/reviews/E2Es remain required before readiness. - Completed failed-job log and predicate audit for cycle 2. Its standard E2E run had ten runtime failures, with production smoke/policy defects and route-oracle mismatches grouped here. AMD64 encountered an external archive HTTP 429; the earlier OpenClaw resume Error transition lacks enough retained diagnostics to attribute. No speculative code or retry was added for those conditions. - The existing main conflict was resolved with one merge of `80491e92e2cda863d6bf42869a8f8a5b0a975628`. Source-route reservation and backup safety changes from #13051 are preserved. - This candidate requires fresh CI, CodeRabbit, all nine Advisor reviews, exact-head images and every applicable recommended E2E. Earlier results do not establish cycle 3 runtime success. ### Cycle 2 - Candidate `e14836f5d134d146099079773bf52ba0c6fba188` repairs selected-provider receipt filtering, recovery verification/messages, and receipt-aware export. Existing legacy fixtures remain explicit. - Focused CLI regression batch: 558 tests passed across 7 files. After final export/profile refactoring: 310 tests passed across 3 files. Reconciliation/growth checks: 32 tests passed. Selected-agent E2E probe support: 41 tests passed. - CLI type checking passed with an 8 GiB heap. Normal commit and publication hooks passed, including the final CLI type check. Cycle 2 required CI and CodeRabbit completed; all nine Advisor artifacts were collected. The native Anthropic evidence gap is addressed in cycle 3. Its standard E2E run failed; its separate staging Launchable run passed and confirmed workspace deletion. - Genuine AMD64/ARM64 Pi qualification receipts came from [successful image run 38643585682](https://github.com/NVIDIA/NemoClaw/actions/runs/38643585682); strict source parity passed. These receipts do not replace current-head runtime results. - The added selected-agent route probe checks endpoint, API, selected model, and opaque credential reference. Rebecca approved its exact assertion-growth delta in [the maintainer record](#13201 (comment)). Independent CI must verify that record. No new E2E target or API matrix was added. - Cycle 1 CI exposed the repair batch above. Its manual E2E stopped at an image-publication prerequisite before runtime tests. All applicable cycle 2 Advisor-recommended selectors were dispatched. New-head evidence remains required. ### Cycle 1 local evidence - Focused endpoint, native lifecycle, inference get, switch, probe, onboarding, and E2E-support tests: 11 files, 456 tests passed. This includes the final dependency consolidation and custom route observation changes. - Earlier focused lifecycle, cleanup, registry, health, generation, and E2E-support batch: 14 files, 379 tests passed. The final switch rollback regression reproduced before repair and passed afterward with all 45 native switching tests. - `npm run test:projects:check`: passed. `npm run e2e:assertions:check`: passed after the final fixture consolidation, with 1,256 direct expectations across 77 live test files. The final E2E probe/support file passed all 37 tests. - Normal pre-push hooks passed publication validation and the CLI, plugin, and checked-JavaScript type checks for commit `05f5dafe655482077b42bd36c0230c88dcc6ec81`, with `NODE_OPTIONS=--max-old-space-size=8192`. - Source architecture budget: passed without changing limits. The normal commit hooks passed lint, formatting, repository checks, growth checks, secret scanning, and commitlint. - Live runtime verification and broad CI are pending. The local readiness doctor reported the default Node heap limit and insufficient local Docker allocation; runtime evidence will use the repository GitHub E2E workflow. - The diff contains no credentials, API keys, or other secrets. Test values are synthetic. ## Review notes ### Accepted cycle 5 infrastructure exception Rebecca explicitly accepted the known upstream disk-exhaustion failure in protected runtime job [116419249017](https://github.com/NVIDIA/NemoClaw/actions/runs/38729390261/job/116419249017) as nonblocking for `9029658852ce5600bbfcfae943be13e265a482cf`. The failure occurred before NIM agent qualification and subsequent rollback checks; those checks remain incomplete. The current-run owned-container, fixture, registry and Docker-auth cleanup evidence passed. Historical cycle 4 cleanup remains unverified. All active required checks pass; Advisor findings retain their [recorded dispositions](#13201 (comment)). Sensitive paths include the changed files under `agents/**`, `src/lib/inference/**`, `src/lib/onboard/**`, `src/lib/onboard.ts`, and `tools/e2e/**`, classified against the canonical sensitive-path policy. Self-review of NVIDIA/NemoClaw commit `05f5dafe655482077b42bd36c0230c88dcc6ec81` covered the candidate diff, ownership and endpoint boundaries, rollback, failure states, and focused negative tests. No independent pre-publication review or waiver is claimed. These paths await independent automated review and runtime evidence. Cycle 5 self-review covered NVIDIA/NemoClaw `9029658852ce5600bbfcfae943be13e265a482cf`, including sensitive `scripts/nemoclaw-start.sh` and `src/lib/sandbox/**` changes: credential custody, scoped handle equality, endpoint guards, atomic config mutation, startup handoff, affected callers and negative tests. CodeRabbit and all nine Advisor specialist reviews are complete, with source-backed dispositions recorded. Current-commit E2E evidence and the accepted upstream storage-capacity exception are summarized above. Cycle 4 self-review covered NVIDIA/NemoClaw `92e4cc4cda6cbe72f712df4e62dbb080d96ce1de`: the six repair groups, exact binding and credential-retirement guards, all affected callers, complete failed logs, and unchanged live assertions/cleanup. Current-head independent review and runtime evidence remain pending. Cycle 3 self-review covered NVIDIA/NemoClaw `77b4c216412ddc16f2b7c559202cadcaa9eb9202`: native attachment and registry authority, selected config/API/credential checks, Hermes managed policy, the trusted selector boundary, negative local tests, and retained cleanup/retry contracts. Independent reviews and live results for this commit remain pending. Rebecca subsequently authorized the local merge repair, its publication and GPU qualification on the next commit. That monitoring cycle completed and paused. Rebecca then authorized the conflict resolution and resume repair above, followed by the current selector repair and up to three further justified revision cycles after complete candidate evaluation. The protected timeout fix is deferred outside this PR as an infrastructure fast-follow; no all-pass result or broad waiver is claimed. PR merge remains unauthorized. Current repair self-review covers the provider-key mapping, fixed-provider endpoint guards, custom-native scoped-handle refresh, atomic persistence, and affected negative tests; independent review for `94c8ec0b0dd0ada2161719bb082cab0bb693f739` completed with the routing findings repaired in `19f521416a5dbb62ac9fa169e5f03eb817c72297`. Current repair self-review checked API defaults, explicit Responses, namespace/endpoint/credential agreement and shared-route controls. Independent review of `19f5214` completed with both routing findings resolved and retained Advisor dispositions. Receipt follow-up self-review authenticates source, image digests, paired run/attempt, exact contract bytes, authority hashes and strict source parity. Independent review of the receipt-only commit completed, including the single resume defect repaired above. Resume repair self-review covers NVIDIA/NemoClaw `53bc89879a90e533a191f8b8df2022dcad092ab6`, specifically `src/lib/onboard/machine/resume-provider-shim.ts`, `src/lib/inference/native-provider/custom-setup.ts` and their existing test owners. It checked credential-name precedence, receipt validation, provider/endpoint binding, API and live resource identity checks, no-rotation reuse, alternate legacy/reservation paths and mismatch failures. Independent automated review of `53bc898` completed with the source-selection repair and retained policy/gap dispositions above. No merge approval or new waiver is claimed. Current merge self-review covers NVIDIA/NemoClaw `afea1a7c05f3383bd20d5f79a477c0c8c0179423`, the affected native-local/custom-hosted route owners, credential and endpoint authority, actual probe request/response boundaries, export/profile validation, sibling configuration paths, and deterministic source selection. Independent current-candidate review and live results remain pending. --- Signed-off-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com> Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Configure custom OpenAI-compatible and Anthropic-compatible endpoints for native inference, with support for multiple APIs and provider-specific credentials. * Custom native routes are supported during onboarding, inference switching, sandbox rebuilds, and model changes. * Bedrock-compatible Anthropic endpoints can use the native adapter route. * **Bug Fixes** * Inference status now reflects the endpoint attached to the sandbox. * Failed route checks provide guidance when a legacy inference route is unreachable. * Failed provider changes restore the previous attachment when possible, and custom attachments are retained only when their route settings still match. * Custom endpoint credentials and routing are checked against the selected provider configuration. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com> Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com> Co-authored-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com> Co-authored-by: Prekshi Vyas <prekshiv@nvidia.com>
| await verifyNativeCustomStatusAttachment({ | ||
| gatewayName: options.gatewayName, | ||
| sandboxName: options.sandboxName, | ||
| expected: options.attachment, | ||
| environment: options.environment, | ||
| }); |
| await verifyNativeCustomStatusAttachment({ | ||
| gatewayName: context.gatewayName, | ||
| sandboxName: context.sandboxName, | ||
| expected: nativeCustomProviderAttachment, | ||
| }); |
| NativeCustomProviderError, | ||
| profilePath, |
| logicalProvider: NVIDIA_HOSTED_LOGICAL_PROVIDER, | ||
| label: "NVIDIA", | ||
| }, | ||
| NativeNvidiaProviderError, |
Outcome
Fresh hosted OpenAI-compatible and Anthropic-compatible endpoints use native providers attached to the selected sandbox. Their validated endpoint, provider identity and credential authority survive supported lifecycle operations. Host-local endpoints and existing shared-route sandboxes retain their supported behavior.
Reason
Slice 3 requires endpoint-specific ownership through setup, authenticated requests, observation, restart, rebuild and cleanup. Missing or conflicting authority must fail without silently selecting another route or discarding recovery records.
Related issues
Fixes #12636. Part of #12558. Consumes merged #12895 for native NVIDIA credential handling and #13158 for compatible curl image packages; #13129 restores the durable E2E toolchain source.
Changes
Validate endpoint admission, API family, resolved addresses and immutable provider identity before granting access. Persist ownership before attachment and observe ambiguous mutations before proceeding.
Carry native attachments through onboarding, inference selection, status, health, export, restart, rebuild and recovery. Reconcile only the selected sandbox's endpoint policy and preserve unrelated entries. Keep upstream credentials outside agents through issued credential references, scoped HTTPS-pin or Bedrock adapter authority, and bounded projection observation.
Verify replacement registration before retiring superseded provider and HTTPS authority. Retain recovery records when cleanup cannot be confirmed; reject malformed or conflicting ownership before destructive mutation.
Use the matching provider/model session endpoint when an implicit inference switch selects its route owner. Forward the lifecycle environment to rebuilt-attachment verification. These changes are protected by OpenClaw/Hermes routing and rebuilt-smoke regressions.
Return to interactive endpoint selection only for credential-free URL/DNS admission failures before adapter/provider mutation; noninteractive setup exits unsuccessfully. Authority, ownership and cleanup failures remain hard failures. The existing admission table tests both modes and confirms rejection creates no provider or adapter.
Preserve all native live cases while separating Deep Agents Code and Docker-only Pi terminal cases from the core custom-hosted shard. The same
inference-routingselector includes retained, core and terminal shards for Docker/Podman. Core limits remain 330 minutes per test and 350 per job; terminal limits are 122/142 minutes. Timeout, selector and coverage tests protect the complete group, and the terminal helper retains all live assertions while forwarding Pi’s required runtime arguments.Align setup, switching and quickstart guidance with native attachments versus retained shared routes, qualifying HTTPS adapter behavior as DNS-backed HTTPS. Preserve capability audit evidence states and existing agent/API boundaries.
Finish later native-home removals owned by the current identity after a permission denial, then retain failure for the existing privileged/pinned-user cleanup pass. Root, protected-path and durable-identity safeguards remain enforced.
Preserve maintenance stop/start/readiness/abort diagnostics and add bounded failure observations for native selector timing, Hermes export policy and DCode consent/round files. These diagnostics do not establish that the underlying runtime failures are repaired.
Resolve an actual main conflict by retaining canonical Pi qualification receipts and their matching authority together.
Verification
Current published candidate:
a6c26a7e5b4d6a7db3823d7bc358b57fb91b1e76. PR API base:876e20ef733444b3a9c141242c84e1f16b6a7722.The startup repair regenerates the shipped runtime, pins its exact digest, and executes it against four valid native profiles and sixteen rejection cases while retaining NVIDIA/OpenRouter exports. Pi receipt authority exactly matches its checked-in qualification receipts.
The required conflict integration preserves both the PR's native custom authority and incoming main's native local Deep Agents Code support, onboarding auto-approval interface and lifecycle behavior. The runtime bundle was regenerated from the combined source. Onboarding verification carries recorded custom/local/fixed attachments through its existing context owner. Dependency and growth limits were satisfied without raising thresholds. GitHub now reports the PR mergeable; review approval remains outstanding.
Local validation includes 530/530 final dependency-owner tests and 42/42 affected deployment-verification tests, plus the retained integration tests recorded in the delivery checkpoint. Normal isolated commit and pre-push hooks, publication builds and CLI typecheck passed. Documentation checks and the pinned Fern check passed (two warnings retained). Remote branch and PR match the signed commit; all 141 PR commits are GitHub Verified. The worktree is clean.
All 38 failed runtime logs from
3addd6aa83were read completely and all 38 failure archives authenticated. Thirty-six first failures share the repaired startup rejection; two are inherited unsupported Podman custom-Dockerfile fixtures. Downstream assertions and prior failures remain separately dispositioned. This does not establish that all 38 runtime jobs now pass.0857dac8a4run had nine clear bounded reports, not full-diff hosted coverage. Prior challenged findings are not claimed withdrawn.Prior selected runtime results on
3addd6aa83: staging 1/1 passed; native 4/6 passed, 2 failed; regressions 22/58 passed, 36 failed. Setup and aggregate jobs are excluded. All runs are terminal. Retain missing original Hermes export-time policy, native Podman HTTP 530 origin, hosted authorization cleanup permissions and survival final-absence evidence; no speculative repairs or weakening of security policy. The two unsupported Podman fixtures remain excluded from source repairs, not waived.The complete prior diff and current repair delta have specialist-based self-review, not independent approval or a full-diff hosted Advisor pass. Preserve all source/assertion ledgers, failure history, retry counts and the separately pinned terminal VM.
Review notes
Sensitive paths include inference, onboarding, credential custody, ownership persistence, sandbox lifecycle and E2E enforcement. Current-head independent review, original six-task acceptance and all selected runtime qualification remain outstanding. No merge authorization.
The latest batch reconciles the census metadata to the actual unchanged live assertions; it does not raise an enforcement limit or edit the approved exception. The exact assertion transition approval remains resolved: maintainer record, digest
92e6d8923acbda85e9d817f68145ccd260a5f7be0e33ec87af201079d1323c2f. Per-case deadlines remain unchanged. The retained native routing job includes incoming main cases and uses their combined calculated 348-minute budget. Credential scope, isolation rules and product scope are preserved.Signed-off-by: Deepak Jain deepujain@gmail.com
Signed-off-by: Prekshi Vyas prekshiv@nvidia.com
Summary by CodeRabbit