Skip to content

test: apply, refresh, and tear down the Kubernetes resources in the nemoclaw contract tests - #12972

Merged
cv merged 1 commit into
v1from
test/kubernetes-contract
Oct 11, 2026
Merged

cv merged 1 commit into
v1from
test/kubernetes-contract

Conversation

@cv

@cv cv commented Oct 10, 2026

Copy link
Copy Markdown
Collaborator

Part of #12876. Closes #12879.

Failure

The nemoclaw provider's contract binary checked nemoclaw_kubernetes_storage, nemoclaw_kubernetes_auth, and nemoclaw_kubernetes_gateway only through plan (kubernetes_hcl). Apply, refresh, and teardown ran only in the live Kind suite.

Decision

  • The provider reaches the cluster only through kube-rs: get, create, and delete by UID precondition, a StorageClass list, and a metadata list of Helm release Secrets. It never runs Helm; the bundle's Helm provider installs the chart. The SDK tests already had an in-memory Kubernetes API that covers these calls (helm_recovery runs the bundled Helm provider against it too). This PR moves that fake to crates/test-support/kube_api.rs so the provider contract binary can include it. It is shared as a source module, like http.rs, and not through nemoclaw-test-fixtures, because that crate depends on the SDK. The kube-rs client helper stays in the SDK tests (tests/support/kube_client.rs), so the provider takes no kube dependency.
  • New kubernetes_lifecycle contract tests run the production provider through pinned OpenTofu. The provider connects through a kubeconfig that points at the fake and uses NEMOCLAW_KUBERNETES_STATE. The test writes and deletes the StatefulSet and release record that Helm would, in the bundle's order. The live Kind tests remain the end-to-end check of the chart.
    • kubernetes_resources_apply_observe_the_release_and_tear_down_keeping_storage:
      • Planning changes nothing.
      • Apply creates the namespace, key, and six issuer objects with the owner label and records their UIDs.
      • A gateway that is not ready plans another observation, then reports running once ready, and release_present follows the release record.
      • Teardown (destroy = true) keeps the issuer while the release exists, then deletes it once Helm's objects are gone. The namespace and key are kept.
      • tofu destroy refuses storage.
      • A later apply prepares a new issuer in the retained storage.
    • kubernetes_resources_refuse_replaced_or_missing_objects_without_recreating_them: a deleted namespace, a replaced key, a deleted or replaced issuer object, or a replaced StatefulSet fails plan and apply with the ownership error. Nothing is recreated or removed, and state is unchanged.
  • Fake fix: a GET for an absent object whose name ends in s (such as namespace agents) returned 200 with an empty list instead of 404. The fake now tells collection paths from object paths by their shape. the_in_memory_api_reports_absent_objects_as_missing covers the fix.
  • docs/contributing/integration-tests.md describes the new tests. It also drops the claim that ELSEWHERE lists types, because that list is now empty.

Validation

Run locally on linux_arm64:

  • the_in_memory_api_reports_absent_objects_as_missing failed before the fake fix and passes after it.
  • Both kubernetes_lifecycle tests pass in about 6 s each with pinned OpenTofu 1.12.6. kubernetes_hcl and every_type_has_a_contract_test also pass.
  • Mutation check: with storage verification disabled in Operations::read_storage, the drift test fails at the deleted namespace. I reverted the mutation.
  • All 85 SDK kubernetes_ integration tests pass, plus the helm_recovery lib tests, including the five ignored ones against a bundle.
  • cargo fmt --all and cargo clippy --workspace --all-targets -- -D warnings pass.

I did not run Windows locally. The tests reach the fake over loopback TCP, not through the ssh relay, and are not gated.

…emoclaw contract tests

The provider's contract binary checked nemoclaw_kubernetes_storage,
nemoclaw_kubernetes_auth, and nemoclaw_kubernetes_gateway only through plan.
kubernetes_lifecycle now applies, refreshes, and tears them down through
pinned OpenTofu against the in-memory Kubernetes API the SDK tests use. The
test writes and deletes the StatefulSet and release record Helm would.

The fake moves to crates/test-support/kube_api.rs so both crates include it,
and its Kubernetes client helper stays with the SDK tests. The fake now
answers an absent object whose name ends in "s", such as the namespace
"agents", as 404 instead of an empty list.
@cv cv added area: e2e End-to-end tests, nightly failures, or validation infrastructure v1 NemoClaw v1 branch labels Oct 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

@cv
cv merged commit a49fc77 into v1 Oct 11, 2026
77 of 78 checks passed
@cv
cv deleted the test/kubernetes-contract branch October 11, 2026 00:24
cv added a commit that referenced this pull request Oct 11, 2026
## Failure

`v1` doesn't compile since #12972. Its new
`nemoclaw-provider/tests/contract/kubernetes_lifecycle.rs` calls
`TofuWorkspace::new(tofu, provider)`, with OpenTofu and the provider
read from `NEMOCLAW_TEST_TOFU` and `NEMOCLAW_TEST_PROVIDER`. #12971
merged first and changed that function to `TofuWorkspace::new(&Bundle,
nemoclaw)`. Since #12971, `cargo ci lifecycle` also no longer sets
either variable. Each PR passed CI against a `v1` without the other.

Every PR's CI now fails at `Lint Rust`:

```
error[E0308]: mismatched types
   --> crates/nemoclaw-provider/tests/contract/kubernetes_lifecycle.rs:111:44
    |
111 |         let workspace = TofuWorkspace::new(tofu, provider);
    |                                            ^^^^ expected `&Bundle`, found `PathBuf`
```

## Decision

Build the workspace with `crate::workspace()`, as the crate's other
contract tests do since #12971: OpenTofu and the `openshell` and
`fabric` providers from `NEMOCLAW_TEST_BUNDLE`, and the `nemoclaw`
provider built from the checkout. That drops the two environment
variables and the now-unused `PathBuf` import.

## Validation

- `cargo clippy -p nemoclaw-provider --tests -- -D warnings` is clean.
- `cargo ci` passed: 1,331 workspace tests and 131 lifecycle tests. Both
of the file's tests ran from the bundle and passed:
-
`kubernetes_resources_apply_observe_the_release_and_tear_down_keeping_storage`
-
`kubernetes_resources_refuse_replaced_or_missing_objects_without_recreating_them`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: e2e End-to-end tests, nightly failures, or validation infrastructure v1 NemoClaw v1 branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant