[drew]

Summary

Add explicit sandbox suspend and resume across the gateway, CLI, SDKs, and compute-driver contract. Backends can retain a workload while suspended; resume attaches a new supervisor generation with fresh credentials.

The V8 implementation and manual counter demo are extracted into dependent PR #4858, tracked by #4857.

Related Issue

Closes #4763

Related: #4352

The author has verified repository ADMIN access; the linked issue does not yet have an acceptance label.

Changes

• Add SuspendSandbox and ResumeSandbox RPCs, Suspending/Suspended phases, and persisted resume intent for lifecycle recovery.
• Gate suspend/resume on compute-driver capabilities; drivers without support return Unimplemented.
• Add CLI and SDK lifecycle operations and update phase handling in the TUI and telemetry.
• Keep restore coordination in the compute driver and isolation backend, outside shared supervisor bootstrap metadata. CLI resume is strict; explicit start discards retained state. API clients can opt into restart_if_restore_fails.
• Preserve terminal process results when exit races suspension, keep failed resume recovery retryable, and leave forwards running when suspension is rejected.
• Keep the existing V8 example building with unsupported-operation compatibility stubs. The dependent PR supplies its concrete suspend/resume support.
• Update crate implementation READMEs and the CLI skill. Published documentation additions remain deferred.

Testing

• Repository commit checks passed, including workspace/example Clippy, formatting, protobuf lint, Python/TypeScript lint, license checks, and Helm checks.
• Fifteen gateway suspend/resume tests passed, including exit races and failed resume recovery on retry and gateway startup.
• Thirteen process-exit checks and the CLI rejected-suspend/active-forward regression test passed during review fixes.
• Existing V8 example builds on this parent branch with cargo check --manifest-path examples/isolation-backend-v8/Cargo.toml --locked --all-targets.
• Focused supervisor, sandbox-backend, CLI, and SDK checks passed during implementation.

The native V8 end-to-end evidence and retained-state tests belong to the dependent example PR. This parent defines and tests the shared contracts; it does not enable suspension in the existing example or bundled production drivers.

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Relevant crate implementation documentation updated

[github-actions]

Compatibility review (advisory)

New or updated breaking changes; see the full report for all findings.

1. rust-sdk — No explicit risk acceptance found

The SDK-exposed generated OpenShell server trait now requires suspend_sandbox and resume_sandbox, while the ComputeDriver trait now requires suspend_sandbox.

• Impact: Existing downstream mock/server and external compute-driver implementations fail to compile without new handlers, although existing client callers remain compatible.
• Action: Generate backward-compatible default UNIMPLEMENTED handlers for the new trait methods in v0.1.6 instead of requiring downstream implementations to change.
• PR: feat(sandbox): add suspend and resume lifecycle #4765

2. rust-sdk — No explicit risk acceptance found

raw::proto::compute::v1::StartSandboxRequest gains the public resume and restart_if_restore_fails fields beyond its baseline six-field shape.

• Impact: Downstream code using complete Rust struct literals or exhaustive destructuring for the existing compute-driver request no longer compiles.
• Action: Preserve the baseline request shape for v0.1.6 using a separate resume request, or defer the incompatible shape change to a minor train with literal and destructuring migration guidance.
• PR: feat(sandbox): add suspend and resume lifecycle #4765

3. rust-sdk — No explicit risk acceptance found

The exhaustive raw ProtoSandboxPhase enum gains Suspending and Suspended variants beyond the baseline lifecycle variants.

• Impact: Existing exhaustive matches over raw::ProtoSandboxPhase or raw::proto::SandboxPhase fail to compile, unlike matches over the already non_exhaustive high-level SandboxPhase.
• Action: Keep the existing exhaustive raw enum unchanged on the patch train, or defer its expansion to a minor train with guidance for adding suspension cases or wildcard matches.
• PR: feat(sandbox): add suspend and resume lifecycle #4765

4. go-sdk — No explicit risk acceptance found

ClientInterface now requires SandboxSuspendResume(), and generated OpenShellClient now requires SuspendSandbox and ResumeSandbox.

• Impact: Existing downstream fakes and adapters implementing either baseline interface fail interface-satisfaction checks, although ordinary SDK callers remain compatible.
• Action: Preserve the baseline interface method sets in v0.1.6 and expose suspend/resume through opt-in extension interfaces or separately versioned clients.
• PR: feat(sandbox): add suspend and resume lifecycle #4765

5. typescript-sdk — No explicit risk acceptance found

SandboxPhaseName expands from its baseline closed union to include 'suspending' and 'suspended'.

• Impact: Existing exhaustive switches and Record<SandboxPhaseName, …> maps fail TypeScript compilation because they do not handle the new returned phase values.
• Action: Preserve the existing phase union on v0.1.6 with an opt-in suspension-aware representation, or defer widening it to a minor train with exhaustive-switch and Record migration examples.
• PR: feat(sandbox): add suspend and resume lifecycle #4765

Review details

• Execution: complete
• Candidate: PR #4765 (5e6600f196ca31ecad22f63ecfd15a3e957620c4)
• Baseline: 9a85667d0d87e8131b64351ec1fc4eb26e7bb865 (9a85667d0d87e8131b64351ec1fc4eb26e7bb865)
• Reviewed: rust-sdk, python-sdk, go-sdk, typescript-sdk, api-behavior, cli, configuration, policy, helm, state

Advisory review; publication gates are unchanged.

Branch check run

[mayawang]

Thanks Drew, this reads well, and it maps cleanly onto how Agent Substrate suspends and resumes: SuspendBoundary to suspending the actor, the actor's snapshot as the restore reference, and ResumeBoundary to resuming it, possibly on another node.

A few questions inline. I'd like to settle the first one before merge, since it's about what the contract guarantees and is easier to get right now than to change later; it may only need a wording tweak. The other two are fine as follow-ups.

[mayawang]

On the ResumeBoundary comment just below ("keeping the restored workload frozen until StartAgent"):

Agent Substrate brings an actor back running when it resumes a memory snapshot. There isn't a way today to restore it and hold it paused until StartAgent.

One way we could meet this is to defer the actual resume until StartAgent arrives, but then a failed restore would only show up at StartAgent, not when ResumeBoundary returns. Would that be acceptable? Or is the real requirement that no untrusted instruction reaches the workload before StartAgent, rather than that the whole workload is frozen?

[mayawang]

For a RESTORED boundary, what should StartAgent do: start a new agent process, or hand back the restored one? And provider_env / provider_files: the restored processes keep the environment they had at suspend, so if a credential rotated while the sandbox was suspended, is UpdateProviderEnvironment the expected way to deliver it after resume?

[mayawang]

This covers suspends the gateway asks for. On backends that suspend idle sandboxes themselves, which is the case in #4352, the driver would need a way to report a sandbox as Suspended through WatchSandboxes, and a connection to a suspended sandbox might trigger a resume. Is that planned as a follow-up on this contract? Not needed for this PR.