Skip to content

fix(gateway): reject an overlong VM driver socket path with a clear error - #4184

Open
shiju-nv wants to merge 2 commits into
NVIDIA:mainfrom
shiju-nv:fix/vm-driver-socket-path-diagnostic-4178
Open

shiju-nv wants to merge 2 commits into
NVIDIA:mainfrom
shiju-nv:fix/vm-driver-socket-path-diagnostic-4178

Conversation

@shiju-nv

@shiju-nv shiju-nv commented Oct 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

The gateway gives the VM driver a control socket at <state_dir>/run/compute-driver.sock. When that path is longer than the platform's Unix socket limit, the driver exits with path must be shorter than SUN_LEN and the gateway can report only the driver's exit status. The gateway now checks the path before it creates directories or starts the driver, and the error names the path, its length, the limit and the setting that moves it.

Related Issue

Closes #4178

Changes

  • crates/openshell-gateway/src/vm.rs: add MAX_UNIX_SOCKET_PATH_LEN (107 bytes on Linux and Android, 103 elsewhere, excluding the terminating NUL, matching sockaddr_un.sun_path of 108 and 104 bytes) and check_compute_driver_socket_path_len. prepare_compute_driver_socket_path calls it first, so a rejected path leaves no directory behind. vm::spawn is the only launcher and the only caller of prepare_compute_driver_socket_path.
  • The error reads: vm compute driver socket path '<path>' is <n> bytes, over the <limit>-byte limit for Unix socket paths on this platform; set [openshell.drivers.vm] state_dir to a shorter directory (it defaults to openshell/vm-driver under $XDG_STATE_HOME, or under ~/.local/state when that is unset).
  • docs/how-it-works/gateways/configuration.mdx: one comment on state_dir in the VM example saying the socket path is <state_dir>/run/compute-driver.sock and that the gateway refuses to start past the limit.
  • Two unit tests.

Testing

  • Checks appropriate to the affected code and behavior pass
  • Unit tests added/updated (if applicable)
  • E2E tests added/updated (if applicable): none added

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

…rror

The VM driver listens on <state_dir>/run/compute-driver.sock. When that path exceeds the platform's Unix socket limit the driver exits with a bare 'path must be shorter than SUN_LEN' and the gateway can only report the driver's exit status.

Check the path in prepare_compute_driver_socket_path before any directory is created, and name the path, its length, the limit and the state_dir setting in the error. The limit is 107 bytes on Linux and 103 elsewhere. Note the constraint in the VM driver configuration docs.

Closes NVIDIA#4178

Signed-off-by: Shiju <shiju@nvidia.com>
Comment thread crates/openshell-gateway/src/vm.rs Outdated
@drew

drew commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator

Could we extend the approach from #3544 to the remaining gateway-to-driver compute-driver.sock? That PR moved the per-sandbox control.sock and ssh.sock into private, randomly named directories under /tmp so socket paths no longer depend on the depth of the persistent state directory.

The diagnostic here is useful, but users with long state paths still cannot start the gateway. Please allocate a short, gateway-owned socket directory under /tmp, pass the resulting path through the existing --bind-socket argument, and tie directory cleanup to the managed driver lifecycle. Preserve the existing socket permissions and peer checks, and follow #3544's private-directory and collision-handling approach. Keep persistent VM state at the configured state_dir.

Please also cover startup with an overlong state_dir to verify that the control socket no longer inherits that limit. Use /tmp deliberately, as in #3544, since macOS's $TMPDIR can itself be too long.

Allocate a private random directory under /tmp and retain it with the managed driver process. Keep persistent VM state at the configured path and cover long-path startup, collision handling, and cleanup.

Signed-off-by: Shiju <shiju@nvidia.com>
@shiju-nv

shiju-nv commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

The PR initially focused on giving users a clearer error message. The gateway now creates a private, randomly named directory under /tmp for compute-driver.sock.

@elezar elezar left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a question: Are there other paths where we could expect this to happen? Should we expose a shared check to sanitize these paths?

@benoitf

benoitf commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

it's all the *.sock files

@shiju-nv

shiju-nv commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

Audited source:

Socket family Socket pathname and creator Source Current behavior and proposed treatment
Gateway-managed VM compute-driver.sock /tmp/os-gw-<euid>-<32-hex-random>/compute-driver.sock; gateway chooses the path, VM driver binds it. crates/openshell-gateway/src/vm.rs:344, :490; crates/openshell-server/src/compute/mod.rs:362 Already short and private; consolidate allocation and validate the final endpoint. Keep the directory alive through managed subprocess ownership.
VM sandbox control.sock and ssh.sock /tmp/os-<euid>-<32-hex-random>/<sandbox-id>/control.sock and .../<sandbox-id>/ssh.sock; VM runtime creates control, supervisor creates SSH. crates/openshell-driver-vm/src/driver.rs:5977, :6016, :951, :1539 Short random root, but full sandbox ID is still a variable-length leaf. Compact the leaf and share address validation. Preserve directory-fd operations.
Standalone VM --bind-socket Exact pathname supplied through --bind-socket <path>; VM driver creates the socket there. crates/openshell-driver-vm/src/main.rs:335, :423 Validate before driver initialization and socket preparation. Preserve supplied path and expected-peer-PID authentication.
Docker, Podman and Kubernetes external-driver listeners Exact --bind-socket <path> pathname; the selected driver creates it. No fixed basename or directory. crates/openshell-core/src/external_driver_socket.rs:16; driver main.rs:70, :253, :308 respectively Shared binder prepares/removes/binds without an address preflight. Validate at entry, before any side effect.
Remote compute-driver connector Configured compute driver socket_path (including CLI overrides); remote driver creates it, gateway only connects. crates/openshell-server/src/compute/driver_config.rs:191; compute/mod.rs:5807 Nonempty-path check followed by readiness retries. Reject invalid addresses before the readiness loop.
Credential driver launch/connect and standalone Vault/Kubernetes Secrets bind Configured credential driver socket_path, passed as --bind-socket <path> for managed launches; credential driver creates it. crates/openshell-server/src/credentials.rs:1016, :1565, :1833; driver main.rs:73, :50 respectively Absolute-path check is insufficient. Validate at configuration ingestion, launch/connect entry and standalone bind entry. Launched credentials still use explicitly configured paths.
Extension/interceptor/middleware unix:// transport The pathname after unix:// in the configured endpoint; external extension service creates it. crates/openshell-extension-core/src/transport.rs:152, :178 Checks nonempty/absolute paths, then connects. Use shared validation after parsing the scheme.
Docker/Podman engine discovery and connect Docker: DOCKER_HOST Unix path, /var/run/docker.sock, $HOME/.docker/run/docker.sock, $XDG_RUNTIME_DIR/docker.sock. Podman: OPENSHELL_PODMAN_SOCKET, $XDG_RUNTIME_DIR/podman/podman.sock, /run/user/<euid>/podman/podman.sock, $HOME/.local/share/containers/podman/machine/podman.sock, or CLI-discovered path. Engine creates these sockets; explicit driver socket_path can select another. crates/openshell-driver-docker/src/lib.rs:820; crates/openshell-driver-podman/src/socket_discovery.rs:23; client.rs:323; crates/openshell-core/src/local_api_socket.rs:50 HOME, runtime directory and explicit overrides can be long. Reject explicit invalid paths; skip invalid discovery candidates while retaining a useful diagnostic. The container engine owns these sockets.
Supervisor SSH listen/relay Configured SSH pathname, commonly /run/openshell/ssh.sock; VM host path is the sandbox SSH path above. Supervisor binds it. Linux @name selects an abstract address and creates no filesystem socket. crates/openshell-supervisor-process/src/ssh.rs:100; supervisor_session.rs:858; unix_socket.rs:14 Validate filesystem paths before preparation. Linux @name addresses require abstract-name validation, with existing runtime conversion preserved.
Supervisor health/readiness --health-socket-path <path>; drivers supply /run/openshell/health.sock (Docker/Kubernetes) or /run/openshell/supervisor-health.sock (Podman). Supervisor creates it. crates/openshell-supervisor/src/lib.rs:112, :247, :297 Defaults are short; configured paths need preflight before prepare/bind/connect.
Sandbox boundary listener and backend connector Listener transport pathname; common boundary mount target /.openshell/channel/sandbox/control.sock, with sidecar control path /run/openshell-sidecar/control.sock. VM host control path is above. The runtime listener creates the socket; backend connects through its descriptor/mount mapping. crates/openshell-sandbox/src/boundary_server.rs:3283; crates/openshell-sandbox-backend/src/runtime.rs:1933 Validate supplied paths before stale removal or retries. Keep mount mappings and descriptor endpoints consistent.
SPIFFE workload socket and Python gateway unix: endpoint Configured SPIFFE Workload API pathname, projected into containers as /spiffe-workload-api/<socket-basename>; SPIRE creates the host socket. Python uses the pathname in its configured gateway unix: endpoint; the gateway service creates it. crates/openshell-core/src/spiffe.rs:52; python/openshell/sandbox.py:232, :678, :691 External services own these addresses. Apply Rust validation where the host actually dials a socket. Python consistency is separate work; preserve its endpoint identity.

@shiju-nv

shiju-nv commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

Would the team prefer expanding this PR to cover the other socket paths? I'm thinking of adding shared address validation before filesystem changes or connection retries, and consolidating private short-directory allocation. The validator could live in extension-core and be re-exported from core.

VM sandbox sockets would also need compact directory names so long sandbox IDs don’t exceed the limit. Alternatively, I can keep this PR focused and address the broader changes in a follow-up.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(vm-driver): driver startup fails with a bare "path must be shorter than SUN_LEN" and does not name the path

4 participants