docs(gpu): document CDI-derived GPU sandbox policy

Describe how Docker GPU sandboxes derive filesystem access from the
selected CDI devices, which CDI mounts are rejected, the NVIDIA MPS pipe
directory exception, missing additionalGids handling on Docker releases
before 29.5, and SELinux requirements for the projected spec
directories. Add cluster debugging entries for CDI projection,
admission, probe and restart failures, and document the CDI context in
the core, sandbox and Docker driver READMEs.

Signed-off-by: Evan Lezar <elezar@nvidia.com>

Author: elezar

crates/openshell-core/README.md

@@ -64,3 +64,10 @@ Use `openshell_core::proto_struct` when crossing between `serde_json` values and
 `prost_types::{Struct, Value}`. Both conversion directions live in this module;
 JSON-to-protobuf conversion is fallible so callers cannot silently replace an
 unrepresentable number.
+
+## CDI context
+
+The shared `cdi` module defines the versioned context passed from compute drivers
+to the workload, including selected devices and projected specification directories.
+The CDI resolver and its dependency live in `openshell-sandbox`; see that crate's
+README for resource validation and enforcement.

crates/openshell-driver-docker/README.md

@@ -93,7 +93,8 @@ traverse every parent and write and enter the workdir; OpenShell does not
 change its ownership or mode.
 
 Image `VOLUME` declarations and user mounts must not cover the workdir, one of
-its parents, or the reserved `/.openshell` runtime/channel tree. OpenShell asks
+its parents, or the reserved `/.openshell/channel` and `/.openshell/cdi-specs`
+paths. OpenShell asks
 the kernel to validate access under the final identity, so POSIX ACL and host
 LSM decisions remain authoritative.
 
@@ -111,6 +112,7 @@ LSM decisions remain authoritative.
 | Private named volumes | One carries the authenticated sandbox/supervisor channel. The other is mounted only into the supervisor and contains its JWT and private gateway credentials. |
 | In-memory `/run/openshell-supervisor-ca` tmpfs | Holds only the public supervisor CA certificate and trust bundle without making all of `/run` writable. |
 | CDI GPU request | Assigns the exact validated CDI devices requested by driver config or count-based selection. |
+| CDI workload projection | For GPU sandboxes only, embeds selected device metadata in the protected boundary config and mounts Docker daemon CDI spec directories read-only into the workload. |
 
 ## Stop, Start, and Delete
 
@@ -124,6 +126,39 @@ Delete force-removes both containers, the driver-owned runtime volumes, and the
 host-private runtime descriptor. Missing or altered descriptor and channel resources
 fail closed; the driver does not run an older combined-supervisor layout.
 
+## CDI GPU Metadata
+
+Docker remains the source of truth for GPU injection. The driver selects opaque
+CDI device IDs from `driver_config.cdi_devices` or the daemon's discovered CDI
+inventory, then passes the same IDs to Docker with a CDI `DeviceRequest`.
+
+For a GPU sandbox, the driver writes those IDs and projected CDI spec directory
+paths to a versioned context in the protected workload boundary configuration.
+It uses structured, read-only, non-recursive binds at `/.openshell/cdi-specs/<index>`.
+The driver checks source existence on the daemon host using one temporary,
+stopped container per directory and the prepared workload image. Each probe has
+one read-only, private bind with source creation disabled. Successful probes are
+removed with anonymous image-volume cleanup, including after provisioning
+cancellation. Confirmed missing directories are omitted; other errors fail
+provisioning. Results are cached for 30 seconds and refreshed on demand; observed
+changes to daemon identity, configured directories, or GPU inventory invalidate
+the cache. The resulting context records only available projections. The actual
+sandbox is created once, and selected devices must still resolve from the
+projected specs. No probe starts a process, creates missing directories, or
+relabels host spec directories. Administrators must make the existing directories readable
+under the host's SELinux policy when SELinux is enabled.
+
+Whole spec directories are available to the trusted runtime, which resolves only
+the selected devices. The mandatory Landlock baseline hides these projections
+from agent processes, even when the authored policy grants a broad ancestor.
+The sandbox runtime resolves the selected devices in that mount namespace before
+it launches agent processes, derives Landlock paths from CDI `containerEdits`,
+and warns when the runtime did not apply the spec's `additionalGids`. Containers with CDI context fail startup if their
+selected devices cannot be resolved. Existing GPU containers without context
+retain legacy baseline handling on restart. Admission checks persisted context
+for existing containers and the creation context for new ones. Non-GPU sandboxes
+receive no CDI context or spec mounts.
+
 ## Driver Config Mounts
 
 The gateway forwards the `docker` block from `--driver-config-json`. Supported

crates/openshell-sandbox/README.md

@@ -17,3 +17,54 @@ the kernel then tracks the lifetime through workload-owned descriptors. Native
 the last workload reference to a listener removes it without waiting for registry
 collection. Broker source descriptors always use close-on-exec independently of
 the workload's requested descriptor flags.
+
+## CDI policy provenance
+
+The workload resolves trusted CDI context into filesystem requirements. Library
+mounts expand to their parent directory; the expanded path must pass the same
+broad-root rejection as other CDI destinations. Specs are trusted host
+administrator input. A shared-library parent grant also makes sibling files
+readable, so administrators must choose library directories accordingly, including
+when a spec places libraries under `/etc`, `/home`, or `/root`. These checks do
+not turn administrator-authored specs into an untrusted-input interface.
+CDI enrichment adds only spec-derived paths; it does not grant all of `/sys`.
+
+`WorkloadLaunchContext` retains derived filesystem requirements and required GIDs
+separately from the authored policy. The initial process launcher and retained
+exec executor share this context. Protobuf and boundary policy messages carry
+no derived requirements or GIDs; the workload resolves them from trusted CDI context.
+The sandbox processes deduplicated CDI targets sequentially: open a target,
+validate its type and broad-root object identity, add matching Landlock rules
+using that same descriptor, and close it before opening the next target.
+Targets covered by authored ancestors are validated without adding a rule.
+Only path requirements and root identities are retained; the number of open
+CDI descriptors does not grow with the spec. Symlinks to legitimate library
+directories remain supported. CDI validation and enforcement failures abort launch, including
+when the authored Landlock compatibility is best effort. Authored paths retain
+their configured opening behavior; absent authored paths can still be skipped.
+Ruleset construction and enforcement remain mandatory when CDI filesystem targets
+are present, so failures cannot silently discard CDI confinement.
+
+Projected specs reside under `/.openshell/cdi-specs`, which the mandatory baseline
+excludes from agent access independently of authored grants. Fixture tests use
+the production resolver with injected workload mount observations; real Linux
+mount-state tests verify the OS inspection separately.
+
+Original CDI mount destinations remain separate from expanded library parents.
+The workload classifies their effective read-only flags rather than trusting
+spec option defaults, and checks original directory mounts for writable
+submounts using one mount-table snapshot per pass. Ordinary writable file and
+directory mounts are rejected, even when an exact or ancestor `read_write`
+grant covers the destination. Dedicated authorization for CDI host writes is
+not currently supported. The sequential Landlock pass rechecks mount access
+on its opened descriptors and rejects persisted writable requirements too.
+CDI does not add write access to `/proc`; existing thread-name mediation applies.
+Required groups must already be supplied by the container runtime. The sandbox
+checks membership before both the initial process launch and subsequent execs.
+
+CDI Unix socket mounts are recorded as exact socket destinations, separately from
+ordinary file mounts. Their opened objects must remain Unix sockets at each
+launch or exec. They do not derive filesystem grants, expand to parent
+directories, or require writable filesystem grants, regardless of mount flags.
+Communication follows the existing Unix IPC and inode permission checks; this
+validation does not pin the endpoint for later pathname-based connections.

docs/how-it-works/policies/default-policy.mdx

@@ -87,18 +87,58 @@ the sandbox's policy.
 
 ### GPU Sandboxes
 
-On the Docker and VM compute drivers, a sandbox that requests a GPU receives
-additional paths when the corresponding GPU device is present:
+GPU sandboxes without a CDI context receive additional paths when the
+corresponding GPU device is present:
 
 | Access | Paths |
 |---|---|
 | Read-only | `/run/nvidia-persistenced`, `/usr/lib/wsl` |
-| Read-write | `/dev/nvidiactl`, `/dev/nvidia-uvm`, `/dev/nvidia-uvm-tools`, `/dev/nvidia-modeset`, `/dev/dxg`, numbered `/dev/nvidia<N>` device nodes, and `/proc` |
-
-CUDA writes thread names under `/proc` during initialization, so GPU enrichment
-moves `/proc` from read-only to read-write. OpenShell adds each path only when
-it exists in the workload. These paths apply at runtime and are not saved in
-the sandbox's policy.
+| Read-write | `/dev/nvidiactl`, `/dev/nvidia-uvm`, `/dev/nvidia-uvm-tools`, `/dev/nvidia-modeset`, `/dev/dxg`, numbered `/dev/nvidia<N>` device nodes |
+
+CUDA thread-name writes under `/proc` are handled by the existing process
+open mediation. GPU enrichment preserves authored `/proc` access instead of
+adding a blanket write grant. OpenShell adds each path
+only when it exists in the workload. These paths apply at runtime and are not
+saved in the sandbox's policy.
+
+Docker GPU sandboxes receive CDI-derived filesystem
+requirements instead of the hard-coded GPU device and library baseline. The
+workload-side sandbox runtime resolves the selected CDI device specs and adds
+CDI device nodes as read-write paths; listing a device node in
+`filesystem_policy.read_only` stops the launch. For read-only mounts, it adds the parent
+directories of shared libraries whose filenames end in `.so` or a numeric
+SONAME suffix with any number of dot-separated components, such as `.so.1`,
+`.so.1.2`, or `.so.1.2.3`. It retains exact destinations for other read-only
+mounts. OpenShell does not add a derived path when an existing read-only or
+read-write parent already covers it. The container runtime supplies CDI `additionalGids` as supplemental groups.
+Docker releases before 29.5 ignore them. OpenShell checks the workload's groups
+before initial launch and subsequent execs and logs a warning naming any missing
+group. The process still starts, but devices restricted to that group may be
+inaccessible.
+
+Unix socket mounts are validated as sockets at their exact destinations at each
+launch or exec. They do not add filesystem grants or parent directory access and
+do not require `read_write` entries, regardless of mount flags. Communication
+uses the existing Unix IPC and socket permission checks.
+
+Ordinary CDI file and directory mounts must be effectively read-only. OpenShell
+rejects writable mounts even when `filesystem_policy.read_write` names their
+exact destination or an ancestor. Explicit authorization for CDI host writes is
+not currently supported. This restriction does not apply to device nodes or
+Unix socket communication.
+
+The NVIDIA MPS pipe directory `/tmp/nvidia-mps` is the only writable directory
+mount allowed. When CDI mounts it with `nosuid`, `nodev`, and `noexec`, OpenShell
+grants read-write access to that exact directory so MPS clients can open its
+pipes. OpenShell checks the directory and its mount flags at each launch or
+exec. Listing it in `filesystem_policy.read_only` stops the launch.
+
+CDI paths such as `/`, `/dev`, `/proc`, `/sys`, `/run`, or `/usr` remain rejected
+during sandbox startup, even when authored policy grants access to those roots.
+
+CDI does not add a blanket `/sys` grant. If an application needs sysfs paths
+that the selected CDI spec does not supply, include those paths in your authored
+filesystem policy.
 
 ### Protected Paths
 
@@ -108,6 +148,12 @@ filesystem policy is applied on top of that baseline and can narrow access, but
 it cannot expose `/.openshell`. The `best_effort` compatibility setting does not
 disable this protection or allow a kernel without ABI v3.
 
+CDI targets are validated strictly before each launch or exec. Authored paths
+retain their configured opening behavior, so an absent authored path can still
+be skipped in best-effort mode. When CDI filesystem targets are present,
+ruleset construction and enforcement failures abort launch instead of falling
+back to an unrestricted workload.
+
 ## Inspect the Selected Policy
 
 To see which policy a sandbox uses, print its base and effective policies.

docs/how-it-works/sandboxes/overview.mdx

@@ -155,6 +155,12 @@ For Docker-backed sandboxes, GPU injection uses Docker CDI. If you enable Docker
 CDI after the gateway starts, restart the gateway so OpenShell can detect the
 updated Docker daemon capability.
 
+For CDI-enabled Docker sandboxes, OpenShell derives filesystem access from the
+selected CDI devices and validates those targets before each launch and exec.
+Invalid targets stop the launch. Refer to
+[GPU Sandboxes](/how-it-works/policies/default-policy#gpu-sandboxes) for the
+paths OpenShell adds and the mounts it rejects.
+
 Docker and Podman refresh the CDI inventory before validating or creating a
 default GPU request, so CDI devices added or removed after the driver starts can
 be reflected in later sandbox creates. On WSL2 all-only runtimes, the default

docs/how-it-works/sandboxes/runtimes.mdx

@@ -121,6 +121,31 @@ Common options in `[openshell.drivers.docker]` are `socket_path`, `grpc_endpoint
 
 Docker Desktop must have host networking enabled, and it cannot use Enhanced Container Isolation. Set `grpc_endpoint` when sandboxes cannot reach the gateway on host loopback. For GPU sandboxes, configure Docker CDI before starting the gateway.
 
+For Docker GPU/CDI sandboxes, OpenShell uses Docker's selected CDI device IDs
+and daemon-reported CDI spec directories to build a workload CDI context. The
+driver embeds the context in the protected boundary configuration and
+projects the spec directories read-only under the protected `/.openshell/cdi-specs`
+hierarchy. User mounts and image volumes cannot target this hierarchy. These
+structured binds neither create missing host directories nor relabel existing
+directories. OpenShell skips spec directories that do not exist on the Docker
+host. At least one directory must remain and every selected device must resolve from
+its specs. On SELinux hosts, the host policy must allow containers to read the
+directories; a denial fails the workload launch. Agent processes cannot read the
+projections, even with a broad authored filesystem grant. The sandbox runtime derives filesystem requirements from CDI specs and
+checks supplemental groups before launching agent processes. Non-GPU Docker
+sandboxes do not receive the CDI context, spec mounts, or CDI-derived policy
+changes.
+
+Existing Docker GPU containers without CDI context retain the legacy GPU
+filesystem baseline on restart. Admission requires spec projections only when
+the driver-owned boundary configuration contains CDI context, and validates
+those projections against the directories recorded at creation. Missing or
+unreadable boundary configuration blocks restart. To use CDI-derived access on
+an older sandbox, recreate it. Ordinary CDI file and directory mounts must be
+effectively read-only. Filesystem `read_write` grants do not authorize writable
+CDI mounts; explicit authorization for CDI host writes is not currently
+supported. Device nodes and Unix socket communication remain supported.
+
 ### Docker Corporate Proxy Egress
 
 For proxy-required networks, the Docker driver accepts `https_proxy`,