Skip to content

Commit d365bb1

Browse files
committed
test(cdi): assert CDI policy scenarios in the tmachine suite
Run the device-node-with-gid policy fixture as a non-root workload and require the device group among its groups and read-write access to the device. Add MPS pipe directory fixtures mounted with the NVIDIA Container Toolkit IPC options, and a negative variant without noexec. Signed-off-by: Evan Lezar <elezar@nvidia.com>
1 parent 427aadd commit d365bb1

4 files changed

Lines changed: 61 additions & 1 deletion

File tree

‎TESTING.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -384,14 +384,17 @@ the two-device default GPU inventory. Select them with `--gpu 1` and the Docker
384384
| `omitted-access-file` | Bind without `ro` or `rw` | Effective writable access causes launch rejection. |
385385
| `device-node` | Null character device at `/dev/tmachine-cdi-null` | Non-root read/write succeeds; an exact authored read-only entry fails launch. |
386386
| `device-node-with-gid` | Group-restricted null device at `/dev/tmachine-cdi-group-null` | Docker supplies the generated supplementary GID and permits non-root access. |
387+
| `mps-pipe-directory` | Pipe directory with a `control` FIFO at `/tmp/nvidia-mps`, mounted with the NVIDIA Container Toolkit's IPC options | Non-root workload can open the FIFO read-write and create files in the directory. |
388+
| `mps-pipe-directory-without-noexec` | The same directory without `noexec` | Launch rejects the IPC directory. |
387389
| `unix-socket` | Echo socket at `/tmachine-cdi/socket/echo.sock` | Socket communication succeeds without deriving parent filesystem grants. |
388390

389391
Every scenario also supplies a spec-level environment marker and a read-only
390392
mount at `/tmachine-cdi/common/spec-marker`, allowing checks of spec/device edit
391393
merging. The device group is allocated in the disposable guest. A systemd
392394
socket unit restores the echo endpoint after reboot; the existing tmpfiles
393395
configuration restores the scenario spec. These fixtures prepare the resources;
394-
the existing eight tests do not yet assert the policy scenarios. Negative
396+
only `device-node-with-gid` is asserted so far. It requires Docker 29.5 or
397+
later, which applies CDI `additionalGids`. Negative
395398
scenarios must be selected individually, not combined with positive cases.
396399

397400
Directory absence, daemon configuration refresh, and restart tests require

‎tests/ansible/roles/cdi_fixtures/tasks/scenarios.yaml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,21 @@
6161
group: openshell-cdi-fixture
6262
mode: "0660"
6363

64+
- name: Create MPS pipe directory fixture
65+
become: true
66+
ansible.builtin.file:
67+
path: /var/lib/openshell-test-inputs/cdi/policy/mps-pipes
68+
state: directory
69+
owner: root
70+
group: root
71+
mode: "0777"
72+
73+
- name: Create MPS control pipe fixture
74+
become: true
75+
ansible.builtin.command:
76+
argv: [mkfifo, --mode=0666, /var/lib/openshell-test-inputs/cdi/policy/mps-pipes/control]
77+
creates: /var/lib/openshell-test-inputs/cdi/policy/mps-pipes/control
78+
6479
- name: Install socket fixture units
6580
become: true
6681
ansible.builtin.copy:

‎tests/ansible/roles/cdi_fixtures/templates/policy.yaml.j2‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,18 @@ devices:
6969
uid: 0
7070
gid: {{ cdi_fixture_group.gid }}
7171
fileMode: 432
72+
- name: mps-pipe-directory
73+
containerEdits:
74+
mounts:
75+
- hostPath: /var/lib/openshell-test-inputs/cdi/policy/mps-pipes
76+
containerPath: /tmp/nvidia-mps
77+
options: [nosuid, nodev, rbind, rprivate, noexec]
78+
- name: mps-pipe-directory-without-noexec
79+
containerEdits:
80+
mounts:
81+
- hostPath: /var/lib/openshell-test-inputs/cdi/policy/mps-pipes
82+
containerPath: /tmp/nvidia-mps
83+
options: [nosuid, nodev, rbind, rprivate]
7284
- name: unix-socket
7385
containerEdits:
7486
mounts:

‎tests/suites/features/cdi/tests/injection.rs‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -196,3 +196,33 @@ async fn explicit_device_count_mismatch_is_rejected() {
196196
)
197197
.await;
198198
}
199+
200+
/// The group-restricted device is mode 0660 with the group the spec supplies
201+
/// through `additionalGids`, so non-root access requires the runtime to apply it.
202+
const GROUP_DEVICE_PROBE: &str = r#"
203+
set -eu
204+
device=/dev/tmachine-cdi-group-null
205+
test "$(id -u)" != 0
206+
test -c "$device"
207+
set -- $(ls -ln "$device")
208+
gid=$4
209+
case " $(id -G) " in
210+
*" $gid "*) ;;
211+
*) echo "workload groups $(id -G) lack device group $gid" >&2; exit 1 ;;
212+
esac
213+
head -c 0 "$device"
214+
printf x > "$device"
215+
"#;
216+
217+
#[tokio::test]
218+
async fn additional_gids_grant_group_restricted_device_access() {
219+
check_case(
220+
"cdi-policy-device-node-with-gid",
221+
Some("1"),
222+
Some(&["tmachine.example/cdi-policy=device-node-with-gid"]),
223+
GROUP_DEVICE_PROBE,
224+
Some(&[]),
225+
None,
226+
)
227+
.await;
228+
}

0 commit comments

Comments
 (0)