From 08eab4eb55eca2d8bb9d1e9cb59925f21c6c863b Mon Sep 17 00:00:00 2001 From: Elliot Michlin <31219104+theelliotm@users.noreply.github.com> Date: Thu, 24 Sep 2026 18:30:25 -0700 Subject: [PATCH 1/7] Update bubblewrap testing --- docs/bwrap-support/bubblewrap-backend.md | 42 +++++-- tests/configs/bubblewrap_basic.json | 2 +- tests/configs/bubblewrap_filesystem.json | 2 +- tests/configs/bubblewrap_network_block.json | 2 +- .../configs/bubblewrap_network_firewall.json | 2 +- .../bubblewrap_network_firewall_cidr.json | 3 +- .../bubblewrap_network_proxy_allowlist.json | 2 +- .../bubblewrap_network_proxy_blocklist.json | 2 +- .../bubblewrap_network_proxy_builtin.json | 2 +- .../bubblewrap_network_proxy_connect.json | 10 +- ...ubblewrap_network_proxy_egress_denied.json | 9 +- .../bubblewrap_network_proxy_namespace.json | 9 +- tests/configs/linux_process_abstract.json | 2 +- tests/configs/linux_process_default.json | 2 +- tests/scripts/run_bwrap_network_proxy_test.sh | 109 ++++++++++++------ tests/scripts/run_bwrap_network_test.sh | 8 +- 16 files changed, 135 insertions(+), 73 deletions(-) diff --git a/docs/bwrap-support/bubblewrap-backend.md b/docs/bwrap-support/bubblewrap-backend.md index 866d1012d..e6716d8ee 100644 --- a/docs/bwrap-support/bubblewrap-backend.md +++ b/docs/bwrap-support/bubblewrap-backend.md @@ -668,11 +668,12 @@ request fails if its private namespace cannot be configured. proxy-mode execution on the host indefinitely. A successful probe is cached for the life of the process; failures are not, so installing the missing tool takes effect without a restart. -1. When `network.proxy` is set, the runner launches an unprivileged HTTP - proxy on loopback (`127.0.0.1:N`). For tests, the bundled - `unix-test-proxy` binary is used (`builtinTestServer: true`, - testing-only and gated behind `--allow-testing-features`); in production callers - supply their own proxy via `localhost: ` or `url: `. +1. When a proxy is requested, the runner routes the sandbox to it. On v0.9 the + proxy is named by `runtimeConfig.networkProxy` and must already be listening + on loopback; the caller starts it. On schema 0.6–0.8 it is named by + `network.proxy`, and `builtinTestServer: true` additionally makes the runner + launch the bundled `unix-test-proxy` on loopback (testing-only, gated behind + `--allow-testing-features`). 2. The runner creates a same-UID user-namespace supervisor, starts Bubblewrap with `--unshare-net`, and keeps the workload behind a startup barrier. 3. The supervisor attaches `slirp4netns` to Bubblewrap's private network @@ -747,7 +748,25 @@ The monitor is disarmed *before* teardown stops the supervisor, so an ordinary shutdown — which closes the same descriptor — is never reported as a loss. Only an exit is detected; see [Limitations](#limitations). -### Example: builtin test proxy with allowlist +### Example: proxy on v0.9 + +```json +{ + "version": "0.9.0-alpha", + "containment": "bubblewrap", + "process": { "commandLine": "curl -fsSL https://example.com" }, + "network": { + "egress": { "default": "deny" }, + "ingress": { "default": "deny", "hostLoopback": "deny" } + }, + "runtimeConfig": { "networkProxy": "http://127.0.0.1:8080" } +} +``` + +A proxy request is the proxy-only posture, so `egress.default` must be `deny` +with no `allow` / `deny` rules; the chain opens the proxy endpoint alone. + +### Example (legacy, ≤0.8): builtin test proxy with allowlist ```json { @@ -765,7 +784,7 @@ an exit is detected; see [Limitations](#limitations). } ``` -### Example: external proxy on loopback +### Example (legacy, ≤0.8): external proxy on loopback ```json { @@ -778,10 +797,11 @@ an exit is detected; see [Limitations](#limitations). } ``` -> Both examples declare `0.8.0-alpha` deliberately: the private-namespace and -> egress-enforcement behavior described above is selected by the schema version, -> so the same config on `0.6`/`0.7` runs the legacy shared-host-network proxy -> path instead. +> Both legacy examples declare `0.8.0-alpha` deliberately: the +> private-namespace and egress-enforcement behavior described above is selected +> by the schema version, so the same config on `0.6`/`0.7` runs the legacy +> shared-host-network proxy path instead. `network.proxy`, `allowedHosts`, and +> `blockedHosts` are not accepted on v0.9. ### Checking host support before you run diff --git a/tests/configs/bubblewrap_basic.json b/tests/configs/bubblewrap_basic.json index 376f81234..33ea0f460 100644 --- a/tests/configs/bubblewrap_basic.json +++ b/tests/configs/bubblewrap_basic.json @@ -1,5 +1,5 @@ { - "version": "0.6.0-alpha", + "version": "0.9.0-alpha", "containerId": "CLI-Bubblewrap-Hello-World", "containment": "bubblewrap", "process": { diff --git a/tests/configs/bubblewrap_filesystem.json b/tests/configs/bubblewrap_filesystem.json index 66e7016ce..b63f5871a 100644 --- a/tests/configs/bubblewrap_filesystem.json +++ b/tests/configs/bubblewrap_filesystem.json @@ -1,5 +1,5 @@ { - "version": "0.6.0-alpha", + "version": "0.9.0-alpha", "containerId": "CLI-Bubblewrap-Filesystem-Test", "containment": "bubblewrap", "process": { diff --git a/tests/configs/bubblewrap_network_block.json b/tests/configs/bubblewrap_network_block.json index 4f64e59e5..8749b2edb 100644 --- a/tests/configs/bubblewrap_network_block.json +++ b/tests/configs/bubblewrap_network_block.json @@ -1,5 +1,5 @@ { - "version": "0.6.0-alpha", + "version": "0.7.0-alpha", "containerId": "CLI-Bubblewrap-Network-Block", "containment": "bubblewrap", "process": { diff --git a/tests/configs/bubblewrap_network_firewall.json b/tests/configs/bubblewrap_network_firewall.json index 2f0c30f81..eaf8e17e7 100644 --- a/tests/configs/bubblewrap_network_firewall.json +++ b/tests/configs/bubblewrap_network_firewall.json @@ -1,5 +1,5 @@ { - "version": "0.6.0-alpha", + "version": "0.7.0-alpha", "containerId": "CLI-Bubblewrap-Network-Firewall", "containment": "bubblewrap", "process": { diff --git a/tests/configs/bubblewrap_network_firewall_cidr.json b/tests/configs/bubblewrap_network_firewall_cidr.json index f7e566d59..9f4400834 100644 --- a/tests/configs/bubblewrap_network_firewall_cidr.json +++ b/tests/configs/bubblewrap_network_firewall_cidr.json @@ -3,7 +3,8 @@ "containerId": "CLI-Bubblewrap-Network-Firewall-CIDR", "containment": "bubblewrap", "process": { - "commandLine": "bash -c 'set -u; echo SANDBOX_NETNS=$(readlink /proc/self/ns/net); if ! timeout 6 bash -c \"exec 3<>/dev/tcp/10.0.2.2/{{ALLOWED_PORT}}\" >/dev/null 2>&1; then echo ALLOWED_DEST_UNREACHABLE; exit 1; fi; echo ALLOWED_DEST_OK; timeout 6 bash -c \"exec 3<>/dev/tcp/1.1.1.1/443\" >/dev/null 2>&1; if [ $? = 0 ]; then echo DENIED_DEST_LEAKED; exit 1; fi; echo DENIED_DEST_BLOCKED_OK; timeout 4 bash -c \"exec 3<>/dev/tcp/127.0.0.1/9\" >/dev/null 2>&1; if [ $? = 124 ]; then echo LOOPBACK_DROPPED; exit 1; fi; echo LOOPBACK_EXEMPT_OK; capeff=$(grep \"^CapEff\" /proc/self/status | cut -f2); if [ $((0x$capeff & 0x1000)) -ne 0 ]; then echo CAP_NET_ADMIN_RETAINED; exit 1; fi; echo CAP_NET_ADMIN_DROPPED_OK; iptables -F MXC_EGRESS >/dev/null 2>&1; if [ $? = 0 ]; then echo TAMPER_FLUSH_SUCCEEDED; exit 1; fi; echo TAMPER_REFUSED_OK; timeout 6 bash -c \"exec 3<>/dev/tcp/1.1.1.1/443\" >/dev/null 2>&1; if [ $? = 0 ]; then echo TAMPER_DISABLED_EGRESS; exit 1; fi; echo TAMPER_INEFFECTIVE_OK'" + "commandLine": "bash -c 'set -u; echo SANDBOX_NETNS=$(readlink /proc/self/ns/net); if ! timeout 6 bash -c \"exec 3<>/dev/tcp/10.0.2.2/{{ALLOWED_PORT}}\" >/dev/null 2>&1; then echo ALLOWED_DEST_UNREACHABLE; exit 1; fi; echo ALLOWED_DEST_OK; timeout 6 bash -c \"exec 3<>/dev/tcp/1.1.1.1/443\" >/dev/null 2>&1; if [ $? = 0 ]; then echo DENIED_DEST_LEAKED; exit 1; fi; echo DENIED_DEST_BLOCKED_OK; timeout 4 bash -c \"exec 3<>/dev/tcp/127.0.0.1/9\" >/dev/null 2>&1; if [ $? = 124 ]; then echo LOOPBACK_DROPPED; exit 1; fi; echo LOOPBACK_EXEMPT_OK; if ! command -v iptables >/dev/null 2>&1; then echo NO_IPTABLES_BINARY; exit 1; fi; capeff=$(grep \"^CapEff\" /proc/self/status | cut -f2); if [ $((0x$capeff & 0x1000)) -ne 0 ]; then echo CAP_NET_ADMIN_RETAINED; exit 1; fi; echo CAP_NET_ADMIN_DROPPED_OK; terr=$(iptables -F MXC_EGRESS 2>&1); trc=$?; if [ \"$trc\" = 0 ]; then echo \"TAMPER_FLUSH_SUCCEEDED: $terr\"; exit 1; fi; echo \"TAMPER_REFUSED_OK rc=$trc\"; timeout 6 bash -c \"exec 3<>/dev/tcp/1.1.1.1/443\" >/dev/null 2>&1; if [ $? = 0 ]; then echo TAMPER_DISABLED_EGRESS; exit 1; fi; echo TAMPER_INEFFECTIVE_OK'", + "env": ["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"] }, "network": { "defaultPolicy": "block", diff --git a/tests/configs/bubblewrap_network_proxy_allowlist.json b/tests/configs/bubblewrap_network_proxy_allowlist.json index d780e6f46..3d640beb0 100644 --- a/tests/configs/bubblewrap_network_proxy_allowlist.json +++ b/tests/configs/bubblewrap_network_proxy_allowlist.json @@ -1,5 +1,5 @@ { - "version": "0.6.0-alpha", + "version": "0.7.0-alpha", "containerId": "CLI-Bubblewrap-Network-Proxy-Allowlist", "containment": "bubblewrap", "process": { diff --git a/tests/configs/bubblewrap_network_proxy_blocklist.json b/tests/configs/bubblewrap_network_proxy_blocklist.json index 9ffd87118..48f76ee8e 100644 --- a/tests/configs/bubblewrap_network_proxy_blocklist.json +++ b/tests/configs/bubblewrap_network_proxy_blocklist.json @@ -1,5 +1,5 @@ { - "version": "0.6.0-alpha", + "version": "0.7.0-alpha", "containerId": "CLI-Bubblewrap-Network-Proxy-Blocklist", "containment": "bubblewrap", "process": { diff --git a/tests/configs/bubblewrap_network_proxy_builtin.json b/tests/configs/bubblewrap_network_proxy_builtin.json index 028bed5a8..37b033386 100644 --- a/tests/configs/bubblewrap_network_proxy_builtin.json +++ b/tests/configs/bubblewrap_network_proxy_builtin.json @@ -1,5 +1,5 @@ { - "version": "0.6.0-alpha", + "version": "0.7.0-alpha", "containerId": "CLI-Bubblewrap-Network-Proxy-Builtin", "containment": "bubblewrap", "process": { diff --git a/tests/configs/bubblewrap_network_proxy_connect.json b/tests/configs/bubblewrap_network_proxy_connect.json index 2bcee95bb..0b74c48a3 100644 --- a/tests/configs/bubblewrap_network_proxy_connect.json +++ b/tests/configs/bubblewrap_network_proxy_connect.json @@ -1,13 +1,15 @@ { - "version": "0.8.0-alpha", + "version": "0.9.0-alpha", "containerId": "CLI-Bubblewrap-Network-Proxy-Connect", "containment": "bubblewrap", "process": { "commandLine": "bash -c 'set -u; pu=\"${HTTP_PROXY:-${http_proxy:-}}\"; if [ -z \"$pu\" ]; then echo CONNECT_NO_PROXY_ENV; exit 1; fi; pa=\"${pu#*://}\"; pa=\"${pa%%/*}\"; ph=\"${pa%%:*}\"; pp=\"${pa##*:}\"; if [ \"$pp\" = \"$ph\" ]; then pp=80; fi; if timeout 6 bash -c \"exec 3<>/dev/tcp/10.0.2.2/{{CONTROL_PORT}}\" >/dev/null 2>&1; then echo CONNECT_DIRECT_LEAKED; exit 1; fi; echo CONNECT_DIRECT_BLOCKED_OK; exec 3<>/dev/tcp/$ph/$pp; printf \"CONNECT 127.0.0.1:{{CONTROL_PORT}} HTTP/1.1\\r\\nHost: 127.0.0.1:{{CONTROL_PORT}}\\r\\n\\r\\n\" >&3; if ! IFS= read -r -t 10 status <&3; then echo CONNECT_NO_RESPONSE; exit 1; fi; case \"$status\" in *200*) echo CONNECT_TUNNEL_ESTABLISHED_OK;; *) echo \"CONNECT_TUNNEL_REFUSED:$status\"; exit 1;; esac; printf \"GET http://mxc-test.invalid/ HTTP/1.1\\r\\nHost: mxc-test.invalid\\r\\nConnection: close\\r\\n\\r\\n\" >&3; resp=\"$(timeout 10 cat <&3)\"; case \"$resp\" in *MXC_TEST_ORIGIN_OK*) echo CONNECT_TUNNEL_BODY_OK;; *) echo CONNECT_TUNNEL_BODY_MISSING; exit 1;; esac'" }, "network": { - "defaultPolicy": "block", - "proxy": { "builtinTestServer": true }, - "allowedHosts": ["127.0.0.1"] + "egress": { "default": "deny" }, + "ingress": { "default": "deny", "hostLoopback": "deny" } + }, + "runtimeConfig": { + "networkProxy": "http://127.0.0.1:{{PROXY_PORT}}" } } diff --git a/tests/configs/bubblewrap_network_proxy_egress_denied.json b/tests/configs/bubblewrap_network_proxy_egress_denied.json index e64d1a40d..b48f30e69 100644 --- a/tests/configs/bubblewrap_network_proxy_egress_denied.json +++ b/tests/configs/bubblewrap_network_proxy_egress_denied.json @@ -1,12 +1,15 @@ { - "version": "0.8.0-alpha", + "version": "0.9.0-alpha", "containerId": "CLI-Bubblewrap-Network-Proxy-Egress-Denied", "containment": "bubblewrap", "process": { "commandLine": "bash -c 'set -u; pu=\"${HTTP_PROXY:-${http_proxy:-}}\"; if [ -z \"$pu\" ]; then echo NO_PROXY_ENV; exit 1; fi; pa=\"${pu#*://}\"; pa=\"${pa%%/*}\"; ph=\"${pa%%:*}\"; pp=\"${pa##*:}\"; if [ \"$pp\" = \"$ph\" ]; then pp=80; fi; if ! timeout 6 bash -c \"exec 3<>/dev/tcp/$ph/$pp\" >/dev/null 2>&1; then echo CONTROL_PROXY_UNREACHABLE; exit 1; fi; echo CONTROL_PROXY_REACHABLE_OK; timeout 6 bash -c \"exec 3<>/dev/tcp/10.0.2.2/{{CONTROL_PORT}}\" >/dev/null 2>&1; rc=$?; if [ \"$rc\" = 0 ]; then echo DIRECT_EGRESS_LEAKED; exit 1; fi; echo DIRECT_EGRESS_BLOCKED_OK; timeout 4 bash -c \"exec 3<>/dev/tcp/127.0.0.1/9\" >/dev/null 2>&1; lrc=$?; if [ \"$lrc\" = 124 ]; then echo LOOPBACK_DROPPED; exit 1; fi; echo LOOPBACK_EXEMPT_OK; if ! command -v iptables >/dev/null 2>&1; then echo NO_IPTABLES_BINARY; exit 1; fi; capeff=$(grep \"^CapEff\" /proc/self/status | cut -f2); if [ -z \"$capeff\" ]; then echo NO_CAPEFF; exit 1; fi; if [ $((0x$capeff & 0x1000)) -ne 0 ]; then echo CAP_NET_ADMIN_RETAINED; exit 1; fi; echo CAP_NET_ADMIN_DROPPED_OK; terr=$(iptables -F MXC_EGRESS 2>&1); trc=$?; if [ \"$trc\" = 0 ]; then echo \"TAMPER_FLUSH_SUCCEEDED: $terr\"; exit 1; fi; echo \"TAMPER_REFUSED_OK rc=$trc\"; timeout 6 bash -c \"exec 3<>/dev/tcp/10.0.2.2/{{CONTROL_PORT}}\" >/dev/null 2>&1; if [ $? = 0 ]; then echo TAMPER_DISABLED_EGRESS; exit 1; fi; echo TAMPER_INEFFECTIVE_OK; if ! curl -fsSL --max-time 15 http://mxc-test.invalid/ 2>/dev/null | grep -q MXC_TEST_ORIGIN_OK; then echo PROXY_UNREACHABLE; exit 1; fi; echo PROXY_STILL_OK'" }, "network": { - "defaultPolicy": "allow", - "proxy": { "builtinTestServer": true } + "egress": { "default": "deny" }, + "ingress": { "default": "deny", "hostLoopback": "deny" } + }, + "runtimeConfig": { + "networkProxy": "http://127.0.0.1:{{PROXY_PORT}}" } } diff --git a/tests/configs/bubblewrap_network_proxy_namespace.json b/tests/configs/bubblewrap_network_proxy_namespace.json index 4137c6878..2aa966966 100644 --- a/tests/configs/bubblewrap_network_proxy_namespace.json +++ b/tests/configs/bubblewrap_network_proxy_namespace.json @@ -1,12 +1,15 @@ { - "version": "0.8.0-alpha", + "version": "0.9.0-alpha", "containerId": "CLI-Bubblewrap-Network-Proxy-Namespace", "containment": "bubblewrap", "process": { "commandLine": "set -e; echo SANDBOX_NETNS=$(readlink /proc/self/ns/net); echo SANDBOX_CAPBND=$(grep ^CapBnd: /proc/self/status | cut -f2); echo SANDBOX_CAPEFF=$(grep ^CapEff: /proc/self/status | cut -f2); echo SANDBOX_CAPPRM=$(grep ^CapPrm: /proc/self/status | cut -f2); curl -fsSL --max-time 15 http://mxc-test.invalid/ | grep -q MXC_TEST_ORIGIN_OK; echo PROXY_NAMESPACE_OK" }, "network": { - "defaultPolicy": "allow", - "proxy": { "builtinTestServer": true } + "egress": { "default": "deny" }, + "ingress": { "default": "deny", "hostLoopback": "deny" } + }, + "runtimeConfig": { + "networkProxy": "http://127.0.0.1:{{PROXY_PORT}}" } } diff --git a/tests/configs/linux_process_abstract.json b/tests/configs/linux_process_abstract.json index c4e47d428..4533be5e6 100644 --- a/tests/configs/linux_process_abstract.json +++ b/tests/configs/linux_process_abstract.json @@ -1,5 +1,5 @@ { - "version": "0.6.0-alpha", + "version": "0.9.0-alpha", "containerId": "CLI-Linux-Process-Abstract", "containment": "process", "process": { diff --git a/tests/configs/linux_process_default.json b/tests/configs/linux_process_default.json index 6fc85bc54..1becb6e3f 100644 --- a/tests/configs/linux_process_default.json +++ b/tests/configs/linux_process_default.json @@ -1,5 +1,5 @@ { - "version": "0.6.0-alpha", + "version": "0.9.0-alpha", "containerId": "CLI-Linux-Process-Default", "process": { "commandLine": "PID1=$(cat /proc/1/comm 2>/dev/null || echo unknown); MOUNTS=$(wc -l /dev/null || true - wait "$CONTROL_PID" 2>/dev/null || true - fi + local pid + for pid in "$CONTROL_PID" "$PROXY_ENDPOINT_PID"; do + if [ -n "$pid" ]; then + kill "$pid" 2>/dev/null || true + wait "$pid" 2>/dev/null || true + fi + done + exec 7>&- 2>/dev/null || true exec 8>&- 2>/dev/null || true rm -rf "$CONTROL_DIR" } @@ -70,7 +77,7 @@ mkfifo "$CONTROL_DIR/parent.pipe" exec 8<>"$CONTROL_DIR/parent.pipe" # 8>&- keeps the listener from inheriting the write end: holding one itself # would mean its stdin never reports EOF, orphaning it if this script is killed. -"$CONTROL_PROXY" --ready-file "$CONTROL_DIR/ready.port" --bind-address 127.0.0.1 \ +"$PROXY_BIN" --ready-file "$CONTROL_DIR/ready.port" --bind-address 127.0.0.1 \ <"$CONTROL_DIR/parent.pipe" >"$CONTROL_DIR/listener.log" 2>&1 8>&- & CONTROL_PID=$! for _ in $(seq 1 100); do @@ -90,10 +97,40 @@ if [ -z "$CONTROL_PORT" ]; then fi echo " control listener on 127.0.0.1:$CONTROL_PORT (10.0.2.2:$CONTROL_PORT from the sandbox)" -# The control port is assigned by the OS per run, so configs carry a -# placeholder and are rendered into the run's scratch directory. +# Schema 0.9 removed `network.proxy` entirely, so `builtinTestServer` has no +# 0.9 spelling: a proxy is a real endpoint named by +# `runtimeConfig.networkProxy`, which the parser accepts only on loopback. This +# second listener is that endpoint, and the backend translates it to slirp's +# gateway on the way in. It serves `http://mxc-test.invalid/`, so the 0.9 +# workloads assert the same sentinels the builtin server used to produce. +mkfifo "$CONTROL_DIR/proxy.pipe" +exec 7<>"$CONTROL_DIR/proxy.pipe" +# 7>&- and 8>&- keep this listener from holding either fifo's write end open, +# which would stop its own stdin from ever reporting EOF. +"$PROXY_BIN" --ready-file "$CONTROL_DIR/proxy.port" --bind-address 127.0.0.1 \ + <"$CONTROL_DIR/proxy.pipe" >"$CONTROL_DIR/proxy.log" 2>&1 7>&- 8>&- & +PROXY_ENDPOINT_PID=$! +for _ in $(seq 1 100); do + [ -s "$CONTROL_DIR/proxy.port" ] && break + if ! kill -0 "$PROXY_ENDPOINT_PID" 2>/dev/null; then + cat "$CONTROL_DIR/proxy.log" + echo "FAIL: the proxy endpoint exited before publishing its port." + exit 1 + fi + sleep 0.1 +done +PROXY_PORT="$(cat "$CONTROL_DIR/proxy.port" 2>/dev/null || true)" +if [ -z "$PROXY_PORT" ]; then + cat "$CONTROL_DIR/proxy.log" + echo "FAIL: the proxy endpoint did not publish a port." + exit 1 +fi +echo " proxy endpoint on 127.0.0.1:$PROXY_PORT (10.0.2.2:$PROXY_PORT from the sandbox)" + +# Both ports are assigned by the OS per run, so configs carry placeholders and +# are rendered into the run's scratch directory. render_config() { - sed -e "s/{{CONTROL_PORT}}/$CONTROL_PORT/g" \ + sed -e "s/{{CONTROL_PORT}}/$CONTROL_PORT/g" -e "s/{{PROXY_PORT}}/$PROXY_PORT/g" \ "$REPO_DIR/tests/configs/$1" >"$CONTROL_DIR/$1" printf '%s\n' "$CONTROL_DIR/$1" } @@ -154,10 +191,10 @@ if [ -z "$HOSTRULES_NETNS" ] || [ "$HOSTRULES_NETNS" = "$HOST_NETNS" ]; then fi echo "PASS: proxy host-rules egress" -echo "Running Bubblewrap private proxy namespace test..." +echo "Running Bubblewrap private proxy namespace test (schema 0.9)..." HOST_NETNS="$(readlink /proc/self/ns/net)" -if ! NAMESPACE_OUT=$("$LXC_EXEC" --experimental --allow-testing-features \ - "$REPO_DIR/tests/configs/bubblewrap_network_proxy_namespace.json" 2>&1); then +NAMESPACE_CONFIG="$(render_config bubblewrap_network_proxy_namespace.json)" +if ! NAMESPACE_OUT=$("$LXC_EXEC" --experimental "$NAMESPACE_CONFIG" 2>&1); then echo "$NAMESPACE_OUT" echo "FAIL: private proxy namespace (lxc-exec returned non-zero)" exit 1 @@ -271,8 +308,8 @@ STUB chmod +x "$BWRAP_STUB_DIR/bwrap" SUPERVISOR_PATTERN="mxc-bwrap-proxy-supervisor" -PATH="$BWRAP_STUB_DIR:$PATH" "$LXC_EXEC" --experimental --allow-testing-features \ - "$REPO_DIR/tests/configs/bubblewrap_network_proxy_namespace.json" >/dev/null 2>&1 & +PATH="$BWRAP_STUB_DIR:$PATH" "$LXC_EXEC" --experimental \ + "$NAMESPACE_CONFIG" >/dev/null 2>&1 & ORPHAN_EXEC_PID=$! SUPERVISOR_SEEN=0 @@ -319,8 +356,8 @@ if [ "$SUPERVISOR_REAPED" -ne 1 ]; then fi echo "PASS: supervisor orphan reaping" -echo "Running Bubblewrap proxy-only egress enforcement test..." -if ! EGRESS_OUT=$("$LXC_EXEC" --experimental --allow-testing-features \ +echo "Running Bubblewrap proxy-only egress enforcement test (schema 0.9)..." +if ! EGRESS_OUT=$("$LXC_EXEC" --experimental \ "$(render_config bubblewrap_network_proxy_egress_denied.json)" 2>&1); then echo "$EGRESS_OUT" echo "FAIL: proxy-only egress (lxc-exec returned non-zero)" @@ -340,8 +377,8 @@ echo "PASS: proxy-only egress enforcement" # path could regress while the suite stayed green. The target is the control # listener, so the sentinel coming back proves both that the tunnel carried # bytes and that the endpoint refused above was live. -echo "Running Bubblewrap proxy CONNECT tunnel test..." -if ! CONNECT_OUT=$("$LXC_EXEC" --experimental --allow-testing-features \ +echo "Running Bubblewrap proxy CONNECT tunnel test (schema 0.9)..." +if ! CONNECT_OUT=$("$LXC_EXEC" --experimental \ "$(render_config bubblewrap_network_proxy_connect.json)" 2>&1); then echo "$CONNECT_OUT" echo "FAIL: proxy CONNECT tunnel (lxc-exec returned non-zero)" @@ -356,9 +393,12 @@ for sentinel in CONNECT_DIRECT_BLOCKED_OK CONNECT_TUNNEL_ESTABLISHED_OK CONNECT_ done echo "PASS: proxy CONNECT tunnel" -# Hostname proxy endpoints. The endpoint is resolved on the host and pinned -# into the sandbox's /etc/hosts, because DNS is closed inside the sandbox. -echo "Running Bubblewrap hostname proxy pin test..." +# Hostname proxy endpoints, which are legacy-only: 0.9 names the proxy through +# `runtimeConfig.networkProxy`, and the parser accepts only loopback there, so +# this and the two /etc/hosts cases below stay on 0.8. The endpoint is resolved +# on the host and pinned into the sandbox's /etc/hosts, because DNS is closed +# inside the sandbox. +echo "Running Bubblewrap hostname proxy pin test (schema 0.8)..." PROXY_HOST="$(hostname)" # The host's own name is used because it resolves everywhere without editing # /etc/hosts (which would need root). Where it points is not fixed, though: a @@ -379,15 +419,6 @@ case "$PROXY_BIND" in 127.*) PROXY_BIND=127.0.0.1 ;; esac echo " host name '$PROXY_HOST' resolves to $PROXY_BIND" -# The proxy lives beside lxc-exec, wherever that came from: CI overrides -# LXC_EXEC with a --target build under src/target//release, which the -# repo-relative fallbacks below do not cover. Every other case in this file -# reaches the proxy through the coordinator, which already resolves it that -# way -- this is the one that spawns it directly. -TEST_PROXY="$(resolve_test_proxy)" || { - echo "FAIL: hostname proxy pin (unix-test-proxy not built)" - exit 1 -} PIN_DIR="$(mktemp -d)" PIN_PROXY_PID="" @@ -410,9 +441,11 @@ mkfifo "$PIN_DIR/parent.pipe" exec 9<>"$PIN_DIR/parent.pipe" # The proxy binds an OS-assigned port and publishes it, so the config is -# generated per run rather than committed with a fixed port. -"$TEST_PROXY" --ready-file "$PIN_DIR/ready.port" --bind-address "$PROXY_BIND" \ - <"$PIN_DIR/parent.pipe" >"$PIN_DIR/proxy.log" 2>&1 & +# generated per run rather than committed with a fixed port. The fifo write +# ends are closed so this proxy cannot hold open its own stdin or either +# listener's. +"$PROXY_BIN" --ready-file "$PIN_DIR/ready.port" --bind-address "$PROXY_BIND" \ + <"$PIN_DIR/parent.pipe" >"$PIN_DIR/proxy.log" 2>&1 7>&- 8>&- 9>&- & PIN_PROXY_PID=$! for _ in $(seq 1 100); do [ -s "$PIN_DIR/ready.port" ] && break @@ -423,14 +456,14 @@ for _ in $(seq 1 100); do fi sleep 0.1 done -PROXY_PORT="$(cat "$PIN_DIR/ready.port" 2>/dev/null || true)" -if [ -z "$PROXY_PORT" ]; then +PIN_PROXY_PORT="$(cat "$PIN_DIR/ready.port" 2>/dev/null || true)" +if [ -z "$PIN_PROXY_PORT" ]; then cat "$PIN_DIR/proxy.log" echo "FAIL: hostname proxy pin (test proxy did not publish a port)" exit 1 fi -sed -e "s/{{PROXY_HOST}}/$PROXY_HOST/g" -e "s/{{PROXY_PORT}}/$PROXY_PORT/g" \ +sed -e "s/{{PROXY_HOST}}/$PROXY_HOST/g" -e "s/{{PROXY_PORT}}/$PIN_PROXY_PORT/g" \ -e "s/{{CONTROL_PORT}}/$CONTROL_PORT/g" \ "$REPO_DIR/tests/configs/bubblewrap_network_proxy_hostname.json" \ >"$PIN_DIR/hostname.json" @@ -506,7 +539,7 @@ cat >"$DOTDOT_DIR/dotdot.json" <&1) @@ -42,8 +42,8 @@ set -e # gate leaked into the legacy schema. if echo "$LEGACY_OUTPUT" | grep -qF "Configuration parse error"; then echo "$LEGACY_OUTPUT" - echo "FAIL: schema 0.6 firewall mode failed config validation (status $LEGACY_STATUS)." + echo "FAIL: schema 0.7 firewall mode failed config validation (status $LEGACY_STATUS)." exit 1 fi -echo "PASS: schema 0.6 firewall mode still accepted." +echo "PASS: schema 0.7 firewall mode still accepted." echo "Bubblewrap firewall-mode tests complete." From 9211450e2ee15f2b045110faa683b6611ef6e9a1 Mon Sep 17 00:00:00 2001 From: Elliot Michlin <31219104+theelliotm@users.noreply.github.com> Date: Fri, 25 Sep 2026 11:44:49 -0700 Subject: [PATCH 2/7] 0.9 sdk/e2e tests + updating 0.6 tests --- .../integration/linux-bubblewrap.test.ts | 135 +++++++++++++++++- .../tests/e2e_bubblewrap_characterization.rs | 126 +++++++++++++++- 2 files changed, 256 insertions(+), 5 deletions(-) diff --git a/sdk/node/tests/integration/linux-bubblewrap.test.ts b/sdk/node/tests/integration/linux-bubblewrap.test.ts index e1a0505f8..d3cd4f14a 100644 --- a/sdk/node/tests/integration/linux-bubblewrap.test.ts +++ b/sdk/node/tests/integration/linux-bubblewrap.test.ts @@ -86,10 +86,14 @@ describe(`Linux Bubblewrap (schema ${schemaVersion})`, { // Network proxy tests use the cooperative env-var proxy, which is // unprivileged by design -- the entire reason the proxy path exists is to // avoid the root requirement of iptables-based enforcement. Gate on -// "Linux + bwrap available" rather than "Linux + root". Pinned to schema -// 0.6.0-alpha because Bubblewrap proxy support is only available in 0.6+. -const PROXY_SCHEMA = '0.6.0-alpha'; -describe('Linux Bubblewrap network proxy (schema 0.6.0-alpha)', { +// "Linux + bwrap available" rather than "Linux + root". +// +// Pinned to 0.7 to hold the *legacy* proxy shape: `network.proxy`, +// `defaultPolicy` and `allowedHosts` were removed in 0.9, and below 0.8 they +// run on the shared host network, so this block needs no slirp4netns. The 0.9 +// spelling is covered separately below. +const PROXY_SCHEMA = '0.7.0-alpha'; +describe(`Linux Bubblewrap network proxy, legacy shape (schema ${PROXY_SCHEMA})`, { skip: !isLinuxBubblewrap ? 'Linux Bubblewrap proxy tests require Linux with bwrap installed' : undefined, @@ -178,6 +182,129 @@ describe('Linux Bubblewrap network proxy (schema 0.6.0-alpha)', { }); }); +// Schema 0.9 removed `network.proxy` along with `defaultPolicy` and the host +// lists, so the legacy block above has no 0.9 translation. A 0.9 proxy is a +// real endpoint named by `runtimeConfig.networkProxy`, which the parser accepts +// only on loopback, and the request resolves to the proxy-only posture: egress +// must be deny-by-default with no rules, and the backend opens the proxy +// endpoint alone. +// +// That posture is enforced from inside a private network namespace routed by +// rootless slirp4netns, which the legacy path does not need -- hence the extra +// prerequisite here. +const PROXY_SCHEMA_09 = '0.9.0-alpha'; +const hasSlirp4netns = (() => { + if (os.platform() !== 'linux') return false; + const pathDirs = (process.env.PATH ?? '').split(path.delimiter); + return pathDirs.some((dir) => { + if (!dir) return false; + try { + return fs.existsSync(path.join(dir, 'slirp4netns')); + } catch { + return false; + } + }); +})(); + +describe(`Linux Bubblewrap network proxy (schema ${PROXY_SCHEMA_09})`, { + skip: !isLinuxBubblewrap + ? 'Linux Bubblewrap proxy tests require Linux with bwrap installed' + : !hasSlirp4netns + ? 'the 0.9 proxy posture needs slirp4netns to route its private namespace' + : undefined, +}, () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'mxc-sdk-bwrap-proxy-09-')); + const proxies: ChildProcess[] = []; + + // The helper announces its port through a fixed-name ready file, so two + // proxies sharing a directory would have the second read the first one's + // port. Each gets its own directory instead. + const startProxy = (): number => { + const { port, proxyProcess } = startUnixTestProxy( + fs.mkdtempSync(path.join(tmpDir, 'proxy-')), + ); + proxies.push(proxyProcess); + return port; + }; + + after(() => { + for (const p of proxies) { + try { p.kill('SIGTERM'); } catch { /* ignore */ } + } + try { fs.rmSync(tmpDir, { recursive: true, force: true }); } catch { /* ignore */ } + }); + + it('should route traffic through the endpoint named by runtimeConfig.networkProxy', async () => { + const port = startProxy(); + + const config = sdk.createConfigFromPolicy( + { version: PROXY_SCHEMA_09 }, + 'bubblewrap', + 'bwrap-runtime-proxy-09', + ); + config.process!.commandLine = + `curl -fsSL '${NETWORK_TEST_URL}' > /dev/null && echo PROXY_09_OK`; + config.network = { + egress: { default: 'deny' }, + ingress: { default: 'deny', hostLoopback: 'deny' }, + }; + config.runtimeConfig = { + ...(config.runtimeConfig ?? {}), + networkProxy: `http://127.0.0.1:${port}`, + }; + + // No allowTestingFeatures: that flag gates `builtinTestServer`, which has + // no 0.9 spelling. Needing it here would mean the gate had gone slack. + const result = await spawnFromConfigAsync(config, { ...debugSpawnOptions, experimental: true }); + assert.strictEqual(result.exitCode, 0, `0.9 proxy run failed: ${result.stdout}`); + assert.ok(result.stdout.includes('PROXY_09_OK'), `missing PROXY_09_OK in: ${result.stdout}`); + }); + + it('should confine egress to the proxy endpoint', async () => { + const port = startProxy(); + + const config = sdk.createConfigFromPolicy( + { version: PROXY_SCHEMA_09 }, + 'bubblewrap', + 'bwrap-runtime-proxy-09-egress', + ); + // `--noproxy '*'` is the load-bearing part: it opts the request out of the + // proxy env vars, so a success would mean the sandbox reached the internet + // directly and the proxy-only posture was never enforced. + config.process!.commandLine = + 'set -e; ' + + `if curl -fsS --noproxy '*' --max-time 10 '${NETWORK_TEST_URL}' > /dev/null 2>&1; then ` + + ' echo DIRECT_09_LEAKED; exit 1; ' + + 'else ' + + ' echo DIRECT_09_BLOCKED_OK; ' + + 'fi; ' + + `curl -fsSL '${NETWORK_TEST_URL}' > /dev/null && echo PROXY_09_STILL_OK`; + config.network = { + egress: { default: 'deny' }, + ingress: { default: 'deny', hostLoopback: 'deny' }, + }; + config.runtimeConfig = { + ...(config.runtimeConfig ?? {}), + networkProxy: `http://127.0.0.1:${port}`, + }; + + const result = await spawnFromConfigAsync(config, { ...debugSpawnOptions, experimental: true }); + assert.strictEqual(result.exitCode, 0, `0.9 egress run failed: ${result.stdout}`); + assert.ok( + result.stdout.includes('DIRECT_09_BLOCKED_OK'), + `direct egress was not blocked: ${result.stdout}`, + ); + assert.ok( + result.stdout.includes('PROXY_09_STILL_OK'), + `the proxied request did not complete: ${result.stdout}`, + ); + assert.ok( + !result.stdout.includes('DIRECT_09_LEAKED'), + `proxy-only egress leaked: ${result.stdout}`, + ); + }); +}); + // The Rust serializer and the TypeScript parser are each unit-tested against // fixtures, but a fixture cannot catch the two drifting apart. This pins the // transport: the real `lxc-exec --available-backends` payload is fed to the diff --git a/src/testing/wxc_e2e_tests/tests/e2e_bubblewrap_characterization.rs b/src/testing/wxc_e2e_tests/tests/e2e_bubblewrap_characterization.rs index 9133cbe8a..f5b2df54e 100644 --- a/src/testing/wxc_e2e_tests/tests/e2e_bubblewrap_characterization.rs +++ b/src/testing/wxc_e2e_tests/tests/e2e_bubblewrap_characterization.rs @@ -17,11 +17,17 @@ #![cfg(target_os = "linux")] use serde_json::json; +use std::fs; +use std::path::PathBuf; use std::time::{Duration, Instant}; use wxc_e2e_tests::{has_bwrap, has_platform_exec, run_platform_config_value}; const SCHEMA_VERSION: &str = "0.7.0-alpha"; +/// The schema that introduced the default environment block and start-directory +/// normalization, so cases asserting either must name it explicitly. +const SCHEMA_VERSION_0_9: &str = "0.9.0-alpha"; + /// Whether the Bubblewrap characterization prerequisites are present. fn ready() -> bool { has_platform_exec() && has_bwrap() @@ -30,13 +36,30 @@ fn ready() -> bool { /// Build a one-shot config that omits `containment` so the binary selects its /// OS-native backend (Bubblewrap on Linux). fn config(label: &str, command_line: &str) -> serde_json::Value { + config_at(SCHEMA_VERSION, label, command_line) +} + +/// [`config`] against an explicit schema version. +fn config_at(version: &str, label: &str, command_line: &str) -> serde_json::Value { json!({ - "version": SCHEMA_VERSION, + "version": version, "containerId": format!("char-bwrap-{label}"), "process": { "commandLine": command_line } }) } +/// A private directory on the host, used as a policy grant or as the launching +/// process's own working directory. +fn unique_tempdir(tag: &str) -> PathBuf { + let nanos = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .expect("system clock is after the unix epoch") + .as_nanos(); + let dir = std::env::temp_dir().join(format!("mxc-char-bwrap-{tag}-{nanos}")); + fs::create_dir_all(&dir).expect("create temp dir"); + dir +} + #[test] fn bubblewrap_propagates_exit_code() { if !ready() { @@ -118,6 +141,107 @@ fn bubblewrap_applies_requested_env() { ); } +/// Locks in that a filesystem grant is **not** a working directory. +/// +/// Seatbelt and ProcessContainer both resolve an empty `process.cwd` to the +/// first read-write policy path. Bubblewrap deliberately does not: it emits +/// `--chdir` only for `process.cwd`, because a granted directory is somewhere +/// the child was permitted to go, not somewhere it was asked to start. The +/// divergence is easy to "fix" into conformity by mistake, so it is pinned +/// end-to-end here and not only in `bwrap_command.rs`. +#[test] +fn bubblewrap_does_not_adopt_a_policy_grant_as_the_working_directory() { + if !ready() { + return; + } + let grant = fs::canonicalize(unique_tempdir("cwd-grant")).expect("canonicalize grant"); + let mut cfg = config("cwd-grant", "pwd -P"); + cfg["filesystem"] = json!({ "readwritePaths": [grant.to_string_lossy()] }); + let result = run_platform_config_value("bwrap cwd grant", &cfg, &[], None); + let landed = result.stdout.trim().to_string(); + let _ = fs::remove_dir_all(&grant); + + assert_eq!( + result.code, + Some(0), + "run failed:\n{}", + result.combined_output() + ); + assert_ne!( + landed, + grant.to_string_lossy(), + "a read-write grant must not become the child's working directory" + ); +} + +/// Locks in that from schema 0.9 a relative `process.cwd` is anchored to the +/// sandbox root rather than resolving against whatever directory `bwrap` +/// carried into the namespace. +/// +/// `tmp` is the probe because the backend always mounts a `--tmpfs /tmp`, so +/// `/tmp` is guaranteed to exist inside the sandbox while the launching +/// process's own directory is not. Anchoring is what keeps `HOME` and +/// `--chdir` naming the same directory, so both are asserted together. +#[test] +fn bubblewrap_anchors_a_relative_process_cwd_from_0_9() { + if !ready() { + return; + } + // Launched from a directory that is *not* the filesystem root, so an + // unanchored `tmp` could not coincidentally resolve to `/tmp`. + let launch = fs::canonicalize(unique_tempdir("cwd-relative")).expect("canonicalize launch"); + let mut cfg = config_at( + SCHEMA_VERSION_0_9, + "cwd-relative", + "printf 'PWD=[%s] HOME=[%s]\\n' \"$(pwd -P)\" \"$HOME\"", + ); + cfg["process"]["cwd"] = json!("tmp"); + let result = run_platform_config_value("bwrap cwd relative", &cfg, &[], Some(launch.as_path())); + let _ = fs::remove_dir_all(&launch); + + let out = result.combined_output(); + assert_eq!(result.code, Some(0), "run failed:\n{out}"); + assert!( + out.contains("PWD=[/tmp]"), + "a relative process.cwd should anchor to the sandbox root. Output:\n{out}" + ); + assert!( + out.contains("HOME=[/tmp]"), + "HOME must name the directory the child actually started in. Output:\n{out}" + ); +} + +/// Locks in that a command which does not exist fails the run instead of +/// reporting success or hanging. +/// +/// The sandbox is torn down on the same path as a normal exit, so a shell that +/// never execs anything must still release the run. A hang here would show up +/// as the harness blocking rather than as a wrong exit code, which is why the +/// elapsed time is bounded too. +#[test] +fn bubblewrap_reports_a_missing_command() { + if !ready() { + return; + } + const MAX_ELAPSED: Duration = Duration::from_secs(30); + + let cfg = config("missing-command", "mxc-char-definitely-not-a-real-binary"); + let started = Instant::now(); + let result = run_platform_config_value("bwrap missing command", &cfg, &[], None); + let elapsed = started.elapsed(); + let out = result.combined_output(); + + assert_ne!( + result.code, + Some(0), + "a missing command should fail the run. Output:\n{out}" + ); + assert!( + elapsed < MAX_ELAPSED, + "a missing command should fail promptly; took {elapsed:?}" + ); +} + /// Locks in that an explicit `process.cwd` is honored (Bubblewrap emits /// `--chdir` for a non-empty working directory). `/` always exists inside the /// sandbox, so it is a stable target. From 3ba1470a01d368736f8e7ccd1d3160504cbab7bf Mon Sep 17 00:00:00 2001 From: Elliot Michlin <31219104+theelliotm@users.noreply.github.com> Date: Fri, 25 Sep 2026 11:53:14 -0700 Subject: [PATCH 3/7] Schema doc fix to reflect bubblewrap's cwd difference --- docs/schema.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/schema.md b/docs/schema.md index 51992753a..d717b80a1 100644 --- a/docs/schema.md +++ b/docs/schema.md @@ -282,6 +282,7 @@ use: |---------|----------------------------------------| | Windows ProcessContainer (AppContainer / BaseContainer) | First `readwritePaths` entry that is an existing directory, else the first such `readonlyPaths` entry, else the system drive root (`%SystemDrive%\`). Never `NULL`. | | Seatbelt (macOS) | Same precedence, with `~` expanded as the profile expands it; falls back to `/`. | +| Bubblewrap (Linux) | No substitution — a policy grant is never adopted. `--chdir` is emitted only for an explicit `process.cwd`, and `HOME` is left unset — see [`docs/bwrap-support/bubblewrap-backend.md`](bwrap-support/bubblewrap-backend.md). | | LXC / WSL Container | The container root — see [`docs/lxc-support/lxc-backend.md`](lxc-support/lxc-backend.md). | | MicroVM (NanVix) / Hyperlight | Not applicable — these backends reject a working directory outright. | From a057a658aca46ebe9a2c3b2622ad085234f4e71f Mon Sep 17 00:00:00 2001 From: Elliot Michlin <31219104+theelliotm@users.noreply.github.com> Date: Fri, 25 Sep 2026 13:47:37 -0700 Subject: [PATCH 4/7] Timeout test now actually checks for timeout --- src/testing/wxc_e2e_tests/src/lib.rs | 1631 +++++++++-------- .../tests/e2e_bubblewrap_characterization.rs | 34 +- 2 files changed, 878 insertions(+), 787 deletions(-) diff --git a/src/testing/wxc_e2e_tests/src/lib.rs b/src/testing/wxc_e2e_tests/src/lib.rs index b7d647fb1..3e3af1c03 100644 --- a/src/testing/wxc_e2e_tests/src/lib.rs +++ b/src/testing/wxc_e2e_tests/src/lib.rs @@ -1,773 +1,858 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -//! Shared helpers for MXC end-to-end integration tests. -//! -//! Tests live in `tests/e2e_windows.rs` and invoke MXC executables directly so -//! failures can be debugged from Rust test code. - -use std::fs; -use std::path::{Path, PathBuf}; -use std::process::{Command, Output}; -use std::time::{Duration, Instant}; - -use base64::{engine::general_purpose::STANDARD, Engine}; - -// --------------------------------------------------------------------------- -// Path helpers -// --------------------------------------------------------------------------- - -/// Locate the repository root. -/// `CARGO_MANIFEST_DIR` points to `src/testing/wxc_e2e_tests/` during `cargo test`. -pub fn repo_root() -> PathBuf { - let manifest_dir = PathBuf::from(env!("CARGO_MANIFEST_DIR")); - manifest_dir - .parent() // src/testing - .and_then(|p| p.parent()) // src - .and_then(|p| p.parent()) // repo root - .expect("could not determine repo root") - .to_path_buf() -} - -/// Return the repository `tests/configs/` directory. -pub fn test_configs_dir() -> PathBuf { - repo_root().join("tests").join("configs") -} - -/// Return the repository `tests/examples/` directory. -pub fn examples_dir() -> PathBuf { - repo_root().join("tests").join("examples") -} - -fn src_dir() -> PathBuf { - PathBuf::from(env!("CARGO_MANIFEST_DIR")) - .parent() // src/testing - .and_then(|p| p.parent()) // src - .expect("could not find src/") - .to_path_buf() -} - -// --------------------------------------------------------------------------- -// Build-mode detection -// --------------------------------------------------------------------------- - -/// Whether the test binary was compiled in release mode. -pub fn is_release_mode() -> bool { - !cfg!(debug_assertions) -} - -/// The target triple for the current platform (used for cross-compiled paths). -fn current_triple() -> &'static str { - if cfg!(all(target_os = "windows", target_arch = "x86_64")) { - "x86_64-pc-windows-msvc" - } else if cfg!(all(target_os = "windows", target_arch = "aarch64")) { - "aarch64-pc-windows-msvc" - } else if cfg!(all(target_os = "linux", target_arch = "x86_64")) { - "x86_64-unknown-linux-gnu" - } else if cfg!(all(target_os = "linux", target_arch = "aarch64")) { - "aarch64-unknown-linux-gnu" - } else if cfg!(all(target_os = "macos", target_arch = "aarch64")) { - "aarch64-apple-darwin" - } else if cfg!(all(target_os = "macos", target_arch = "x86_64")) { - "x86_64-apple-darwin" - } else { - "" - } -} - -/// Search for a binary in the target directory, checking multiple locations. -/// -/// Two profile directories may both contain the binary on a given dev host — -/// the triple-prefixed `target///` from explicit-target -/// builds, and the plain `target//` that `cargo build` / -/// `cargo test` write by default. Returning the *most recently modified* -/// candidate keeps tests aligned with the build the user just ran, instead -/// of latching onto a stale triple-prefixed copy from a previous session. -/// Profile preference (debug-vs-release) only resolves ties when neither -/// candidate has been built more recently than the other. -pub fn find_binary(name: &str) -> Option { - let src = src_dir(); - let (primary, fallback) = if is_release_mode() { - ("release", "debug") - } else { - ("debug", "release") - }; - let triple = current_triple(); - - let mut candidates = Vec::new(); - if !triple.is_empty() { - candidates.push(src.join("target").join(triple).join(primary).join(name)); - } - candidates.push(src.join("target").join(primary).join(name)); - if !triple.is_empty() { - candidates.push(src.join("target").join(triple).join(fallback).join(name)); - } - candidates.push(src.join("target").join(fallback).join(name)); - - // Pick the most recently modified candidate that exists. Falls back to - // existence-order when mtimes are unavailable (read errors). - candidates - .into_iter() - .filter(|p| p.exists()) - .max_by_key(|p| std::fs::metadata(p).and_then(|m| m.modified()).ok()) -} - -// --------------------------------------------------------------------------- -// Prerequisite checks (has_* pattern — returns true when present) -// --------------------------------------------------------------------------- - -/// Return whether `wxc-exec.exe` is available for direct E2E execution. -pub fn has_wxc_exe() -> bool { - match find_binary("wxc-exec.exe") { - Some(p) => { - println!("Using wxc-exec.exe at {}", p.display()); - true - } - None => { - println!("SKIPPED: wxc-exec.exe not found — build first"); - false - } - } -} - -/// Return whether `wxc-test-driver.exe` is available for direct E2E execution. -pub fn has_test_driver() -> bool { - match find_binary("wxc-test-driver.exe") { - Some(p) => { - println!("Using wxc-test-driver.exe at {}", p.display()); - true - } - None => { - println!("SKIPPED: wxc-test-driver.exe not found — build first"); - false - } - } -} - -/// Return whether the Windows Sandbox daemon binary is available. -pub fn has_daemon() -> bool { - match find_binary("wxc-windows-sandbox-daemon.exe") { - Some(p) => { - println!("Using daemon at {}", p.display()); - true - } - None => { - println!("SKIPPED: wxc-windows-sandbox-daemon.exe not found — build first"); - false - } - } -} - -/// Return whether the NanVix runtime binaries are available next to wxc-exec. -pub fn has_nanvix_binaries() -> bool { - let Some(exe) = find_binary("wxc-exec.exe") else { - return false; - }; - let exe_dir = exe.parent().unwrap_or(Path::new(".")); - // Flat binaries staged next to wxc-exec.exe by `nanvix_binaries`. - let flat_present = ["nanvixd.exe", "nanvix_rootfs.img", "python3.initrd"] - .iter() - .all(|name| exe_dir.join(name).exists()); - // Kernel binary now lives under `bin/` (nanvixd locates it via -bin-dir). - let bin_present = ["kernel.elf"] - .iter() - .all(|name| exe_dir.join("bin").join(name).exists()); - let present = flat_present && bin_present; - if !present { - println!("SKIPPED: NanVix binaries not found next to wxc-exec.exe"); - } - present -} - -/// Return whether `lxc-exec` is available for direct E2E execution. -pub fn has_lxc_exe() -> bool { - match find_binary("lxc-exec") { - Some(p) => { - println!("Using lxc-exec at {}", p.display()); - true - } - None => { - println!("SKIPPED: lxc-exec not found — build with `cargo build -p lxc --features microvm` first"); - false - } - } -} - -/// Return whether this host can start a system container. -/// -/// [`has_lxc_exe`] is not enough: the Linux build lane builds the binary and -/// never installs LXC. -/// -/// A lane provisioned to run these tests reports a skip as success, so the gate -/// would go green having tested nothing. `MXC_LXC_TESTS_REQUIRE_EXECUTION` is -/// the same switch the shell suite reads. -pub fn has_lxc_host() -> bool { - match Command::new("lxc-start").arg("--version").output() { - Ok(output) if output.status.success() => true, - _ => { - let reason = "lxc-start not installed — this host cannot start a system container"; - if std::env::var("MXC_LXC_TESTS_REQUIRE_EXECUTION").is_ok_and(|value| value != "0") { - panic!("strict mode: {reason}"); - } - println!("SKIPPED: {reason}"); - false - } - } -} - -/// Return whether the NanVix runtime binaries are available next to lxc-exec (Linux). -pub fn has_lxc_nanvix_binaries() -> bool { - let Some(exe) = find_binary("lxc-exec") else { - return false; - }; - let exe_dir = exe.parent().unwrap_or(Path::new(".")); - // Flat binaries staged next to lxc-exec by `nanvix_binaries`. - let flat_present = ["nanvixd.elf", "nanvix_rootfs.img", "python3.initrd"] - .iter() - .all(|name| exe_dir.join(name).exists()); - // Kernel binary under `bin/` (nanvixd locates it relative to cwd). - let bin_present = ["kernel.elf"] - .iter() - .all(|name| exe_dir.join("bin").join(name).exists()); - let present = flat_present && bin_present; - if !present { - println!("SKIPPED: NanVix binaries not found next to lxc-exec — build with `cargo build -p lxc --features microvm`"); - } - present -} - -/// Return whether `/dev/kvm` is available for KVM-based execution. -pub fn has_kvm() -> bool { - let available = Path::new("/dev/kvm").exists(); - if !available { - println!("SKIPPED: /dev/kvm not available — KVM required for NanVix on Linux"); - } - available -} - -/// Run `lxc-exec` with the supplied config file and extra arguments. -pub fn run_lxc_config(config_file: &str, extra_args: &[&str]) -> CommandResult { - let exe = find_binary("lxc-exec").expect("lxc-exec should be available"); - let config = test_configs_dir().join(config_file); - let mut args: Vec = extra_args.iter().map(|arg| (*arg).to_string()).collect(); - args.push(config.display().to_string()); - - run_executable(config_file, &exe, args) -} - -/// The Hyperlight image home the runner resolves first: `MXC_HYPERLIGHT_HOME` -/// when set, else `%LOCALAPPDATA%\mxc-hyperlight`. -fn hyperlight_home() -> PathBuf { - if let Some(home) = std::env::var_os("MXC_HYPERLIGHT_HOME") { - return PathBuf::from(home); - } - std::env::var_os("LOCALAPPDATA") - .map(PathBuf::from) - .unwrap_or_else(|| { - std::env::var_os("USERPROFILE") - .map(|v| PathBuf::from(v).join("AppData").join("Local")) - .unwrap_or_default() - }) - .join("mxc-hyperlight") -} - -/// Return whether `runtime`'s snapshot is installed in the default home -/// (`%LOCALAPPDATA%\mxc-hyperlight\\snapshot\index.json`). -pub fn has_hyperlight_runtime(runtime: &str) -> bool { - hyperlight_home() - .join(runtime) - .join("snapshot") - .join("index.json") - .is_file() -} - -/// Return whether the default Hyperlight runtime's snapshot is installed -/// (`%LOCALAPPDATA%\mxc-hyperlight\agent\snapshot\index.json`). -pub fn has_hyperlight_snapshot() -> bool { - let snapshot = hyperlight_home() - .join("agent") - .join("snapshot") - .join("index.json"); - if snapshot.is_file() { - println!("Using Hyperlight snapshot at {}", snapshot.display()); - true - } else { - println!( - "SKIPPED: Hyperlight snapshot not found at {} — run --setup-hyperlight first", - snapshot.display() - ); - false - } -} - -/// Return whether the Windows Sandbox optional feature is enabled. -pub fn has_windows_sandbox_feature() -> bool { - let available = Command::new("dism") - .args([ - "/online", - "/get-featureinfo", - "/featurename:Containers-DisposableClientVM", - ]) - .output() - .map(|output| { - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - output.status.success() - && format!("{stdout}\n{stderr}") - .lines() - .any(|line| line.contains("State") && line.contains("Enabled")) - }) - .unwrap_or(false); - - if !available { - println!("SKIPPED: Windows Sandbox feature is not enabled"); - } - - available -} - -/// Check whether `python.exe` is available and the *first* match in PATH -/// is NOT a Windows Store App Execution Alias. Store aliases are reparse -/// points under `WindowsApps` that cannot be launched inside -/// AppContainer/BaseContainer sandboxes. Even when a real Python exists -/// later in PATH, the sandbox will try to launch the first match and fail. -/// -/// Panics with a clear remediation message when Python is missing or -/// the first PATH match is a Store alias. -pub fn assert_python() { - let output = Command::new("where.exe").arg("python.exe").output().ok(); - - let Some(output) = output else { - panic!( - "python.exe not found.\n\ - E2E tests require a system-wide Python install.\n\ - Fix: Run scripts\\setup-test-prereqs.ps1 (elevated) or install Python system-wide \ - (winget install Python.Python.3.12 --scope machine)" - ); - }; - - if !output.status.success() { - panic!( - "python.exe not found.\n\ - E2E tests require a system-wide Python install.\n\ - Fix: Run scripts\\setup-test-prereqs.ps1 (elevated) or install Python system-wide \ - (winget install Python.Python.3.12 --scope machine)" - ); - } - - let stdout = String::from_utf8_lossy(&output.stdout); - let first_path = stdout.lines().next().unwrap_or(""); - if first_path.to_ascii_lowercase().contains("windowsapps") { - panic!( - "python.exe first resolves to a Windows Store alias ({first_path}).\n\ - Store aliases shadow real installs and cannot be launched inside sandbox containers.\n\ - Fix: Run scripts\\setup-test-prereqs.ps1 (elevated) or disable App Execution Aliases for Python" - ); - } -} - -/// The hardcoded path used by `processcontainer_pwsh_setlocation.json`. -const PWSH_PATH: &str = r"C:\Program Files\PowerShell\7\pwsh.exe"; - -/// Check whether PowerShell 7 is available at the expected path. -/// The test config `processcontainer_pwsh_setlocation.json` uses a hardcoded fully-qualified -/// path, so we validate that specific path exists rather than relying on -/// PATH resolution. -/// -/// Panics with a clear remediation message when pwsh is missing. -pub fn assert_pwsh() { - if !std::path::Path::new(PWSH_PATH).exists() { - panic!( - "PowerShell 7 not found at {PWSH_PATH}.\n\ - The pwsh_setlocation test requires PowerShell 7 installed at this path.\n\ - Fix: Run scripts\\setup-test-prereqs.ps1 (elevated) or install PowerShell 7 system-wide" - ); - } -} - -// --------------------------------------------------------------------------- -// Direct process execution -// --------------------------------------------------------------------------- - -/// Captured process result with decoded text output. -#[derive(Debug)] -pub struct CommandResult { - /// Human-readable command label. - pub label: String, - /// Process exit code, or `None` if the process terminated without one. - pub code: Option, - /// Captured stdout as UTF-8 lossy text. - pub stdout: String, - /// Captured stderr as UTF-8 lossy text. - pub stderr: String, - /// Wall-clock process duration in milliseconds. - pub wall_time_ms: u128, -} - -impl CommandResult { - /// Combine stdout and stderr. - pub fn combined_output(&self) -> String { - format!("{}\n{}", self.stdout, self.stderr) - } - - /// Combine stdout, stderr, and any base64-encoded text lines found in them. - pub fn combined_output_with_decoded_base64(&self) -> String { - let combined = self.combined_output(); - let mut decoded = Vec::new(); - - for line in combined - .lines() - .map(str::trim) - .filter(|line| !line.is_empty()) - { - if !looks_like_base64_text(line) { - continue; - } - - let Ok(bytes) = STANDARD.decode(line) else { - continue; - }; - let Ok(text) = String::from_utf8(bytes) else { - continue; - }; - decoded.push(text); - } - - if decoded.is_empty() { - combined - } else { - format!("{combined}\n{}", decoded.join("\n")) - } - } - - /// Return whether the command failed because the local test environment is - /// missing a runtime that the sandboxed process needs to launch. - pub fn is_missing_process_prerequisite(&self) -> bool { - let combined = self.combined_output(); - combined.contains("CreateProcessW failed: The system cannot find the file specified") - || combined.contains("Unsupported Windows branch or build version") - } -} - -fn is_base64_byte(byte: u8) -> bool { - matches!(byte, b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'+' | b'/' | b'=') -} - -fn looks_like_base64_text(line: &str) -> bool { - line.len() >= 16 && line.len().is_multiple_of(4) && line.bytes().all(is_base64_byte) -} - -/// Run an executable and capture its result. -pub fn run_executable(label: &str, exe: &Path, args: I) -> CommandResult -where - I: IntoIterator, - S: AsRef, -{ - let start = Instant::now(); - let output = Command::new(exe) - .args(args) - .output() - .unwrap_or_else(|error| panic!("failed to execute {label}: {error}")); - - command_result(label, output, start.elapsed().as_millis()) -} - -fn command_result(label: &str, output: Output, wall_time_ms: u128) -> CommandResult { - CommandResult { - label: label.to_string(), - code: output.status.code(), - stdout: String::from_utf8_lossy(&output.stdout).into_owned(), - stderr: String::from_utf8_lossy(&output.stderr).into_owned(), - wall_time_ms, - } -} - -/// Run `wxc-exec.exe` with a config file from `tests/configs/` and extra arguments. -pub fn run_wxc_config(config_file: &str, extra_args: &[&str]) -> CommandResult { - let exe = find_binary("wxc-exec.exe").expect("wxc-exec.exe should be available"); - let config = test_configs_dir().join(config_file); - let mut args: Vec = extra_args.iter().map(|arg| (*arg).to_string()).collect(); - args.push(config.display().to_string()); - - run_executable(config_file, &exe, args) -} - -/// Run `wxc-exec.exe` with a config file from `tests/examples/` and extra arguments. -pub fn run_wxc_example(config_file: &str, extra_args: &[&str]) -> CommandResult { - let exe = find_binary("wxc-exec.exe").expect("wxc-exec.exe should be available"); - let config = examples_dir().join(config_file); - let mut args: Vec = extra_args.iter().map(|arg| (*arg).to_string()).collect(); - args.push(config.display().to_string()); - - run_executable(config_file, &exe, args) -} - -/// Run `wxc-exec.exe` with a state-aware request envelope. The JSON value is -/// serialised, base64-encoded, and passed via `--config-base64`. Used by the -/// state-aware smoke tests. -pub fn run_wxc_state_aware( - label: &str, - request: &serde_json::Value, - extra_args: &[&str], -) -> CommandResult { - let exe = find_binary("wxc-exec.exe").expect("wxc-exec.exe should be available"); - let json = request.to_string(); - let encoded = STANDARD.encode(json.as_bytes()); - - let mut args: Vec = extra_args.iter().map(|s| (*s).to_string()).collect(); - args.push("--config-base64".to_string()); - args.push(encoded); - - run_executable(label, &exe, args) -} - -/// Run `wxc-exec.exe` with a one-shot config supplied as an in-memory JSON -/// value. The value is serialised, base64-encoded, and passed via -/// `--config-base64`, so callers can build configs with values only known at -/// test time (e.g. a dynamically discovered host IP or port). -pub fn run_wxc_config_value( - label: &str, - config: &serde_json::Value, - extra_args: &[&str], -) -> CommandResult { - let exe = find_binary("wxc-exec.exe").expect("wxc-exec.exe should be available"); - let encoded = STANDARD.encode(config.to_string().as_bytes()); - - let mut args: Vec = extra_args.iter().map(|s| (*s).to_string()).collect(); - args.push("--config-base64".to_string()); - args.push(encoded); - - run_executable(label, &exe, args) -} - -// --------------------------------------------------------------------------- -// Cross-platform executor characterization helpers -// -// These drive the *native* one-shot executor binary for the current OS -// (`mxc-exec-mac` on macOS, `lxc-exec` on Linux, `wxc-exec.exe` on Windows) -// with an in-memory config, optionally setting the child process's environment -// and working directory. They exist to lock in the current run-to-completion -// behavior (exit code, stdout, env/cwd inheritance, timeout) before the -// unified `SandboxBackend`/`Runner` refactor lands. -// --------------------------------------------------------------------------- - -/// The native one-shot executor binary name for the current platform. -pub fn platform_exec_binary_name() -> &'static str { - if cfg!(target_os = "windows") { - "wxc-exec.exe" - } else if cfg!(target_os = "macos") { - "mxc-exec-mac" - } else { - "lxc-exec" - } -} - -/// Locate the native one-shot executor binary for the current platform. -pub fn find_platform_exec() -> Option { - find_binary(platform_exec_binary_name()) -} - -/// Whether the native executor binary for this platform is available. -pub fn has_platform_exec() -> bool { - match find_platform_exec() { - Some(p) => { - println!("Using {} at {}", platform_exec_binary_name(), p.display()); - true - } - None => { - println!( - "SKIPPED: {} not found — build the native executor first", - platform_exec_binary_name() - ); - false - } - } -} - -/// Whether `bwrap` (Bubblewrap) is installed, runnable, and new enough on this -/// Linux host. Bubblewrap characterization tests skip cleanly otherwise (e.g. a -/// CI runner without `bubblewrap` installed, or one shipping a release older -/// than [`bwrap_common::bwrap_version::MIN_BWRAP_VERSION`]) — the backend -/// rejects such hosts up front, so the tests would have nothing to exercise. -pub fn has_bwrap() -> bool { - #[cfg(target_os = "linux")] - { - match bwrap_common::bwrap_version::probe_bwrap() { - Ok(_) => true, - Err(err) => { - println!("SKIPPED: {err}"); - false - } - } - } - - #[cfg(not(target_os = "linux"))] - { - println!("SKIPPED: Bubblewrap is only available on Linux"); - false - } -} - -/// Opt-in switch for the Windows ProcessContainer characterization tests. -/// -/// AppContainer/BaseContainer execution requires an elevated, host-prepped -/// Windows host (see `docs/host-prep.md`). Standard CI runners are NOT capable, -/// so these tests are skipped unless a host-prepped lane explicitly sets -/// `MXC_E2E_HOST_PREPPED=1`. This keeps them from ever red-failing on incapable -/// CI while still being runnable on a prepared box. -pub fn host_prepped_optin() -> bool { - let enabled = std::env::var("MXC_E2E_HOST_PREPPED").as_deref() == Ok("1"); - if !enabled { - println!( - "SKIPPED: ProcessContainer characterization requires a host-prepped Windows host; \ - set MXC_E2E_HOST_PREPPED=1 on a prepared lane to enable" - ); - } - enabled -} - -/// Run the current platform's native executor binary with an in-memory config -/// value (serialised + base64-encoded via `--config-base64`), optionally -/// setting environment variables and a working directory on the *executor* -/// process. `extra_env`/`cwd` are how the inheritance characterization tests -/// observe whether the sandboxed child picks up the launcher's env/cwd. -pub fn run_platform_config_value( - label: &str, - config: &serde_json::Value, - extra_env: &[(&str, &str)], - cwd: Option<&Path>, -) -> CommandResult { - let exe = find_platform_exec().expect("native executor binary should be available"); - let encoded = STANDARD.encode(config.to_string().as_bytes()); - - let start = Instant::now(); - let mut cmd = Command::new(&exe); - cmd.arg("--config-base64").arg(encoded); - for (key, value) in extra_env { - cmd.env(key, value); - } - if let Some(dir) = cwd { - cmd.current_dir(dir); - } - let output = cmd - .output() - .unwrap_or_else(|error| panic!("failed to execute {label}: {error}")); - - command_result(label, output, start.elapsed().as_millis()) -} - -/// Run `wxc-test-driver.exe` against a directory or a single config file. -pub fn run_test_driver(target: &Path, extra_args: &[&str]) -> CommandResult { - let exe = find_binary("wxc-test-driver.exe").expect("wxc-test-driver.exe should be available"); - let mut args = vec![target.display().to_string()]; - args.extend(extra_args.iter().map(|arg| (*arg).to_string())); - - run_executable(&format!("wxc-test-driver {}", target.display()), &exe, args) -} - -/// Assert that a command exited successfully. -pub fn assert_success(result: &CommandResult) { - assert_exit(result, 0, None); -} - -/// Assert success, or skip when the local machine lacks sandbox runtime prerequisites. -pub fn assert_success_or_skip_missing_prerequisite(result: &CommandResult) { - if result.is_missing_process_prerequisite() { - println!( - "SKIPPED: {} requires local sandbox runtime prerequisites not available here", - result.label - ); - return; - } - - assert_success(result); -} - -/// Assert that a command exited with the expected code and optional output. -pub fn assert_exit(result: &CommandResult, expected_exit: i32, output_contains: Option<&str>) { - if result.code != Some(expected_exit) { - panic!( - "{} failed: expected exit {}, got {:?}\n--- stdout ---\n{}\n--- stderr ---\n{}", - result.label, expected_exit, result.code, result.stdout, result.stderr - ); - } - - if let Some(expected) = output_contains { - let combined = result.combined_output_with_decoded_base64(); - if !combined.contains(expected) { - panic!( - "{} failed: output missing '{}'\n--- combined output ---\n{}", - result.label, expected, combined - ); - } - } -} - -// --------------------------------------------------------------------------- -// Temporary filesystem setup -// --------------------------------------------------------------------------- - -/// Temporary directories removed when the guard is dropped. -#[derive(Debug)] -pub struct TempDirs { - paths: Vec, -} - -impl TempDirs { - /// Create temporary directories, removing any stale versions first. - pub fn create(paths: &[&str]) -> Self { - let paths: Vec = paths.iter().map(PathBuf::from).collect(); - for path in &paths { - remove_dir_all_if_exists(path); - fs::create_dir_all(path).unwrap_or_else(|error| { - panic!("failed to create temp dir {}: {error}", path.display()) - }); - } - Self { paths } - } - - /// Write a UTF-8 text file to an absolute path. - /// - /// The file is cleaned up when its parent directory is one of this guard's - /// tracked temporary directories. - pub fn write_absolute_file(&self, absolute_path: &str, contents: &str) { - let path = PathBuf::from(absolute_path); - assert!( - path.is_absolute(), - "temporary test file path must be absolute: {}", - path.display() - ); - if let Some(parent) = path.parent() { - fs::create_dir_all(parent).unwrap_or_else(|error| { - panic!("failed to create parent dir {}: {error}", parent.display()) - }); - } - fs::write(&path, contents) - .unwrap_or_else(|error| panic!("failed to write {}: {error}", path.display())); - } -} - -impl Drop for TempDirs { - fn drop(&mut self) { - for path in &self.paths { - remove_dir_all_if_exists(path); - } - } -} - -fn remove_dir_all_if_exists(path: &Path) { - if !path.exists() { - return; - } - - if fs::remove_dir_all(path).is_ok() { - return; - } - - std::thread::sleep(Duration::from_millis(100)); - if path.exists() { - fs::remove_dir_all(path).unwrap_or_else(|error| { - panic!("failed to remove temp dir {}: {error}", path.display()) - }); - } -} +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Shared helpers for MXC end-to-end integration tests. +//! +//! Tests live in `tests/e2e_windows.rs` and invoke MXC executables directly so +//! failures can be debugged from Rust test code. + +use std::fs; +use std::io::Read; +use std::path::{Path, PathBuf}; +use std::process::{Command, Output, Stdio}; +use std::time::{Duration, Instant}; + +use base64::{engine::general_purpose::STANDARD, Engine}; + +// --------------------------------------------------------------------------- +// Path helpers +// --------------------------------------------------------------------------- + +/// Locate the repository root. +/// `CARGO_MANIFEST_DIR` points to `src/testing/wxc_e2e_tests/` during `cargo test`. +pub fn repo_root() -> PathBuf { + let manifest_dir = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + manifest_dir + .parent() // src/testing + .and_then(|p| p.parent()) // src + .and_then(|p| p.parent()) // repo root + .expect("could not determine repo root") + .to_path_buf() +} + +/// Return the repository `tests/configs/` directory. +pub fn test_configs_dir() -> PathBuf { + repo_root().join("tests").join("configs") +} + +/// Return the repository `tests/examples/` directory. +pub fn examples_dir() -> PathBuf { + repo_root().join("tests").join("examples") +} + +fn src_dir() -> PathBuf { + PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .parent() // src/testing + .and_then(|p| p.parent()) // src + .expect("could not find src/") + .to_path_buf() +} + +// --------------------------------------------------------------------------- +// Build-mode detection +// --------------------------------------------------------------------------- + +/// Whether the test binary was compiled in release mode. +pub fn is_release_mode() -> bool { + !cfg!(debug_assertions) +} + +/// The target triple for the current platform (used for cross-compiled paths). +fn current_triple() -> &'static str { + if cfg!(all(target_os = "windows", target_arch = "x86_64")) { + "x86_64-pc-windows-msvc" + } else if cfg!(all(target_os = "windows", target_arch = "aarch64")) { + "aarch64-pc-windows-msvc" + } else if cfg!(all(target_os = "linux", target_arch = "x86_64")) { + "x86_64-unknown-linux-gnu" + } else if cfg!(all(target_os = "linux", target_arch = "aarch64")) { + "aarch64-unknown-linux-gnu" + } else if cfg!(all(target_os = "macos", target_arch = "aarch64")) { + "aarch64-apple-darwin" + } else if cfg!(all(target_os = "macos", target_arch = "x86_64")) { + "x86_64-apple-darwin" + } else { + "" + } +} + +/// Search for a binary in the target directory, checking multiple locations. +/// +/// Two profile directories may both contain the binary on a given dev host — +/// the triple-prefixed `target///` from explicit-target +/// builds, and the plain `target//` that `cargo build` / +/// `cargo test` write by default. Returning the *most recently modified* +/// candidate keeps tests aligned with the build the user just ran, instead +/// of latching onto a stale triple-prefixed copy from a previous session. +/// Profile preference (debug-vs-release) only resolves ties when neither +/// candidate has been built more recently than the other. +pub fn find_binary(name: &str) -> Option { + let src = src_dir(); + let (primary, fallback) = if is_release_mode() { + ("release", "debug") + } else { + ("debug", "release") + }; + let triple = current_triple(); + + let mut candidates = Vec::new(); + if !triple.is_empty() { + candidates.push(src.join("target").join(triple).join(primary).join(name)); + } + candidates.push(src.join("target").join(primary).join(name)); + if !triple.is_empty() { + candidates.push(src.join("target").join(triple).join(fallback).join(name)); + } + candidates.push(src.join("target").join(fallback).join(name)); + + // Pick the most recently modified candidate that exists. Falls back to + // existence-order when mtimes are unavailable (read errors). + candidates + .into_iter() + .filter(|p| p.exists()) + .max_by_key(|p| std::fs::metadata(p).and_then(|m| m.modified()).ok()) +} + +// --------------------------------------------------------------------------- +// Prerequisite checks (has_* pattern — returns true when present) +// --------------------------------------------------------------------------- + +/// Return whether `wxc-exec.exe` is available for direct E2E execution. +pub fn has_wxc_exe() -> bool { + match find_binary("wxc-exec.exe") { + Some(p) => { + println!("Using wxc-exec.exe at {}", p.display()); + true + } + None => { + println!("SKIPPED: wxc-exec.exe not found — build first"); + false + } + } +} + +/// Return whether `wxc-test-driver.exe` is available for direct E2E execution. +pub fn has_test_driver() -> bool { + match find_binary("wxc-test-driver.exe") { + Some(p) => { + println!("Using wxc-test-driver.exe at {}", p.display()); + true + } + None => { + println!("SKIPPED: wxc-test-driver.exe not found — build first"); + false + } + } +} + +/// Return whether the Windows Sandbox daemon binary is available. +pub fn has_daemon() -> bool { + match find_binary("wxc-windows-sandbox-daemon.exe") { + Some(p) => { + println!("Using daemon at {}", p.display()); + true + } + None => { + println!("SKIPPED: wxc-windows-sandbox-daemon.exe not found — build first"); + false + } + } +} + +/// Return whether the NanVix runtime binaries are available next to wxc-exec. +pub fn has_nanvix_binaries() -> bool { + let Some(exe) = find_binary("wxc-exec.exe") else { + return false; + }; + let exe_dir = exe.parent().unwrap_or(Path::new(".")); + // Flat binaries staged next to wxc-exec.exe by `nanvix_binaries`. + let flat_present = ["nanvixd.exe", "nanvix_rootfs.img", "python3.initrd"] + .iter() + .all(|name| exe_dir.join(name).exists()); + // Kernel binary now lives under `bin/` (nanvixd locates it via -bin-dir). + let bin_present = ["kernel.elf"] + .iter() + .all(|name| exe_dir.join("bin").join(name).exists()); + let present = flat_present && bin_present; + if !present { + println!("SKIPPED: NanVix binaries not found next to wxc-exec.exe"); + } + present +} + +/// Return whether `lxc-exec` is available for direct E2E execution. +pub fn has_lxc_exe() -> bool { + match find_binary("lxc-exec") { + Some(p) => { + println!("Using lxc-exec at {}", p.display()); + true + } + None => { + println!("SKIPPED: lxc-exec not found — build with `cargo build -p lxc --features microvm` first"); + false + } + } +} + +/// Return whether this host can start a system container. +/// +/// [`has_lxc_exe`] is not enough: the Linux build lane builds the binary and +/// never installs LXC. +/// +/// A lane provisioned to run these tests reports a skip as success, so the gate +/// would go green having tested nothing. `MXC_LXC_TESTS_REQUIRE_EXECUTION` is +/// the same switch the shell suite reads. +pub fn has_lxc_host() -> bool { + match Command::new("lxc-start").arg("--version").output() { + Ok(output) if output.status.success() => true, + _ => { + let reason = "lxc-start not installed — this host cannot start a system container"; + if std::env::var("MXC_LXC_TESTS_REQUIRE_EXECUTION").is_ok_and(|value| value != "0") { + panic!("strict mode: {reason}"); + } + println!("SKIPPED: {reason}"); + false + } + } +} + +/// Return whether the NanVix runtime binaries are available next to lxc-exec (Linux). +pub fn has_lxc_nanvix_binaries() -> bool { + let Some(exe) = find_binary("lxc-exec") else { + return false; + }; + let exe_dir = exe.parent().unwrap_or(Path::new(".")); + // Flat binaries staged next to lxc-exec by `nanvix_binaries`. + let flat_present = ["nanvixd.elf", "nanvix_rootfs.img", "python3.initrd"] + .iter() + .all(|name| exe_dir.join(name).exists()); + // Kernel binary under `bin/` (nanvixd locates it relative to cwd). + let bin_present = ["kernel.elf"] + .iter() + .all(|name| exe_dir.join("bin").join(name).exists()); + let present = flat_present && bin_present; + if !present { + println!("SKIPPED: NanVix binaries not found next to lxc-exec — build with `cargo build -p lxc --features microvm`"); + } + present +} + +/// Return whether `/dev/kvm` is available for KVM-based execution. +pub fn has_kvm() -> bool { + let available = Path::new("/dev/kvm").exists(); + if !available { + println!("SKIPPED: /dev/kvm not available — KVM required for NanVix on Linux"); + } + available +} + +/// Run `lxc-exec` with the supplied config file and extra arguments. +pub fn run_lxc_config(config_file: &str, extra_args: &[&str]) -> CommandResult { + let exe = find_binary("lxc-exec").expect("lxc-exec should be available"); + let config = test_configs_dir().join(config_file); + let mut args: Vec = extra_args.iter().map(|arg| (*arg).to_string()).collect(); + args.push(config.display().to_string()); + + run_executable(config_file, &exe, args) +} + +/// The Hyperlight image home the runner resolves first: `MXC_HYPERLIGHT_HOME` +/// when set, else `%LOCALAPPDATA%\mxc-hyperlight`. +fn hyperlight_home() -> PathBuf { + if let Some(home) = std::env::var_os("MXC_HYPERLIGHT_HOME") { + return PathBuf::from(home); + } + std::env::var_os("LOCALAPPDATA") + .map(PathBuf::from) + .unwrap_or_else(|| { + std::env::var_os("USERPROFILE") + .map(|v| PathBuf::from(v).join("AppData").join("Local")) + .unwrap_or_default() + }) + .join("mxc-hyperlight") +} + +/// Return whether `runtime`'s snapshot is installed in the default home +/// (`%LOCALAPPDATA%\mxc-hyperlight\\snapshot\index.json`). +pub fn has_hyperlight_runtime(runtime: &str) -> bool { + hyperlight_home() + .join(runtime) + .join("snapshot") + .join("index.json") + .is_file() +} + +/// Return whether the default Hyperlight runtime's snapshot is installed +/// (`%LOCALAPPDATA%\mxc-hyperlight\agent\snapshot\index.json`). +pub fn has_hyperlight_snapshot() -> bool { + let snapshot = hyperlight_home() + .join("agent") + .join("snapshot") + .join("index.json"); + if snapshot.is_file() { + println!("Using Hyperlight snapshot at {}", snapshot.display()); + true + } else { + println!( + "SKIPPED: Hyperlight snapshot not found at {} — run --setup-hyperlight first", + snapshot.display() + ); + false + } +} + +/// Return whether the Windows Sandbox optional feature is enabled. +pub fn has_windows_sandbox_feature() -> bool { + let available = Command::new("dism") + .args([ + "/online", + "/get-featureinfo", + "/featurename:Containers-DisposableClientVM", + ]) + .output() + .map(|output| { + let stdout = String::from_utf8_lossy(&output.stdout); + let stderr = String::from_utf8_lossy(&output.stderr); + output.status.success() + && format!("{stdout}\n{stderr}") + .lines() + .any(|line| line.contains("State") && line.contains("Enabled")) + }) + .unwrap_or(false); + + if !available { + println!("SKIPPED: Windows Sandbox feature is not enabled"); + } + + available +} + +/// Check whether `python.exe` is available and the *first* match in PATH +/// is NOT a Windows Store App Execution Alias. Store aliases are reparse +/// points under `WindowsApps` that cannot be launched inside +/// AppContainer/BaseContainer sandboxes. Even when a real Python exists +/// later in PATH, the sandbox will try to launch the first match and fail. +/// +/// Panics with a clear remediation message when Python is missing or +/// the first PATH match is a Store alias. +pub fn assert_python() { + let output = Command::new("where.exe").arg("python.exe").output().ok(); + + let Some(output) = output else { + panic!( + "python.exe not found.\n\ + E2E tests require a system-wide Python install.\n\ + Fix: Run scripts\\setup-test-prereqs.ps1 (elevated) or install Python system-wide \ + (winget install Python.Python.3.12 --scope machine)" + ); + }; + + if !output.status.success() { + panic!( + "python.exe not found.\n\ + E2E tests require a system-wide Python install.\n\ + Fix: Run scripts\\setup-test-prereqs.ps1 (elevated) or install Python system-wide \ + (winget install Python.Python.3.12 --scope machine)" + ); + } + + let stdout = String::from_utf8_lossy(&output.stdout); + let first_path = stdout.lines().next().unwrap_or(""); + if first_path.to_ascii_lowercase().contains("windowsapps") { + panic!( + "python.exe first resolves to a Windows Store alias ({first_path}).\n\ + Store aliases shadow real installs and cannot be launched inside sandbox containers.\n\ + Fix: Run scripts\\setup-test-prereqs.ps1 (elevated) or disable App Execution Aliases for Python" + ); + } +} + +/// The hardcoded path used by `processcontainer_pwsh_setlocation.json`. +const PWSH_PATH: &str = r"C:\Program Files\PowerShell\7\pwsh.exe"; + +/// Check whether PowerShell 7 is available at the expected path. +/// The test config `processcontainer_pwsh_setlocation.json` uses a hardcoded fully-qualified +/// path, so we validate that specific path exists rather than relying on +/// PATH resolution. +/// +/// Panics with a clear remediation message when pwsh is missing. +pub fn assert_pwsh() { + if !std::path::Path::new(PWSH_PATH).exists() { + panic!( + "PowerShell 7 not found at {PWSH_PATH}.\n\ + The pwsh_setlocation test requires PowerShell 7 installed at this path.\n\ + Fix: Run scripts\\setup-test-prereqs.ps1 (elevated) or install PowerShell 7 system-wide" + ); + } +} + +// --------------------------------------------------------------------------- +// Direct process execution +// --------------------------------------------------------------------------- + +/// Captured process result with decoded text output. +#[derive(Debug)] +pub struct CommandResult { + /// Human-readable command label. + pub label: String, + /// Process exit code, or `None` if the process terminated without one. + pub code: Option, + /// Captured stdout as UTF-8 lossy text. + pub stdout: String, + /// Captured stderr as UTF-8 lossy text. + pub stderr: String, + /// Wall-clock process duration in milliseconds. + pub wall_time_ms: u128, +} + +impl CommandResult { + /// Combine stdout and stderr. + pub fn combined_output(&self) -> String { + format!("{}\n{}", self.stdout, self.stderr) + } + + /// Combine stdout, stderr, and any base64-encoded text lines found in them. + pub fn combined_output_with_decoded_base64(&self) -> String { + let combined = self.combined_output(); + let mut decoded = Vec::new(); + + for line in combined + .lines() + .map(str::trim) + .filter(|line| !line.is_empty()) + { + if !looks_like_base64_text(line) { + continue; + } + + let Ok(bytes) = STANDARD.decode(line) else { + continue; + }; + let Ok(text) = String::from_utf8(bytes) else { + continue; + }; + decoded.push(text); + } + + if decoded.is_empty() { + combined + } else { + format!("{combined}\n{}", decoded.join("\n")) + } + } + + /// Return whether the command failed because the local test environment is + /// missing a runtime that the sandboxed process needs to launch. + pub fn is_missing_process_prerequisite(&self) -> bool { + let combined = self.combined_output(); + combined.contains("CreateProcessW failed: The system cannot find the file specified") + || combined.contains("Unsupported Windows branch or build version") + } +} + +fn is_base64_byte(byte: u8) -> bool { + matches!(byte, b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'+' | b'/' | b'=') +} + +fn looks_like_base64_text(line: &str) -> bool { + line.len() >= 16 && line.len().is_multiple_of(4) && line.bytes().all(is_base64_byte) +} + +/// Run an executable and capture its result. +pub fn run_executable(label: &str, exe: &Path, args: I) -> CommandResult +where + I: IntoIterator, + S: AsRef, +{ + let start = Instant::now(); + let output = Command::new(exe) + .args(args) + .output() + .unwrap_or_else(|error| panic!("failed to execute {label}: {error}")); + + command_result(label, output, start.elapsed().as_millis()) +} + +fn command_result(label: &str, output: Output, wall_time_ms: u128) -> CommandResult { + CommandResult { + label: label.to_string(), + code: output.status.code(), + stdout: String::from_utf8_lossy(&output.stdout).into_owned(), + stderr: String::from_utf8_lossy(&output.stderr).into_owned(), + wall_time_ms, + } +} + +/// Run `wxc-exec.exe` with a config file from `tests/configs/` and extra arguments. +pub fn run_wxc_config(config_file: &str, extra_args: &[&str]) -> CommandResult { + let exe = find_binary("wxc-exec.exe").expect("wxc-exec.exe should be available"); + let config = test_configs_dir().join(config_file); + let mut args: Vec = extra_args.iter().map(|arg| (*arg).to_string()).collect(); + args.push(config.display().to_string()); + + run_executable(config_file, &exe, args) +} + +/// Run `wxc-exec.exe` with a config file from `tests/examples/` and extra arguments. +pub fn run_wxc_example(config_file: &str, extra_args: &[&str]) -> CommandResult { + let exe = find_binary("wxc-exec.exe").expect("wxc-exec.exe should be available"); + let config = examples_dir().join(config_file); + let mut args: Vec = extra_args.iter().map(|arg| (*arg).to_string()).collect(); + args.push(config.display().to_string()); + + run_executable(config_file, &exe, args) +} + +/// Run `wxc-exec.exe` with a state-aware request envelope. The JSON value is +/// serialised, base64-encoded, and passed via `--config-base64`. Used by the +/// state-aware smoke tests. +pub fn run_wxc_state_aware( + label: &str, + request: &serde_json::Value, + extra_args: &[&str], +) -> CommandResult { + let exe = find_binary("wxc-exec.exe").expect("wxc-exec.exe should be available"); + let json = request.to_string(); + let encoded = STANDARD.encode(json.as_bytes()); + + let mut args: Vec = extra_args.iter().map(|s| (*s).to_string()).collect(); + args.push("--config-base64".to_string()); + args.push(encoded); + + run_executable(label, &exe, args) +} + +/// Run `wxc-exec.exe` with a one-shot config supplied as an in-memory JSON +/// value. The value is serialised, base64-encoded, and passed via +/// `--config-base64`, so callers can build configs with values only known at +/// test time (e.g. a dynamically discovered host IP or port). +pub fn run_wxc_config_value( + label: &str, + config: &serde_json::Value, + extra_args: &[&str], +) -> CommandResult { + let exe = find_binary("wxc-exec.exe").expect("wxc-exec.exe should be available"); + let encoded = STANDARD.encode(config.to_string().as_bytes()); + + let mut args: Vec = extra_args.iter().map(|s| (*s).to_string()).collect(); + args.push("--config-base64".to_string()); + args.push(encoded); + + run_executable(label, &exe, args) +} + +// --------------------------------------------------------------------------- +// Cross-platform executor characterization helpers +// +// These drive the *native* one-shot executor binary for the current OS +// (`mxc-exec-mac` on macOS, `lxc-exec` on Linux, `wxc-exec.exe` on Windows) +// with an in-memory config, optionally setting the child process's environment +// and working directory. They exist to lock in the current run-to-completion +// behavior (exit code, stdout, env/cwd inheritance, timeout) before the +// unified `SandboxBackend`/`Runner` refactor lands. +// --------------------------------------------------------------------------- + +/// The native one-shot executor binary name for the current platform. +pub fn platform_exec_binary_name() -> &'static str { + if cfg!(target_os = "windows") { + "wxc-exec.exe" + } else if cfg!(target_os = "macos") { + "mxc-exec-mac" + } else { + "lxc-exec" + } +} + +/// Locate the native one-shot executor binary for the current platform. +pub fn find_platform_exec() -> Option { + find_binary(platform_exec_binary_name()) +} + +/// Whether the native executor binary for this platform is available. +pub fn has_platform_exec() -> bool { + match find_platform_exec() { + Some(p) => { + println!("Using {} at {}", platform_exec_binary_name(), p.display()); + true + } + None => { + println!( + "SKIPPED: {} not found — build the native executor first", + platform_exec_binary_name() + ); + false + } + } +} + +/// Whether `bwrap` (Bubblewrap) is installed, runnable, and new enough on this +/// Linux host. Bubblewrap characterization tests skip cleanly otherwise (e.g. a +/// CI runner without `bubblewrap` installed, or one shipping a release older +/// than [`bwrap_common::bwrap_version::MIN_BWRAP_VERSION`]) — the backend +/// rejects such hosts up front, so the tests would have nothing to exercise. +pub fn has_bwrap() -> bool { + #[cfg(target_os = "linux")] + { + match bwrap_common::bwrap_version::probe_bwrap() { + Ok(_) => true, + Err(err) => { + println!("SKIPPED: {err}"); + false + } + } + } + + #[cfg(not(target_os = "linux"))] + { + println!("SKIPPED: Bubblewrap is only available on Linux"); + false + } +} + +/// Opt-in switch for the Windows ProcessContainer characterization tests. +/// +/// AppContainer/BaseContainer execution requires an elevated, host-prepped +/// Windows host (see `docs/host-prep.md`). Standard CI runners are NOT capable, +/// so these tests are skipped unless a host-prepped lane explicitly sets +/// `MXC_E2E_HOST_PREPPED=1`. This keeps them from ever red-failing on incapable +/// CI while still being runnable on a prepared box. +pub fn host_prepped_optin() -> bool { + let enabled = std::env::var("MXC_E2E_HOST_PREPPED").as_deref() == Ok("1"); + if !enabled { + println!( + "SKIPPED: ProcessContainer characterization requires a host-prepped Windows host; \ + set MXC_E2E_HOST_PREPPED=1 on a prepared lane to enable" + ); + } + enabled +} + +/// Run the current platform's native executor binary with an in-memory config +/// value (serialised + base64-encoded via `--config-base64`), optionally +/// setting environment variables and a working directory on the *executor* +/// process. `extra_env`/`cwd` are how the inheritance characterization tests +/// observe whether the sandboxed child picks up the launcher's env/cwd. +pub fn run_platform_config_value( + label: &str, + config: &serde_json::Value, + extra_env: &[(&str, &str)], + cwd: Option<&Path>, +) -> CommandResult { + let exe = find_platform_exec().expect("native executor binary should be available"); + let encoded = STANDARD.encode(config.to_string().as_bytes()); + + let start = Instant::now(); + let mut cmd = Command::new(&exe); + cmd.arg("--config-base64").arg(encoded); + for (key, value) in extra_env { + cmd.env(key, value); + } + if let Some(dir) = cwd { + cmd.current_dir(dir); + } + let output = cmd + .output() + .unwrap_or_else(|error| panic!("failed to execute {label}: {error}")); + + command_result(label, output, start.elapsed().as_millis()) +} + +/// [`run_platform_config_value`] bounded by a host-side deadline, returning +/// `None` if it expired. Use this wherever termination is the +/// property under test. +/// +/// On expiry the executor is killed and reaped. Its sandboxed descendants are +/// not walked: that is the backend's teardown contract, and a test that has +/// already timed out is in no position to enforce it. +pub fn run_platform_config_value_within_duration( + label: &str, + config: &serde_json::Value, + extra_env: &[(&str, &str)], + cwd: Option<&Path>, + deadline: Duration, +) -> Option { + /// Short enough that a prompt failure is still reported as prompt, long + /// enough that polling is not a busy-wait. + const POLL_INTERVAL: Duration = Duration::from_millis(25); + + let exe = find_platform_exec().expect("native executor binary should be available"); + let encoded = STANDARD.encode(config.to_string().as_bytes()); + + let start = Instant::now(); + let mut cmd = Command::new(&exe); + cmd.arg("--config-base64").arg(encoded); + for (key, value) in extra_env { + cmd.env(key, value); + } + if let Some(dir) = cwd { + cmd.current_dir(dir); + } + cmd.stdin(Stdio::null()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()); + + let mut child = cmd + .spawn() + .unwrap_or_else(|error| panic!("failed to execute {label}: {error}")); + + // Both pipes are drained on their own threads: waiting on the process + // while its output sits unread would deadlock as soon as a chatty child + // filled a pipe buffer. + let mut stdout_pipe = child.stdout.take().expect("stdout was piped"); + let mut stderr_pipe = child.stderr.take().expect("stderr was piped"); + let stdout_reader = std::thread::spawn(move || { + let mut buffer = Vec::new(); + let _ = stdout_pipe.read_to_end(&mut buffer); + buffer + }); + let stderr_reader = std::thread::spawn(move || { + let mut buffer = Vec::new(); + let _ = stderr_pipe.read_to_end(&mut buffer); + buffer + }); + + let status = loop { + match child.try_wait().expect("poll the executor's status") { + Some(status) => break Some(status), + None if start.elapsed() >= deadline => { + let _ = child.kill(); + let _ = child.wait(); + break None; + } + None => std::thread::sleep(POLL_INTERVAL), + } + }; + // Returning here leaves the readers detached on purpose. A descendant that + // inherited the pipes can hold them open past the kill, and joining would + // reintroduce exactly the hang this function exists to bound. + let status = status?; + + let stdout = stdout_reader.join().expect("stdout reader thread panicked"); + let stderr = stderr_reader.join().expect("stderr reader thread panicked"); + + Some(command_result( + label, + Output { + status, + stdout, + stderr, + }, + start.elapsed().as_millis(), + )) +} + +/// Run `wxc-test-driver.exe` against a directory or a single config file. +pub fn run_test_driver(target: &Path, extra_args: &[&str]) -> CommandResult { + let exe = find_binary("wxc-test-driver.exe").expect("wxc-test-driver.exe should be available"); + let mut args = vec![target.display().to_string()]; + args.extend(extra_args.iter().map(|arg| (*arg).to_string())); + + run_executable(&format!("wxc-test-driver {}", target.display()), &exe, args) +} + +/// Assert that a command exited successfully. +pub fn assert_success(result: &CommandResult) { + assert_exit(result, 0, None); +} + +/// Assert success, or skip when the local machine lacks sandbox runtime prerequisites. +pub fn assert_success_or_skip_missing_prerequisite(result: &CommandResult) { + if result.is_missing_process_prerequisite() { + println!( + "SKIPPED: {} requires local sandbox runtime prerequisites not available here", + result.label + ); + return; + } + + assert_success(result); +} + +/// Assert that a command exited with the expected code and optional output. +pub fn assert_exit(result: &CommandResult, expected_exit: i32, output_contains: Option<&str>) { + if result.code != Some(expected_exit) { + panic!( + "{} failed: expected exit {}, got {:?}\n--- stdout ---\n{}\n--- stderr ---\n{}", + result.label, expected_exit, result.code, result.stdout, result.stderr + ); + } + + if let Some(expected) = output_contains { + let combined = result.combined_output_with_decoded_base64(); + if !combined.contains(expected) { + panic!( + "{} failed: output missing '{}'\n--- combined output ---\n{}", + result.label, expected, combined + ); + } + } +} + +// --------------------------------------------------------------------------- +// Temporary filesystem setup +// --------------------------------------------------------------------------- + +/// Temporary directories removed when the guard is dropped. +#[derive(Debug)] +pub struct TempDirs { + paths: Vec, +} + +impl TempDirs { + /// Create temporary directories, removing any stale versions first. + pub fn create(paths: &[&str]) -> Self { + let paths: Vec = paths.iter().map(PathBuf::from).collect(); + for path in &paths { + remove_dir_all_if_exists(path); + fs::create_dir_all(path).unwrap_or_else(|error| { + panic!("failed to create temp dir {}: {error}", path.display()) + }); + } + Self { paths } + } + + /// Write a UTF-8 text file to an absolute path. + /// + /// The file is cleaned up when its parent directory is one of this guard's + /// tracked temporary directories. + pub fn write_absolute_file(&self, absolute_path: &str, contents: &str) { + let path = PathBuf::from(absolute_path); + assert!( + path.is_absolute(), + "temporary test file path must be absolute: {}", + path.display() + ); + if let Some(parent) = path.parent() { + fs::create_dir_all(parent).unwrap_or_else(|error| { + panic!("failed to create parent dir {}: {error}", parent.display()) + }); + } + fs::write(&path, contents) + .unwrap_or_else(|error| panic!("failed to write {}: {error}", path.display())); + } +} + +impl Drop for TempDirs { + fn drop(&mut self) { + for path in &self.paths { + remove_dir_all_if_exists(path); + } + } +} + +fn remove_dir_all_if_exists(path: &Path) { + if !path.exists() { + return; + } + + if fs::remove_dir_all(path).is_ok() { + return; + } + + std::thread::sleep(Duration::from_millis(100)); + if path.exists() { + fs::remove_dir_all(path).unwrap_or_else(|error| { + panic!("failed to remove temp dir {}: {error}", path.display()) + }); + } +} diff --git a/src/testing/wxc_e2e_tests/tests/e2e_bubblewrap_characterization.rs b/src/testing/wxc_e2e_tests/tests/e2e_bubblewrap_characterization.rs index f5b2df54e..27302ad41 100644 --- a/src/testing/wxc_e2e_tests/tests/e2e_bubblewrap_characterization.rs +++ b/src/testing/wxc_e2e_tests/tests/e2e_bubblewrap_characterization.rs @@ -20,7 +20,9 @@ use serde_json::json; use std::fs; use std::path::PathBuf; use std::time::{Duration, Instant}; -use wxc_e2e_tests::{has_bwrap, has_platform_exec, run_platform_config_value}; +use wxc_e2e_tests::{ + has_bwrap, has_platform_exec, run_platform_config_value, +}; const SCHEMA_VERSION: &str = "0.7.0-alpha"; @@ -215,30 +217,34 @@ fn bubblewrap_anchors_a_relative_process_cwd_from_0_9() { /// reporting success or hanging. /// /// The sandbox is torn down on the same path as a normal exit, so a shell that -/// never execs anything must still release the run. A hang here would show up -/// as the harness blocking rather than as a wrong exit code, which is why the -/// elapsed time is bounded too. +/// never execs anything must still release the run. Termination is therefore +/// the property under test, and it is enforced by the harness deadline rather +/// than by an elapsed-time assertion: a run that never returns could not be +/// measured by one. #[test] fn bubblewrap_reports_a_missing_command() { if !ready() { return; } - const MAX_ELAPSED: Duration = Duration::from_secs(30); + const DEADLINE: Duration = Duration::from_secs(30); let cfg = config("missing-command", "mxc-char-definitely-not-a-real-binary"); - let started = Instant::now(); - let result = run_platform_config_value("bwrap missing command", &cfg, &[], None); - let elapsed = started.elapsed(); - let out = result.combined_output(); + let result = run_platform_config_value_within_duration( + "bwrap missing command", + &cfg, + &[], + None, + DEADLINE, + ) + .unwrap_or_else(|| { + panic!("a missing command should fail promptly; it was still running after {DEADLINE:?}") + }); assert_ne!( result.code, Some(0), - "a missing command should fail the run. Output:\n{out}" - ); - assert!( - elapsed < MAX_ELAPSED, - "a missing command should fail promptly; took {elapsed:?}" + "a missing command should fail the run. Output:\n{}", + result.combined_output() ); } From ac637ab515af494a81e34e0130af821845887750 Mon Sep 17 00:00:00 2001 From: Elliot Michlin <31219104+theelliotm@users.noreply.github.com> Date: Fri, 25 Sep 2026 14:01:11 -0700 Subject: [PATCH 5/7] address copilot feedback --- docs/bwrap-support/bubblewrap-backend.md | 14 ++--- docs/schema.md | 2 +- src/testing/wxc_e2e_tests/src/lib.rs | 52 ++++++++++++++----- .../tests/e2e_bubblewrap_characterization.rs | 1 + 4 files changed, 50 insertions(+), 19 deletions(-) diff --git a/docs/bwrap-support/bubblewrap-backend.md b/docs/bwrap-support/bubblewrap-backend.md index e6716d8ee..ea99565ed 100644 --- a/docs/bwrap-support/bubblewrap-backend.md +++ b/docs/bwrap-support/bubblewrap-backend.md @@ -668,12 +668,14 @@ request fails if its private namespace cannot be configured. proxy-mode execution on the host indefinitely. A successful probe is cached for the life of the process; failures are not, so installing the missing tool takes effect without a restart. -1. When a proxy is requested, the runner routes the sandbox to it. On v0.9 the - proxy is named by `runtimeConfig.networkProxy` and must already be listening - on loopback; the caller starts it. On schema 0.6–0.8 it is named by - `network.proxy`, and `builtinTestServer: true` additionally makes the runner - launch the bundled `unix-test-proxy` on loopback (testing-only, gated behind - `--allow-testing-features`). +1. When a proxy is requested, the runner routes the sandbox to it. It must + already be listening on loopback; the caller starts it. + `runtimeConfig.networkProxy` names it from schema 0.8 onward and is the only + spelling on 0.9. The legacy `network.proxy` field names it on 0.6–0.8, where + `builtinTestServer: true` additionally makes the runner launch the bundled + `unix-test-proxy` on loopback (testing-only, gated behind + `--allow-testing-features`). Both spellings normalize to the same + `policy.network_proxy`, so 0.8 accepts either and enforces them identically. 2. The runner creates a same-UID user-namespace supervisor, starts Bubblewrap with `--unshare-net`, and keeps the workload behind a startup barrier. 3. The supervisor attaches `slirp4netns` to Bubblewrap's private network diff --git a/docs/schema.md b/docs/schema.md index d717b80a1..a0e06be7f 100644 --- a/docs/schema.md +++ b/docs/schema.md @@ -282,7 +282,7 @@ use: |---------|----------------------------------------| | Windows ProcessContainer (AppContainer / BaseContainer) | First `readwritePaths` entry that is an existing directory, else the first such `readonlyPaths` entry, else the system drive root (`%SystemDrive%\`). Never `NULL`. | | Seatbelt (macOS) | Same precedence, with `~` expanded as the profile expands it; falls back to `/`. | -| Bubblewrap (Linux) | No substitution — a policy grant is never adopted. `--chdir` is emitted only for an explicit `process.cwd`, and `HOME` is left unset — see [`docs/bwrap-support/bubblewrap-backend.md`](bwrap-support/bubblewrap-backend.md). | +| Bubblewrap (Linux) | No substitution — a policy grant is never adopted. `--chdir` is emitted only for an explicit `process.cwd`, which from 0.9 is also normalized against the sandbox root and used as `HOME`. With no explicit `cwd` there is no `--chdir` and `HOME` is unset — see [`docs/bwrap-support/bubblewrap-backend.md`](bwrap-support/bubblewrap-backend.md). | | LXC / WSL Container | The container root — see [`docs/lxc-support/lxc-backend.md`](lxc-support/lxc-backend.md). | | MicroVM (NanVix) / Hyperlight | Not applicable — these backends reject a working directory outright. | diff --git a/src/testing/wxc_e2e_tests/src/lib.rs b/src/testing/wxc_e2e_tests/src/lib.rs index 3e3af1c03..683502cf2 100644 --- a/src/testing/wxc_e2e_tests/src/lib.rs +++ b/src/testing/wxc_e2e_tests/src/lib.rs @@ -10,6 +10,7 @@ use std::fs; use std::io::Read; use std::path::{Path, PathBuf}; use std::process::{Command, Output, Stdio}; +use std::sync::mpsc; use std::time::{Duration, Instant}; use base64::{engine::general_purpose::STANDARD, Engine}; @@ -661,6 +662,10 @@ pub fn run_platform_config_value( /// `None` if it expired. Use this wherever termination is the /// property under test. /// +/// The deadline covers both the executor's exit and the close of its output +/// pipes, so a descendant that outlives the executor still holding them open +/// is reported as an expiry rather than blocking the caller. +/// /// On expiry the executor is killed and reaped. Its sandboxed descendants are /// not walked: that is the backend's teardown contract, and a test that has /// already timed out is in no position to enforce it. @@ -697,24 +702,28 @@ pub fn run_platform_config_value_within_duration( // Both pipes are drained on their own threads: waiting on the process // while its output sits unread would deadlock as soon as a chatty child - // filled a pipe buffer. + // filled a pipe buffer. Results come back over channels so that collecting + // them can honor the deadline too. let mut stdout_pipe = child.stdout.take().expect("stdout was piped"); let mut stderr_pipe = child.stderr.take().expect("stderr was piped"); - let stdout_reader = std::thread::spawn(move || { + let (stdout_tx, stdout_rx) = mpsc::channel(); + let (stderr_tx, stderr_rx) = mpsc::channel(); + std::thread::spawn(move || { let mut buffer = Vec::new(); let _ = stdout_pipe.read_to_end(&mut buffer); - buffer + let _ = stdout_tx.send(buffer); }); - let stderr_reader = std::thread::spawn(move || { + std::thread::spawn(move || { let mut buffer = Vec::new(); let _ = stderr_pipe.read_to_end(&mut buffer); - buffer + let _ = stderr_tx.send(buffer); }); + let expires_at = start + deadline; let status = loop { match child.try_wait().expect("poll the executor's status") { Some(status) => break Some(status), - None if start.elapsed() >= deadline => { + None if Instant::now() >= expires_at => { let _ = child.kill(); let _ = child.wait(); break None; @@ -722,13 +731,13 @@ pub fn run_platform_config_value_within_duration( None => std::thread::sleep(POLL_INTERVAL), } }; - // Returning here leaves the readers detached on purpose. A descendant that - // inherited the pipes can hold them open past the kill, and joining would - // reintroduce exactly the hang this function exists to bound. + // Each `?` below leaves the readers detached on purpose. A descendant that + // inherited the pipes can hold them open past the executor's own exit, and + // blocking on them would reintroduce exactly the hang this function exists + // to bound. let status = status?; - - let stdout = stdout_reader.join().expect("stdout reader thread panicked"); - let stderr = stderr_reader.join().expect("stderr reader thread panicked"); + let stdout = collect_within_duration(&stdout_rx, expires_at)?; + let stderr = collect_within_duration(&stderr_rx, expires_at)?; Some(command_result( label, @@ -741,6 +750,25 @@ pub fn run_platform_config_value_within_duration( )) } +/// Take a reader thread's output, giving up at `expires_at`. +/// +/// The buffer is only sent once its pipe reaches EOF, so a pending receive +/// means the pipe is still open somewhere. +fn collect_within_duration( + reader: &mpsc::Receiver>, + expires_at: Instant, +) -> Option> { + match reader.try_recv() { + Ok(buffer) => Some(buffer), + // A zero or negative remainder must not discard output that is already + // waiting, hence the `try_recv` first. + Err(mpsc::TryRecvError::Empty) => reader + .recv_timeout(expires_at.saturating_duration_since(Instant::now())) + .ok(), + Err(mpsc::TryRecvError::Disconnected) => None, + } +} + /// Run `wxc-test-driver.exe` against a directory or a single config file. pub fn run_test_driver(target: &Path, extra_args: &[&str]) -> CommandResult { let exe = find_binary("wxc-test-driver.exe").expect("wxc-test-driver.exe should be available"); diff --git a/src/testing/wxc_e2e_tests/tests/e2e_bubblewrap_characterization.rs b/src/testing/wxc_e2e_tests/tests/e2e_bubblewrap_characterization.rs index 27302ad41..8e4438f40 100644 --- a/src/testing/wxc_e2e_tests/tests/e2e_bubblewrap_characterization.rs +++ b/src/testing/wxc_e2e_tests/tests/e2e_bubblewrap_characterization.rs @@ -22,6 +22,7 @@ use std::path::PathBuf; use std::time::{Duration, Instant}; use wxc_e2e_tests::{ has_bwrap, has_platform_exec, run_platform_config_value, + run_platform_config_value_within_duration, }; const SCHEMA_VERSION: &str = "0.7.0-alpha"; From 01dc4fbca55acd723f92c7e31365be0eb17ea757 Mon Sep 17 00:00:00 2001 From: Elliot Michlin <31219104+theelliotm@users.noreply.github.com> Date: Fri, 25 Sep 2026 14:17:13 -0700 Subject: [PATCH 6/7] Make bwrap proxy cleanup idempotent and gate on the real capability --- .../integration/linux-bubblewrap.test.ts | 26 +++++++++---------- tests/scripts/run_bwrap_network_proxy_test.sh | 4 +++ 2 files changed, 16 insertions(+), 14 deletions(-) diff --git a/sdk/node/tests/integration/linux-bubblewrap.test.ts b/sdk/node/tests/integration/linux-bubblewrap.test.ts index d3cd4f14a..16b83b31a 100644 --- a/sdk/node/tests/integration/linux-bubblewrap.test.ts +++ b/sdk/node/tests/integration/linux-bubblewrap.test.ts @@ -192,25 +192,23 @@ describe(`Linux Bubblewrap network proxy, legacy shape (schema ${PROXY_SCHEMA})` // That posture is enforced from inside a private network namespace routed by // rootless slirp4netns, which the legacy path does not need -- hence the extra // prerequisite here. +// +// The SDK's own capability answers it: the native probe runs `slirp4netns +// --version`, checks that private namespaces can actually be unshared, and +// inspects the iptables backend, so it fails closed on any part of the +// dependency set. Testing for the binary alone would let a host with an +// unusable slirp, `unshare`, `nsenter`, `iptables`, or `ip6tables` past the +// gate and report an environmental failure as a test failure. const PROXY_SCHEMA_09 = '0.9.0-alpha'; -const hasSlirp4netns = (() => { - if (os.platform() !== 'linux') return false; - const pathDirs = (process.env.PATH ?? '').split(path.delimiter); - return pathDirs.some((dir) => { - if (!dir) return false; - try { - return fs.existsSync(path.join(dir, 'slirp4netns')); - } catch { - return false; - } - }); -})(); +const hasProxyEnforcement = + isLinuxBubblewrap && + sdk.getPlatformSupport().bubblewrapNetwork?.proxyEnforcement === 'supported'; describe(`Linux Bubblewrap network proxy (schema ${PROXY_SCHEMA_09})`, { skip: !isLinuxBubblewrap ? 'Linux Bubblewrap proxy tests require Linux with bwrap installed' - : !hasSlirp4netns - ? 'the 0.9 proxy posture needs slirp4netns to route its private namespace' + : !hasProxyEnforcement + ? 'this host cannot enforce proxy-only egress (see PlatformSupport.bubblewrapNetwork.warnings)' : undefined, }, () => { const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'mxc-sdk-bwrap-proxy-09-')); diff --git a/tests/scripts/run_bwrap_network_proxy_test.sh b/tests/scripts/run_bwrap_network_proxy_test.sh index ee70a0e95..3d8352974 100644 --- a/tests/scripts/run_bwrap_network_proxy_test.sh +++ b/tests/scripts/run_bwrap_network_proxy_test.sh @@ -65,6 +65,10 @@ cleanup_control() { wait "$pid" 2>/dev/null || true fi done + # This runs once explicitly and again from the EXIT trap. A reaped PID can + # be reused by an unrelated process before then, so forget it once reaped. + CONTROL_PID="" + PROXY_ENDPOINT_PID="" exec 7>&- 2>/dev/null || true exec 8>&- 2>/dev/null || true rm -rf "$CONTROL_DIR" From 40fa298954eeb8a78b5615a655588c88980584a1 Mon Sep 17 00:00:00 2001 From: Elliot Michlin <31219104+theelliotm@users.noreply.github.com> Date: Fri, 25 Sep 2026 14:31:11 -0700 Subject: [PATCH 7/7] Correcting bubblewrap proxy docs --- docs/bwrap-support/bubblewrap-backend.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/docs/bwrap-support/bubblewrap-backend.md b/docs/bwrap-support/bubblewrap-backend.md index ea99565ed..cbbc4a6b5 100644 --- a/docs/bwrap-support/bubblewrap-backend.md +++ b/docs/bwrap-support/bubblewrap-backend.md @@ -668,14 +668,16 @@ request fails if its private namespace cannot be configured. proxy-mode execution on the host indefinitely. A successful probe is cached for the life of the process; failures are not, so installing the missing tool takes effect without a restart. -1. When a proxy is requested, the runner routes the sandbox to it. It must - already be listening on loopback; the caller starts it. - `runtimeConfig.networkProxy` names it from schema 0.8 onward and is the only - spelling on 0.9. The legacy `network.proxy` field names it on 0.6–0.8, where - `builtinTestServer: true` additionally makes the runner launch the bundled - `unix-test-proxy` on loopback (testing-only, gated behind - `--allow-testing-features`). Both spellings normalize to the same - `policy.network_proxy`, so 0.8 accepts either and enforces them identically. +1. When a proxy is requested, the runner routes the sandbox to it; the caller + starts it. `runtimeConfig.networkProxy` names it from schema 0.8 onward and + is the only spelling on 0.9; the parser accepts only a loopback endpoint + there. The legacy `network.proxy` field names it on 0.6–0.8 and also accepts + a hostname or routable endpoint, which is resolved on the host and pinned + into the sandbox's `/etc/hosts`; there `builtinTestServer: true` + additionally makes the runner launch the bundled `unix-test-proxy` on + loopback (testing-only, gated behind `--allow-testing-features`). Both + spellings normalize to the same `policy.network_proxy`, so 0.8 accepts + either and enforces them identically. 2. The runner creates a same-UID user-namespace supervisor, starts Bubblewrap with `--unshare-net`, and keeps the workload behind a startup barrier. 3. The supervisor attaches `slirp4netns` to Bubblewrap's private network