Skip to content

Commit e60098d

Browse files
authored
fix(snap): install openshell snap via install.sh when snap available (#3656)
* fix(snap): update stale snap docs and tests Previously, the `openshell` snap required the `docker` snap. Now, it works with any Docker daemon running on the system. Furthermore, the `snap-declaration` assertion on the `openshell` snap when installed from the Snap Store causes the `openshell` snap to always connect to the system `:docker` slot, rather than a slot provided by the `docker` snap. This commit updates the documentation, including the `description` field in `snapcraft.yaml`, to ensure that all information is correct and up-to-date. Additionally, some tests connected the `openshell:docker` plug to the `docker` snap's `docker:docker-daemon` slot, which is inconsistent with how the `openshell` snap operates when installed from the store. Update those tests to connect to the system `:docker` slot as well. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fix(snap): require snapd 2.76 for openshell snap Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fix(nix): align snap gateway reproducer timeout with release-canary Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fix(snap): require snapd 2.77 for openshell snap Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fixup! fix(snap): require snapd 2.77 for openshell snap Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * feat(snap): install openshell snap via install.sh when snap available Change `install.sh` to install the `openshell` snap by default when snapd is installed on the host. This installs the snap from the `latest/stable` channel, which should match the most up-to-date release tag on github. If `OPENSHELL_VERSION=dev` is set for `install.sh`, then it will install the `openshell` snap from the `latest/edge` channel, which matches the latest dev release available on github. The `openshell` snap currently requires Docker in order to function. If a Docker daemon is already installed on the system, it will be used by the `openshell` snap. Otherwise, `install.sh` will install the `docker` snap first, wait for the Docker daemon to be ready, and then install the `openshell` snap. Also, update the `release-canary.yml` to split the `ubuntu-snap` job into `ubuntu-snap-system-docker` and `ubuntu-snap-provisions-docker`, which test the two aforementioned scenarios. Previously, `ubuntu-snap` manually installed a given snap artifact as built from CI, but with these new jobs, it instead uses `install.sh` to install the published `openshell` snap from the `latest/edge` track, thus matching the behavior of the other release canary jobs. Make corresponding changes to the `nix` guest reproducer, documentation, and tests. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fix(snap): configure local gateway authentication The `openshell` snap runs the gateway as a systemd system service, which runs as root. Thus, the mTLS certs are generated by root and stored in a root-owned directory to which non-root users do not have access. For this reason, the snap's `openshell-gateway-wrapper` script sets `OPENSHELL_DISABLE_TLS=true`. This commit ensures that the `openshell` snap's gateway allows unauthenticated local access by writing a default `gateway.toml` configuration file during the install hook, which runs after the snap is first installed but before services are started. The config file contains sets `allow_unauthenticated_users = true`. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fix(install): configure snap gateway authentication Recently, a new install hook was added which writes a default config file for the `openshell` snap to allow unauthenticated local access to the gateway. This is because the gateway service runs as root and the mTLS certificates are not accessible to non-root users. However, the `install.sh` script installs the `openshell` snap from the snap store, and the published version may not yet have that new install hook. Or, the user may already have the snap installed, in which case the install hook does not run. In either case, we need `install.sh` to ensure that the config file is written to set the gateway auth to `allow_unauthenticated_users = true`, and then restart the openshell gateway service. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fixup! feat(snap): install openshell snap via install.sh when snap available Signed-off-by: Oliver Calder <oliver.calder@canonical.com> --------- Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
1 parent a8f98ec commit e60098d

14 files changed

Lines changed: 1062 additions & 205 deletions

File tree

‎.agents/skills/test-release-canary/SKILL.md‎

Lines changed: 14 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,10 @@ The Release Canary (`.github/workflows/release-canary.yml`) smoke-tests the arti
1414
| Job | Runner | Verifies |
1515
|---|---|---|
1616
| `macos` | `macos-latest-xlarge` | Installs the dev Homebrew artifacts, reaches the VM gateway, and creates, executes in, and deletes a sandbox. |
17-
| `ubuntu` | `ubuntu-latest` | Installs the dev Debian package, reaches the Docker gateway, and creates, executes in, and deletes a sandbox. |
17+
| `ubuntu-deb` | `ubuntu-latest` | Installs the dev Debian package, reaches the Docker gateway, and creates, executes in, and deletes a sandbox. |
1818
| `fedora` | `fedora:latest` container | Installs the dev RPM packages, reaches the Podman gateway, and creates, executes in, and deletes a sandbox. |
19-
| `ubuntu-snap` | `ubuntu-latest` | Installs the Release Dev Snap, connects its interfaces, reaches the Docker gateway, and creates, executes in, and deletes a sandbox. |
19+
| `ubuntu-snap-system-docker` | `ubuntu-latest` | Uses `install.sh` to install the snap from `latest/edge`, reuses system Docker, reaches the Docker gateway, and creates, executes in, and deletes a sandbox, and verifies that the Docker snap is not installed. |
20+
| `ubuntu-snap-provisions-docker` | `ubuntu-latest` | Uses `install.sh` to install the Docker and OpenShell snaps, waits for Docker, reaches the Docker gateway, and creates, executes in, and deletes a sandbox. |
2021
| `kubernetes` | `ubuntu-latest` + kind | Installs the dev Helm chart, reaches the in-cluster gateway, and creates, executes in, and deletes a sandbox using the published runtime images. |
2122

2223
All canary jobs disable anonymous OpenShell telemetry. Host package jobs inject
@@ -25,8 +26,11 @@ Kubernetes job installs with `server.telemetryEnabled=false`, so smoke traffic
2526
does not contribute to product usage metrics.
2627

2728
The workflow sets `OPENSHELL_VERSION=dev`, so every `install.sh` job consumes the
28-
rolling dev release produced by the triggering workflow. Kubernetes pins the
29-
matching `0.0.0-dev` chart and `:dev` images.
29+
rolling dev release produced by the triggering workflow. Both snap lanes track
30+
`latest/edge`; one reuses system Docker while the other verifies automatic
31+
Docker snap provisioning. The Debian and Kubernetes CLI lanes remove snapd so
32+
they continue to exercise the dev Debian package. Kubernetes pins the matching
33+
`0.0.0-dev` chart and `:dev` images.
3034

3135
The host-package jobs exercise fresh installs, not upgrades from a persisted
3236
schema-v1 gateway config. Validate Homebrew and RPM exact-default migration with
@@ -50,7 +54,7 @@ on:
5054
```
5155
5256
- **Automatic.** Every successful `Release Dev` run (on `main` or a manual dispatch of Release Dev) fires the canary. Each job gates on `github.event.workflow_run.conclusion == 'success'` so a failed Release Dev does not run the canary.
53-
- **Manual.** `workflow_dispatch` lets you run the canary on demand against any branch's workflow definition. To include `ubuntu-snap`, supply `release-dev-run-id` for a successful Release Dev run whose Snap artifact should be tested; without it, that job is skipped because no artifact is available.
57+
- **Manual.** `workflow_dispatch` lets you run the canary on demand against any branch's workflow definition.
5458

5559
When dispatched manually, `github.event.workflow_run.head_sha` is empty and the workflow falls back to `github.sha` (the branch tip) for the `install.sh` URL.
5660

@@ -62,13 +66,6 @@ Run the canary as-is on the current branch:
6266
gh workflow run release-canary.yml --ref "$(git branch --show-current)"
6367
```
6468

65-
To exercise the Ubuntu Snap job, pass the successful Release Dev run ID:
66-
67-
```shell
68-
gh workflow run release-canary.yml --ref "$(git branch --show-current)" \
69-
-f release-dev-run-id=<release-dev-run-id>
70-
```
71-
7269
Watch the run that starts:
7370

7471
```shell
@@ -87,7 +84,7 @@ gh run view <run-id> --log-failed
8784

8885
When you change `release-canary.yml` on a branch, a manual dispatch on that branch tests *your branch's workflow logic* against *main's published dev artifacts* (`0.0.0-dev` chart, `:dev` images, and the `dev` GitHub release). This is what you want for iterating on the canary — you're validating that the canary still works against known-good artifacts.
8986

90-
Note `install.sh` is pulled from `raw.githubusercontent.com/NVIDIA/OpenShell/${head_sha}/install.sh`, so changes to `install.sh` on your branch *are* exercised even though the binaries it downloads are from the latest public tag.
87+
Note `install.sh` is pulled from `raw.githubusercontent.com/NVIDIA/OpenShell/${head_sha}/install.sh`, so changes to `install.sh` on your branch *are* exercised even though packages come from the latest public dev release and the snap comes from `latest/edge`.
9188

9289
## Testing artifacts from a specific SHA
9390

@@ -137,10 +134,11 @@ Loopback registration auto-derives the gateway name to `openshell` if `--name` i
137134

138135
| Symptom | Likely cause | Where to look |
139136
|---|---|---|
140-
| `macos`/`ubuntu`/`fedora` job fails on `install.sh` | Dev release missing an asset, checksum mismatch, or `install.sh` regression on this branch. | Job log around the `curl … install.sh \| sh` step. |
137+
| `macos`/`ubuntu-deb`/`fedora` job fails on `install.sh` | Dev release missing an asset, checksum mismatch, or `install.sh` regression on this branch. | Job log around the `curl … install.sh \| sh` step. |
141138
| Sandbox create or exec fails | Published sandbox and supervisor artifacts are missing, incompatible, or cannot establish the protected runtime channel. | Gateway logs plus Docker, Podman, VM, Snap, or Kubernetes runtime diagnostics for the job. |
142-
| `macos`/`ubuntu`/`fedora` job fails on `openshell status` | Local gateway service did not start (systemd/brew/podman). Often a driver issue. | Service logs in the job log; `OPENSHELL_COMPUTE_DRIVER` env in the "Ensure …" step. |
143-
| `ubuntu-snap` fails after interface connection | The gateway did not recover after Docker became available, or did not become reachable within the 30-second bound. | Failure diagnostics dump Snap service/connection/change state, gateway and snapd journals, Snap logs, and port 17670 listeners. |
139+
| `macos`/`ubuntu-deb`/`fedora` job fails on `openshell status` | Local gateway service did not start (systemd/brew/podman). Often a driver issue. | Service logs in the job log; `OPENSHELL_COMPUTE_DRIVER` env in the "Ensure …" step. |
140+
| `ubuntu-snap-system-docker` fails during `install.sh` | System Docker was unavailable, the edge revision or automatic interfaces were unavailable, or the gateway did not become reachable. | Failure diagnostics dump system Docker, snap service/connection/change state, gateway and snapd journals, snap logs, and port 17670 listeners. |
141+
| `ubuntu-snap-provisions-docker` fails during `install.sh` | The Docker snap was not installed or did not become ready before OpenShell installation, or the gateway did not become reachable. | Failure diagnostics also dump the Docker snap service and journal. |
144142
| `kubernetes` job fails on `helm install --wait` | Chart did not deploy in 5 min — usually image pull failure or readiness probe failing. | "Diagnostics on failure" step dumps `helm status`, manifest, pod describe, pod logs. |
145143
| `kubernetes` job fails on `kubectl wait` | Gateway pod stuck `CrashLoopBackOff` or `ImagePullBackOff`. | Diagnostics dump; check `:dev` image existence at `ghcr.io/nvidia/openshell/gateway`. |
146144
| `kubernetes` job fails on `openshell gateway add` or `status` | Port-forward not reachable, or CLI/gateway proto mismatch. | `port-forward.log` and `openshell gateway list` in the diagnostics dump. |

‎.github/workflows/release-canary.yml‎

Lines changed: 104 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,6 @@ name: Release Canary
22

33
on:
44
workflow_dispatch:
5-
inputs:
6-
release-dev-run-id:
7-
description: "Successful Release Dev run ID whose Snap artifact to test"
8-
required: false
9-
type: string
105
workflow_run:
116
workflows: ["Release Dev"]
127
types: [completed]
@@ -58,12 +53,22 @@ jobs:
5853
fi
5954
done
6055
61-
ubuntu:
62-
name: Ubuntu Docker
56+
ubuntu-deb:
57+
name: Ubuntu DEB
6358
if: ${{ github.event_name == 'workflow_dispatch' || github.event.workflow_run.conclusion == 'success' }}
6459
runs-on: ubuntu-latest
6560
timeout-minutes: 20
6661
steps:
62+
- name: Remove snapd
63+
run: |
64+
set -euo pipefail
65+
snaps=$(snap list 2>/dev/null | awk 'NR > 1 { print $1 }')
66+
if [ -n "$snaps" ]; then
67+
sudo snap remove $snaps
68+
fi
69+
sudo apt-get -y --purge remove snapd
70+
! command -v snap >/dev/null 2>&1
71+
6772
- name: Ensure Docker
6873
run: |
6974
if ! command -v docker >/dev/null 2>&1; then
@@ -186,9 +191,9 @@ jobs:
186191
run: |
187192
docker rm -f "${FEDORA_CANARY_CONTAINER}" >/dev/null 2>&1 || true
188193
189-
ubuntu-snap:
190-
name: Ubuntu Snap
191-
if: ${{ github.event.workflow_run.conclusion == 'success' || inputs.release-dev-run-id != '' }}
194+
ubuntu-snap-system-docker:
195+
name: Ubuntu Snap with system Docker
196+
if: ${{ github.event_name == 'workflow_dispatch' || github.event.workflow_run.conclusion == 'success' }}
192197
runs-on: ubuntu-latest
193198
timeout-minutes: 20
194199
steps:
@@ -201,54 +206,100 @@ jobs:
201206
sudo systemctl start snapd
202207
sudo snap wait system seed.loaded
203208
204-
- name: Install Docker snap
209+
- name: Ensure Docker
205210
run: |
206211
set -euo pipefail
207-
sudo snap install docker
208-
209-
- name: Download snap from release-dev artifacts
210-
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
211-
with:
212-
github-token: ${{ github.token }}
213-
run-id: ${{ inputs.release-dev-run-id || github.event.workflow_run.id }}
214-
pattern: snap-linux-amd64
215-
path: release/
216-
merge-multiple: true
212+
if ! command -v docker >/dev/null 2>&1; then
213+
sudo apt-get update
214+
sudo apt-get install -y docker.io
215+
fi
216+
sudo systemctl start docker || sudo service docker start
217+
docker info
217218
218-
- name: Install snap (dangerous — from release, not store)
219+
- name: Install and check status
219220
run: |
220221
set -euo pipefail
221222
sudo systemctl set-environment \
222223
"OPENSHELL_TELEMETRY_ENABLED=${OPENSHELL_TELEMETRY_ENABLED}"
223-
sudo snap install ./release/*.snap --dangerous
224+
curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/${{ github.event.workflow_run.head_sha || github.sha }}/install.sh | sh
225+
sudo snap list openshell
226+
if sudo snap list docker >/dev/null 2>&1; then
227+
echo "install.sh unexpectedly installed the Docker snap" >&2
228+
exit 1
229+
fi
230+
snap info openshell | grep -E '^tracking: +latest/edge$'
231+
docker info
232+
sudo snap connections openshell | grep -E '^docker +openshell:docker +:docker +'
233+
openshell --version
234+
sudo snap services openshell
235+
openshell status
224236
225-
- name: Connect interfaces
237+
- name: Create and exercise a sandbox
226238
run: |
227239
set -euo pipefail
228-
sudo snap connect openshell:docker docker:docker-daemon
229-
sudo snap connect openshell:log-observe
230-
sudo snap connect openshell:system-observe
240+
sandbox="rc-system-docker-${GITHUB_RUN_ID}"
241+
openshell sandbox create --name "$sandbox" --detach
242+
openshell sandbox exec --name "$sandbox" --no-tty -- true
243+
openshell sandbox delete "$sandbox"
231244
232-
- name: Register snap gateway and check status
245+
- name: Collect Snap diagnostics
246+
if: failure()
247+
run: |
248+
set +e
249+
docker info
250+
sudo systemctl status docker.service --no-pager
251+
sudo journalctl -b -u docker.service --no-pager -n 300
252+
sudo snap services openshell
253+
sudo snap connections openshell
254+
sudo snap changes
255+
sudo systemctl status snap.openshell.gateway.service --no-pager
256+
sudo journalctl -b -u snap.openshell.gateway.service --no-pager -n 300
257+
sudo journalctl -b -u snapd.service --no-pager -n 300
258+
sudo snap logs openshell.gateway -n=300
259+
sudo ss -ltnp '( sport = :17670 )'
260+
261+
ubuntu-snap-provisions-docker:
262+
name: Ubuntu Snap provisions Docker
263+
if: ${{ github.event_name == 'workflow_dispatch' || github.event.workflow_run.conclusion == 'success' }}
264+
runs-on: ubuntu-latest
265+
timeout-minutes: 20
266+
steps:
267+
- name: Install snapd
268+
run: |
269+
set -euo pipefail
270+
sudo apt-get update
271+
sudo apt-get install -y snapd
272+
sudo systemctl enable --now snapd.socket
273+
sudo systemctl start snapd
274+
sudo snap wait system seed.loaded
275+
276+
- name: Remove system Docker
277+
run: |
278+
set -euo pipefail
279+
sudo systemctl stop docker.service docker.socket 2>/dev/null || true
280+
sudo apt-get -y --purge remove docker-ce docker-ce-cli containerd.io \
281+
docker-buildx-plugin docker-compose-plugin docker.io 2>/dev/null || true
282+
! command -v docker >/dev/null 2>&1
283+
284+
- name: Install and check status
233285
run: |
234286
set -euo pipefail
287+
sudo systemctl set-environment \
288+
"OPENSHELL_TELEMETRY_ENABLED=${OPENSHELL_TELEMETRY_ENABLED}"
289+
curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/${{ github.event.workflow_run.head_sha || github.sha }}/install.sh | sh
290+
sudo snap list docker
291+
sudo snap list openshell
292+
snap info openshell | grep -E '^tracking: +latest/edge$'
293+
sudo docker info
294+
sudo snap connections openshell | grep -E '^docker +openshell:docker +:docker +'
235295
openshell --version
236296
sudo snap services openshell
237-
openshell gateway add http://127.0.0.1:17670 --local --name snap-docker
238-
openshell gateway select snap-docker
239-
for _ in $(seq 1 30); do
240-
if openshell status; then
241-
exit 0
242-
fi
243-
sleep 1
244-
done
245-
echo "Gateway did not become ready within 30 seconds" >&2
246-
exit 1
297+
openshell status
247298
248299
- name: Create and exercise a sandbox
249300
run: |
250301
set -euo pipefail
251-
sandbox="rc-${GITHUB_RUN_ID}"
302+
sandbox="rc-provisions-docker-${GITHUB_RUN_ID}"
252303
openshell sandbox create --name "$sandbox" --detach
253304
openshell sandbox exec --name "$sandbox" --no-tty -- true
254305
openshell sandbox delete "$sandbox"
@@ -257,10 +308,13 @@ jobs:
257308
if: failure()
258309
run: |
259310
set +e
260-
sudo snap services openshell
311+
sudo docker info
312+
sudo snap services docker openshell
261313
sudo snap connections openshell
262314
sudo snap changes
315+
sudo systemctl status snap.docker.dockerd.service --no-pager
263316
sudo systemctl status snap.openshell.gateway.service --no-pager
317+
sudo journalctl -b -u snap.docker.dockerd.service --no-pager -n 300
264318
sudo journalctl -b -u snap.openshell.gateway.service --no-pager -n 300
265319
sudo journalctl -b -u snapd.service --no-pager -n 300
266320
sudo snap logs openshell.gateway -n=300
@@ -336,6 +390,16 @@ jobs:
336390
cat port-forward.log >&2
337391
exit 1
338392
393+
- name: Remove snapd
394+
run: |
395+
set -euo pipefail
396+
snaps=$(snap list 2>/dev/null | awk 'NR > 1 { print $1 }')
397+
if [ -n "$snaps" ]; then
398+
sudo snap remove $snaps
399+
fi
400+
sudo apt-get -y --purge remove snapd
401+
! command -v snap >/dev/null 2>&1
402+
339403
- name: Install OpenShell CLI
340404
run: |
341405
set -euo pipefail

‎CI.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -399,7 +399,7 @@ These workflows run after merge to publish dev/tagged artifacts and verify them.
399399
|---|---|
400400
| `.github/workflows/release-dev.yml` | Publishes the rolling `dev` build on every push to `main`. Builds gateway, sandbox, and supervisor images and binaries, packages, wheels, and pushes the Helm chart as `oci://ghcr.io/nvidia/openshell/helm-chart:0.0.0-dev` (plus an immutable `0.0.0-dev.<sha>` pin). Also dispatchable manually. |
401401
| `.github/workflows/release-tag.yml` | Publishes tagged stable releases and manually dispatched pre-releases. Its automatic tag trigger excludes `-pre.*`. Security and integration failures do not block pre-release artifact publication. Stable publication requires the currently implemented qualification profile to pass; the summary identifies the remaining RFC 0014 coverage. |
402-
| `.github/workflows/release-canary.yml` | Smoke-tests published dev artifacts on `macos`, `ubuntu`, `fedora`, and `kubernetes` (kind + Helm) runners. Each job reaches its gateway and creates, exercises, and deletes a sandbox. It runs automatically after `Release Dev` succeeds and supports manual dispatch (`gh workflow run release-canary.yml --ref <branch>`). See the `test-release-canary` skill for the playbook and local kind reproduction. |
402+
| `.github/workflows/release-canary.yml` | Smoke-tests published dev artifacts in the `macos`, `ubuntu-deb`, `ubuntu-snap-system-docker`, `ubuntu-snap-provisions-docker`, `fedora`, and `kubernetes` (kind + Helm) jobs. Each job reaches its gateway and creates, exercises, and deletes a sandbox. The Snap jobs cover both an existing system Docker daemon and automatic Docker snap provisioning. It runs automatically after `Release Dev` succeeds and supports manual dispatch (`gh workflow run release-canary.yml --ref <branch>`). See the `test-release-canary` skill for the playbook and local kind reproduction. |
403403

404404
## Required status contexts
405405

‎architecture/build.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -318,6 +318,18 @@ configure the gateway to use them. The Ubuntu `deb` installer consumes
318318
for direct executable installation on every environment. Release Dev and
319319
Release Tag run Ubuntu conformance through the Debian package, while Fedora
320320
continues using direct executable installation until RPM coverage is available.
321+
The release canary separately exercises the public installer on Ubuntu. Its Snap
322+
lanes install the rolling development release from the Snap Store with both an
323+
existing system Docker daemon and a host where the installer must provision and
324+
wait for the Docker snap. Its Debian lane removes snapd before running the
325+
installer so Snap precedence cannot change the package under test.
326+
Snapd runs the gateway as a root-owned system service. Its generated client
327+
certificates reside in root-owned snap state and are unavailable to ordinary CLI
328+
users, so the Snap uses plaintext loopback transport and enables unauthenticated
329+
local users by default. Debian and RPM packages instead run systemd user services
330+
and use user-owned mTLS material. Bootstrap creates the default configuration
331+
only when it is missing. Sandbox-to-gateway sessions remain authenticated with
332+
gateway-minted JWTs.
321333
The Debian qualification profile keeps candidate-image overrides outside the
322334
operator-owned gateway configuration: it writes a harness-owned file under
323335
`/var/lib/openshell-qualification` and selects it through the packaged systemd

0 commit comments

Comments
 (0)