feat(snap): run gateway as a user service by default

[olivercalder]

Summary

Replace the existing system gateway service with separate user-gateway and system-gateway services. For new installs, only the user-gateway service is enabled, which simplifies TLS certificate access and brings parity with the other packaging formats. For existing installs, only the system-gateway service is enabled, for backwards compatibility.

Related Issue

Addresses the need to run the gateway as a user service in the snap package, as discussed with @drew on Slack. There's not a dedicated issue created yet, sorry. I can create one if necessary. This just concerns packaging and test code, no internal logic.

Changes

• Add separate user-gateway and system-gateway Snap services, both disabled by default and selected through Snap hooks.
• Remove the old gateway app so snapd stops it during upgrades before starting the renamed compatibility service.
• Run fresh installations through user-gateway with user-owned configuration, database, TLS, and JWT state under $SNAP_USER_COMMON.
• Preserve upgraded installations through system-gateway, retaining existing configuration, database, TLS identity, and other state under $SNAP_COMMON.
• Reuse one gateway wrapper for both services by making its canonical configuration path overridable alongside the existing database and TLS path overrides.
• Add a gateway-mode Snap setting with user and system values.
• Add an install hook that initializes fresh installations with gateway-mode=user.
• Add a configure hook that applies gateway-mode, disables the inactive service, and enables the selected service.
• Support switching service models with snap set openshell gateway-mode=user or gateway-mode=system.
• Update the post-refresh hook to identify legacy installations with no existing gateway-mode, migrate explicitly unsafe configuration, and set gateway-mode=system.
• Remove unsafe legacy configuration files or unsafe config symlinks without modifying symlink targets.
• Keep existing user or system selections unchanged during later refreshes.
• Update install.sh to distinguish user and system modes, validate Docker access in the appropriate context, and restart the selected enduring service after explicit refreshes.
• Remove root-to-user TLS copying from fresh user-gateway installations and register them using their user-owned local mTLS bundle.
• Retain root-owned TLS enrollment in the installer for legacy system-gateway upgrades.
• Restore manual TLS enrollment documentation for users whose legacy gateway was migrated by automatic refresh or direct snap refresh, where no target user is available.

Testing

• Checks appropriate to the affected code and behavior pass
• Unit tests added/updated (if applicable)
• E2E tests added/updated (if applicable)

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Architecture docs updated (if applicable)

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[elezar]

Under which conditions would we consider adding a breaking change to require a user to reinstall / migrate? Is this something that we can warn on and mark deprecated in some way?

[elezar]

Requesting changes because the advertised switch from an enrolled legacy system gateway to a user gateway fails. The headless system-service use case makes sense, and the explicit gateway-mode setting is useful, but the transitions need to work with existing credentials.

1. Enrolled legacy users cannot switch to user mode

Before this PR, system-gateway enrollment copied only ca.crt, client/tls.crt, and client/tls.key into the user's Snap TLS directory. With this PR, selecting gateway-mode=user stops the system service and starts a user gateway whose wrapper generates full PKI in that same directory. Certificate generation sees three of the six required TLS files and returns "partial PKI state"; it does not provision the missing CA key and server certificate/key. The user gateway therefore exits after the working system gateway has been stopped.

Please handle the existing client-only bundle explicitly, preserve credentials as appropriate, and provide the necessary CLI re-enrollment when changing gateway identity. Add a transition test starting from an enrolled legacy installation rather than an empty user directory.

References: snap/hooks/configure:13-14, tasks/scripts/snap-gateway-wrapper.sh:22-31, and certgen.rs:689.

2. Global enablement starts competing gateways on shared hosts

Previously, one system gateway owned port 17670. The configure hook now globally enables the user daemon, so multiple Docker-capable users can each start a gateway with a different database and CA on the same default listener. One fails to bind, and its CLI credentials cannot authenticate to the gateway that owns the port.

Please define how the owning user is selected or how per-user endpoints are assigned, or explicitly constrain the supported setup. This also affects the installer's assumption that restarting all active user gateways means restarting just one gateway.

References: snap/hooks/configure:14, install.sh:1471-1474, and snapd 2.76 global enablement.

3. Local-install recovery instructions omit the required user selector

The old recovery command ran as root against a system service. The replacement runs "snap start openshell.user-gateway" unprivileged without a user selector. The minimum supported snapd 2.76 rejects this with "non-root users must specify service scope when targeting user services". Use "snap start --user openshell.user-gateway", update both the installation documentation and Snap description, and remove the packaging-test assertion that forbids this syntax.

References: docs/about/installation.mdx:160 and snapd 2.76 validation.

4. User Docker preflight can accept an unsupported endpoint

The new user-context "docker info" follows the user's Docker context, including rootless or remote endpoints. It can succeed even when the user cannot access the system Docker socket. The confined gateway does not inherit Docker CLI context selection, and the Docker interface grants access to /{,var/}run/docker.sock rather than the usual rootless socket. The installer can therefore pass preflight, install the Snap, and time out waiting for its gateway.

Please validate the supported system socket explicitly as the target user. This is lower severity than the failed mode switch.

References: install.sh:1270 and snapd Docker confinement rules.

Companion work for #3795

The tmachine Snap installer still targets openshell.gateway / snap.openshell.gateway.service, writes root-owned system configuration and unit overrides, and copies system TLS credentials. Explicitly setting gateway-mode=system after installation preserves its intended headless service model, but it also needs the new system-gateway service name. Its package builder must include the configure hook and remove the deleted connect-plug-docker hook reference. Fresh user-mode qualification should be a separate scenario using tmachine-owned Snap XDG paths and user service control.

Validation

The focused installer, gateway-wrapper, install/configure/post-refresh-hook, and packaging-asset tests passed against this head. Debian artifact staging was skipped on macOS. These are mocked shell tests; no real Snap E2E was run. Real snapd recovery, enrolled-user mode switching, and multiple active sessions remain coverage gaps.