Skip to content

Sill Viewer for Mac, and Rust hosts for Windows and Linux - #128

Open
Aileuxe wants to merge 3 commits into
Saffsanity:mainfrom
Aileuxe:mac-client
Open

Aileuxe wants to merge 3 commits into
Saffsanity:mainfrom
Aileuxe:mac-client

Conversation

@Aileuxe

@Aileuxe Aileuxe commented Oct 15, 2026

Copy link
Copy Markdown

Two additions that let Sill go beyond iPhone and iPad: a Mac client (show a window of one Mac on another), and Rust hosts for Windows and Linux that speak Sill's protocol, so the existing clients can stream from a PC. Main is merged in (one merge commit, not rebased).

Heads-up on the repo's conventions: the Mac client is Swift and Apple frameworks only, like the rest. The Rust hosts are a separate Cargo workspace in hosts/ with permissively licensed crates (rustls, x11rb, windows-rs, …), which goes against "Apple frameworks only, no third-party dependencies". If you'd rather not have them here, the Mac client stands on its own and I can split the PR.

Sill Viewer (macOS)

A new target, Sill Viewer, in iOSClient/Sill.xcodeproj. It reuses the iOS client as it is (StreamClient and its extensions, pairing, discovery, settings, the clipboard) behind small UIKit→AppKit shims, with new views in MacClient/.

  • Devices window: every paired Mac as a tile (its last picture, how it is reached: Wired, Wi‑Fi, Remote, Not paired, Offline), the unpaired Macs on the network, and Add a Mac… (address and code). Tiles or a list, sort, search.
  • One window per Mac, each with a client of its own, so several Macs stream at once. A Mac that asks to pair shows its code card in that same window.
  • Input: clicks, right clicks, drags; trackpad scrolling with its own phases and momentum (the host already stamps them as trackpad phases); every key by position (MacKeyboard, matching KeyStrokes.virtualKeys exactly: all 82, plus keypad Enter as Return); repeats; keys let go on focus loss. ⌘ shortcuts go to the other Mac while the stream has the keyboard (Stream › Send Keyboard Shortcuts, ⌃⌥⌘K, keeps them local); ⌃⌥⌘ chords stay with the viewer.
  • Cursor and menus: the local cursor takes the streamed Mac's cursor shape, and the remote pointer sprite shows when that Mac moves its own pointer. The streamed app's menus appear in the menu bar under the app's name, each fetched when it opens and pressed on the Mac (kinds 24, 25, 27).
  • Sizing and settings: Text Size fits the remote window to the viewer window (the iPad's Aa detents). The settings popover has the iPad panel's controls, plus Share Clipboard; the clipboard chip uses the system's Paste button.

Shared iOS files change only by import switches and #if os(macOS) branches, with one exception that also helps on iPad: each StreamClient wrote the whole saved-Macs list, so a second window or client working from an older copy could drop a pairing. Clients now reload the list when another one in the process writes it (storeSavedMacs / savedMacsChangedElsewhere).

Verified (Mac, Xcode 27.0/27.1):

  • Debug and Release builds of the viewer; the iOS app with CI's command (generic/platform=iOS Simulator, arm64, unsigned): both succeed, only the known StreamClient warning.
  • Against SillHost --synthetic on loopback (software encoder, test pasteboard), main's host included:
    • streaming at 55–60 fps with 0 ms frame age;
    • the hello as "‹Mac› (Mac15,14), Sill 0.5.1 (4)", and the clipboard subscribed;
    • two sessions to two hosts at once;
    • the menu mirror against Scripts/menufixture.swift: its menus listed in the viewer's menu bar.
  • The key table round-trips every usage through the host's table, and an NSEvent harness passes 11 of 11: chords, sides of modifiers, Caps Lock, keypad text, release-all.

Not tested: clicks and typing driven from the viewer's own UI (no UI automation was allowed here); the TLS home door and pairing from the viewer against a real Sill.app; copying from the remote Mac into this Mac's clipboard (kept off this machine's real clipboard).

Rust hosts for Windows and Linux (hosts/, work in progress)

  • sill-protocol: the wire format.
    • The header and kinds 0–28; JSON shaped as Swift's Codable output, checked against payloads recorded from SillHost.
    • Video: ParameterSets, hvcC framing.
    • TLS 1.3 with pinned self-signed P-256 keys, ALPN sill/1 / sill-pair/1, unpaired keys refused in the handshake.
    • Pairing: Damm codes, PBKDF2 600k, HMAC proofs (checked against Python's hashlib), recognition tags, sill://pair links.
  • sill-host-core: the host without an OS, behind one Platform trait.
    • Doors: the plain dev door and the TLS door with pairing (ask, code, link).
    • The remote door: kind 18, away quality, byte pacing.
    • Kinds 16/17 settings, pings, ticks, eviction, goodbye.
    • Kind 26 pointer and kind 14 cursor.
    • An FFmpeg-subprocess HEVC encoder (libx265, VAAPI, NVENC; never linked).
    • A synthetic backend.
  • sill-host-windows:
    • Windows.Graphics.Capture.
    • Media Foundation HEVC (hardware first, software fallback).
    • SendInput by scancode.
    • Win32 windows with Alt-Tab filtering.
    • Start Menu apps, icons, PrintWindow thumbnails, cursor shape, DnsServiceRegister.
    • Per-monitor DPI.
  • sill-host-linux (X11):
    • XComposite + MIT-SHM capture.
    • XTest input.
    • EWMH windows, .desktop apps.
    • XFixes cursor.
    • Avahi over D-Bus.
    • A Dockerfile with Xvfb for testing.

Verified:

  • cargo test --workspace: 116 tests pass; clippy clean.
  • Core against Scripts/sillclient.py:
    • plain door: settings and their refusals, window picks, input placement, flood and junk handling, the not-draining eviction, goodbye;
    • TLS: pairing ask/code/link, a pinned session after a restart, refusals;
    • the remote door's away quality through sillrelay.py at 2 Mbit/s and +150 ms.
  • Core against a Network.framework client built from StreamProtocol's RemoteTLS: pairing, pinning, kind 18 verification.
  • The Mac viewer (Debug) against the synthetic core: 104–108 fps at 120 fps, 0–12 ms frame age.
  • Linux in Docker (Debian 13, Xvfb, openbox):
    • self-test 32/32;
    • input end to end 15/15: pointer placement against xwininfo, clicks, wheel, keys, ⌘C as Ctrl+C, text with é/€/ж;
    • streamed to the Mac viewer at 33 fps (1080p x265 on the container's CPU), a busy window at 58.8 fps.
  • Windows:
    • cross-builds to an .exe (x86_64-pc-windows-gnu); 45 unit tests;
    • clippy clean for both Windows targets.

Not tested:

  • Windows: nothing has run on Windows; hosts/NOTES-windows.md has a 15-step checklist.
  • Linux: no real Linux desktop, VAAPI or NVENC.
  • Wayland: not built; the plan is the portals.
  • Apps: the iOS app has not been run against a PC host.

Decisions open (in the NOTES files):

  • Pairing default: should the PC hosts default to the TLS door with pairing, as Sill.app does? They default to the plain door like SillHost does now, which Release builds of the apps refuse.
  • Keyboard mapping: ⌘ and ⌃ both become Ctrl, and ⌥ becomes Alt; right ⌥ is left Alt, not AltGr.
  • Resolution cap: 4096 px per side on Windows.
  • Wayland: only the window picked in the portal, plus the Desktop.

Builds: Scripts/make-app.sh is unchanged. For the viewer, pick the Sill Viewer scheme and set a team (it uses me.saffer.sill.viewer). For the hosts, see hosts/README.md and each NOTES-*.md.

🤖 Generated with Claude Code

Aileuxe and others added 3 commits October 15, 2026 20:15
A macOS target in iOSClient/Sill.xcodeproj that shows a window of one Mac on
another, used with the second Mac's mouse, trackpad and keyboard. It reuses
the iOS app's client unchanged where it can (StreamClient and its extensions,
pairing, discovery, settings) behind small shims, with new AppKit/SwiftUI
views in MacClient/:

- A Devices window: every paired Mac as a tile (its last picture, how it is
  reached), the unpaired Macs on the network, Add a Mac.
- One session window per Mac, each with a client of its own, so several Macs
  stream at once. Pairing by typed code in the same window.
- The stream: mouse, right click, drags, trackpad scrolling with its own
  phases and momentum, every key by its position (MacKeyboard, matching the
  host's HID table), the streamed Mac's cursor shape, the streamed app's menus
  mirrored into the menu bar, Text Size fitting the remote window.

Shared iOS files only gain import switches and #if os(macOS) branches, plus
one fix that matters on iPad too: clients of one process now reload the saved
Macs another one writes, so a second window can no longer drop a pairing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Cargo workspace in hosts/ speaking the same protocol as Sill for Mac, so
the existing iPhone, iPad and Mac clients connect to a PC:

- sill-protocol: the wire format (kinds 0-28, hvcC framing), TLS 1.3 with
  pinned self-signed P-256 keys, identities and pairing.
- sill-host-core: the host without an OS (doors, sessions, pacing, settings,
  pairing and trust list, an FFmpeg-subprocess HEVC encoder, a synthetic
  backend) behind one Platform trait.
- sill-host-windows: Windows.Graphics.Capture, Media Foundation HEVC,
  SendInput, Win32 windows, DNS-SD. Builds to an .exe; not yet run on Windows.
- sill-host-linux: X11 (XComposite, MIT-SHM, XTest, EWMH), FFmpeg (VAAPI,
  NVENC, libx265), Avahi. Verified end to end in Docker with Xvfb.

Each crate's NOTES-*.md says what is verified and what is not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Upstream main had moved 112 commits (the clipboard, hardware keyboards,
Larger Picture, restoring sized windows, reconnecting after sleep, the
Settings window's height). Only StreamClient.init conflicted: both sides
added an observer there, both kept.

For the Mac viewer: the Mac target now compiles StreamClient+Clipboard
too, through shims for UIPasteboard, the app's activation notifications
and the panel's VoiceOver check; the iOS ClipboardChip is iOS-styled, so
the viewer has MacClipboardChip (the system's Paste button, Dismiss, the
note) and a Share Clipboard switch in its settings popover. The Mac
target's project IDs moved to an unused range (5A, 5D, 5F), since main's
HardwareKeyboardPolicy took AC01/FC01.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Saffsanity

Copy link
Copy Markdown
Owner

From a pass over all the open pull requests (nothing is decided yet): #66 (with #72 and #73 built on it) is another macOS client also named Sill Viewer, sharing the same StreamClient files through a different platform layer (ClientPlatform and HEVCSampleFeed in a SwiftPM target, against this PR's #if os(macOS) branches and UIKit-named stand-ins), so at most one of the two designs can land and the name needs settling either way, and #158 makes the same files compile for visionOS a third way. Calls that #93, #94, #99, #107 and #117 add to those shared files would also need shims or guards in this viewer's target (its display view has no keyframe recovery, and jpegData(compressionQuality:) and displayView.snapshot do not exist on it); splitting the viewer from the Rust hosts, as you offered, would make the comparison with #66 easier.

addhumal added a commit to addhumal/sill that referenced this pull request Oct 19, 2026
…f canImport(UIKit)

Mac client targets (Saffsanity#66, Saffsanity#128) compile StreamClient.swift without UIKit's
protectedDataWillBecomeUnavailableNotification.
Saffsanity added a commit to addhumal/sill that referenced this pull request Oct 19, 2026
…ished): kept as above

9b59e0d and 34872a5 arrived on addhumal:goodbye-sleep while the commits above were made from
485149e. Merged, not rebased, and every conflict resolved to the commits above, whose tree this
is, because each change in them is done there in the way the maintainer's triage and task ask:
- 9b59e0d guards the protected-data observer for the Mac clients (Saffsanity#66, Saffsanity#128): that observer left
  with the background wording (7425327), so nothing needs the guard.
- 34872a5 closes each connection after the sleep and shutdown goodbyes and keeps a "quit" from
  following a "shutdown" (saidPowerOffGoodbye, set by willPowerOff): here every connection gets one
  goodbye, the first (StreamServer's `told`), the sleep's closes the connections, and the Quit takes
  restart or shut down from the quit's own Apple event, so a log out another app cancels sends
  nothing early and leaves no flag behind.
- It gives "sleep" and "shutdown" the quit's 3 s before a row listed at the goodbye is dialled, and
  a note on the Reconnecting line: here a Mac that said it went to sleep counts as asleep (Saffsanity#77's rule,
  its first dial 20 s later, Saffsanity#69's lines from then on, the goodbye's words until it), and after a
  shut down or a restart a row listed at the goodbye is never dialled, only one listed afresh, so a
  sleep proxy's record is not dialled 3 s after the Mac went to sleep. The note grew from the
  background wording's, which is out of this pull request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dhirensham

Copy link
Copy Markdown

Expanded on this with Aileuxe#1
This adds a Windows UI-based host in addition to the initial console-based host. The new Sill.exe host provides a system tray icon that mimics the behaviour of the original Mac host's menu bar icon, tested on Windows 11.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants