Repository navigation
Sill Viewer: a Mac controlling another Mac - #66
Open
ashleigh-hopkins wants to merge 75 commits into
Open
ashleigh-hopkins wants to merge 75 commits into
ashleigh-hopkins wants to merge 75 commits into
Conversation
…s its own The Mac client (Sources/MacClient, product SillClient, Scripts/make-client-app.sh): the iOS session code compiled unchanged from iOSClient through Shared/ symlinks, with ClientPlatform and HEVCSampleFeed shared by both apps. Each host window opens as a borderless window of its own, streamed at once (side streams, kinds 29–31), with a handle for this Mac's controls, the remote app's menus in the menu bar, a Dock icon per remote app, the clipboard both ways (kind 32), sound (kind 33) and files dropped in and dragged out (kinds 34–37). Input: pointer and all three buttons, trackpad scrolling with momentum, keys, ⌘Tab and ⌘Space when captured, gestures as the host's shortcuts. The host keeps those windows on one shared hidden virtual display (DeskStages), sized to the viewer's largest screen, taps the shown apps' sound and mutes it there (AppAudio), and takes and gives files (FileDrops, DragOut). Every addition goes only to a device that asks or a host that says it takes it; the iOS app is unchanged on the wire. The iPhone build from Xcode 27.0 can leave the 27.1 hinge API out (SILL_NO_DUO_SDK). New checks: mac-keys and main-router; protocol, compatibility, menus and pointer-control know kinds 29–37.
…wer window Two virtual displays with one vendor and product ID are one monitor to macOS, which takes the older offline when the newer comes; that, not a hardware limit, is why windows on displays of their own fell back to the host's screen. Each window's display now has its own identity from a pool (product 0x100 + slot, a serial kept per slot, so macOS remembers how to show each), and the shared display is gone: a window going full screen takes only its own display, never the other streamed windows. While a window is full screen, its display takes the size of the device's window live (VirtualDisplay.resize keeps the display and its ID), so the full-screen Space fills the viewer window and nothing more; leaving full screen gives the display back its size.
A display fitted to a full-screen window is smaller than it was made, and the next placement read that as too small and made it again, which threw the window out of full screen and round again (black, the sound skipping). A display is reused up to the size it was made at, and a window is fitted from wherever it streams: to the device's window as the main stream, to the size it had before full screen beside it. Each window's display is 2× again unless the window itself is larger than a 2× display can be: the device's widest screen made every display 1× before, soft on a Retina screen.
…de once, back to 2× A host file dragged out of one Sill window and dropped on another is dropped on the host from where it is (FileDrop's new `offer` and `window`): a real drag of the real file, so Finder moves it as it would there, and nothing goes to this Mac and back. Every Sill window takes drops: files from this Mac on a window beside the streamed one land in that window. A window's display was made twice: its first size was the device's screen, the placement then asked for that plus room for the menu bar and made it again. It is made once, at the placement's size. A window larger than a 2× display gets a 1× one, and once it is 5 % under the 2× size again it goes back to 2×.
… windows A full-screen window is its display's size, and that size was taken as the window's: the display was made bigger, which threw the window out of full screen and round again. A window full screen (by Accessibility's word, from the first placement) is sized by the size it had before, and fitted to the device's window only by a size its display can take. The viewer opens no app's untitled strips by itself (a browser's full-screen bar). A drag out of a window counts as leaving it when another window is under the pointer, not only outside its rectangle: a drop on a Sill window lying over the first one went back into the first.
A window full screen on its desk display cannot be moved home, and when the display went its Space went to another of Sill's displays, out of sight: a browser looked closed and would not come back after a quit. Leaving the desk now takes it out of full screen first (1.5 s at most), at Quit too.
Only windows on screen were listed, so a window in another Space (a full-screen one, another desktop) could not be reached from a device: a browser stuck full screen on a display out of sight looked gone. Titled windows of regular apps out of sight are listed as well, from the full list the catalog already fetched for its staged windows (the log line for those is gone with it).
…background The streamed desk window going into or out of full screen is seen by a 0.25 s look at it, where the catalog's 2 s poll left the full-screen window at the wrong size for seconds. A full-screen window shown beside the streamed one keeps the size its display was fitted to: it fell back to its size before full screen, a smaller picture, as soon as another window was clicked. The viewer logs each step of a drop on one of its windows.
A right button whose up never reached the host stayed down there: every move was its drag and left clicks read as right clicks until the next right click. A button press now lets go of any other button the host still holds first, and says so in the log.
docs/mac-client-plan.md is the design and justification as the branch stands (the wire, Sill's desk, sound, files, security notes, what was verified, what is open); CLAUDE.md's entry and Layout and DEVELOPMENT.md's section follow it. The protocol check covers every new payload (round-trips and refusals, 209 checks, 5 new mutants, 28 of 28 caught). A drag offer serves only the device it was made to, and a file's parts are taken only from the device that started it; both are dropped when that device leaves (an open file handle and the drag watch's timer outlived it). The viewer's menu loses Virtual Display, and a Mac viewer's windows always go to Sill's desk rather than the iPad's staging. The desk display's functions lose the "shared" names left from the shared-display phase. One compiler warning fixed.
A folder dropped on a Sill window, or dragged out of one, goes as one zip (FolderZip: ditto, the archive Finder makes, with resource forks, extended attributes and symbolic links) and is unpacked where it lands: on the host beside the dropped files, here where the promise is fulfilled. A folder dragged out onto the Trash goes to the host's Trash whole. The protocol check zips and unpacks a folder with a nested file and a link (215 checks).
…Sill windows Between two windows of one size the router tells the new stream apart only by the host's word that it switched, which reached it on the main thread, after the new stream's first sets could: the new window's picture went into the old one and stayed while nothing there redrew (two Finder windows swapping). The window list's source now marks the switch on the network queue, in the stream's own order (MainRouter.listed; main-router 21 checks, 12 of 12 mutants). A drag out allowed no operation inside Sill Viewer, so another Sill window was never offered the drop (no +): it allows copy and move there now, and the other Mac drops its own files.
…o longer takes them) NSImageView registers for file and image drags and, not editable, refuses them; as the deepest view under the pointer it took every drag meant for the window's DropContainer, so another Sill window never offered a drop.
A drop used to reach the other Mac only where it was let go: its app saw no drag, so it could not show where the files would land (a folder, a web page's drop zone, the address bar), and this Mac's modifier keys did nothing. Now, to a host whose list says `liveDrag`: - The host's own files, dragged out of one of its windows: its drag stays under way, the button held. Over any Sill window its pointer follows onto that window (each on its own desk display); let go there, it lands as a drag of its own (Finder moves, ⌥ copies, ⌥⌘ makes an alias). Let go anywhere else (this Mac's Finder, the Trash), the host gives its drag up with Esc. - This Mac's files over a Sill window: sent at once, the host starts its own drag of them under the pointer as they arrive and follows it; dragged away for 0.35 s, given up and the copies removed. Host: LiveDrags (through the coordinator's deliver: the injector's held buttons and keys, the modifier keys pressed as the device holds them), FileDrops' panel and received files split out. Viewer: LiveDrop, the stream view's and DropContainer's drag hooks, the upload split from the drop. A host without it gets kind 35 at the drop, as before. Checks: protocol (LiveDrag round-trips, the list's flag), compatibility, menus and pointer-control (38 taken, 39 unknown; their mutants that renumbered a kind onto 38 use 39).
…n the Dock - The remote app's icon carries Sill Viewer's in its corner, so it never reads as this Mac's app; written again when the icon arrives after the Dock icon started (it used to stay plain). - The Dock menu marks the windows as macOS marks an app's own: ✓ the app's front window, ◆ one minimized; it follows focus and minimizing. - Kept in the Dock and clicked with nothing open: the copy knows its app (SillRemoteApp in its Info.plist), opens Sill Viewer if need be and asks for the app, whose windows on the other Mac open here, or which is launched there when it has none. Docs: the plan and CLAUDE.md name kind 38 and the live drag, its verification and what is open.
ashleigh-hopkins
marked this pull request as draft
October 12, 2026 16:06
Beside the streamed window no fit is asked; the fallback to the size before full screen was taken before the check that keeps a fitted display, so the display shrank and the video filled a quarter of its window, pinned to a corner, whenever another Sill window took focus.
…ts windows, the icon kept - Kind 39 (DockMenu, `dockMenu` in the list): the host reads an app's own Dock menu (Finder's New Finder Window…, a chat app's New Chat) through Accessibility, which has it only while the Dock shows it, so it shows for about half a second there; read without the app's windows and the Dock's own items, and pressed by its place and title. The Dock icon here adds it under its windows, read again at most every 20 s as its menu opens. - Quit from a Dock icon closes that app's windows here (the app runs on there), where it left them open. - The badged icon is also the copy's own (AppIcon.icns, signed again), so one kept in the Dock keeps its icon while not running instead of a blank one. - A full-screen window in the background is captured from its display: an app reports its full-screen window far wider than its Space, and captured alone the video was a quarter of the picture, in a corner. Checks: protocol (DockMenu round-trips), compatibility, menus and pointer-control (39 taken, 40 unknown).
ashleigh-hopkins
force-pushed
the
mac-client
branch
from
October 12, 2026 16:18
ce947f4 to
e294961
Compare
… not a count - A device asks for the sound compressed (SideStreamRequest.audioCodec "aac-eld"): the host encodes the tapped apps' sound as AAC-ELD, about 96 kbit/s at 48 kHz stereo instead of plain PCM's 1.5 Mbit/s, for about 15 ms more delay; a compressed kind 33 carries the decoder's cookie and is never read as plain samples (the channels byte's high bit). Sill Viewer asks for it unless `SillClient.plainAudio` is set; a host that cannot encode, or a device that asks for none, gets PCM. - Sill's desk takes a new window while its displays leave room for one more within a quarter of this Mac's memory (each display holds its own framebuffer, about 130 MB at the largest Retina size); display identities are taken as needed, no longer a pool of 16. Checks: protocol (the compressed packet, the codec asked for).
… the background; full-screen windows reachable after a restart - A button let go where the stream view never hears of it (it moved to another window while the button was held, as the host switched streams) went nowhere, and the other Mac kept it down: every later click read as that button's until it was pressed again (a left click acting as a right click). Every up in the app is looked at first, and one the view would miss is sent from there; a button the view thinks down that this Mac has let go of is let go of at the next move. - A press on a window in the background brings it forward and, still held once the other Mac streams it, goes on there: a drag out of a window in the background starts with one press. - A full-screen window leaves full screen before it goes home and only then is its display removed (an app reports leaving at once, while its window is still the display's size: removed then, the Space outlived the display on the Mac's own, where Accessibility cannot reach it, and after a restart it streamed black and took no input). A window found in such a Space is reached by bringing its app forward, taken out of full screen, and moved to the desk.
…ached only for a pick, Dock menus only of apps - The windows the viewer asked the host to keep live were read from AppKit's list while a window closed, the closing one still in it: the host kept that window on its desk and captured it, the sharing indicator on, and it came back at the next launch. Only windows still open are asked for and kept. - A window in another Space is reached by bringing its app forward only for the window a device picked, at most once a minute (it fired for a side stream's window in another Space and brought its app forward on the host by itself). - An app's Dock menu is read from its own Dock item only (AXApplicationDockItem): a folder or stack of the same name gave the Dock's own menu (Sort by, Display as).
…n helper windows left out; Dock names - A minimized window is taken out of the Dock before it moves to the desk (moved while minimized it never showed there: "the window vanished while being staged"), and minimized again going home. - A picked window in another Space (another desktop) has its app brought forward first, which shows that Space, so it can be moved from it; full screen there, it leaves full screen first. Once a minute a window at most; never for a minimized window or a side stream's. - A window that could not come to the desk and is not on screen is captured on its own, never cropped from its display "with its app": that showed whatever lay there (another app's window) instead. - Windows above every display (an app's hidden helper at y -360) are no longer listed. - A Dock icon is named after its app: the window's app name, else the installed app's, never its bundle ID while either is known.
…t showed the bundle ID, the copy's file name)
…an app's menu; the ◆ it ignored)
…pp with no window gets one - The Desktop source took the first display ScreenCaptureKit listed, which could be one of Sill's hidden desk displays (the Desktop showed a window staged there, not the Mac's screen). It is one of the Mac's own displays now (never VirtualDisplay's vendor), the main one unless a device picked another: the window list names them (`displays`, `desktopDisplay`) and kind 40 (PickDisplay) picks one, the Desktop restarting on it. Sill Viewer lists them under Desktop in its menu bar icon. WindowCatalog's preferMainDisplay, which kept the iPad's Virtual Display off the Desktop, is gone: the rule covers it. - Opening an app that is running with no window (Finder with none open) did nothing: its Dock icon is clicked now, as a person would, which asks it for a window, and the new window streams. Checks: protocol (the displays and PickDisplay), compatibility, menus and pointer-control (40 taken, 41 unknown).
… in a Space - Sill Viewer shows each of the host's own displays as a Desktop window of its own (the menu bar icon lists them). The focused one streams as the Desktop on its display (PickDisplay, then the Desktop); the others stream beside it as side streams, a display's ID with the top bit set where a window ID goes (DesktopWindow), which the host captures whole. Drops and live drags on a Desktop window land on its display. A host that lists no displays keeps the one Desktop window. - A window off screen is listed when it belongs to a Space (another desktop, a full-screen Space, minimized from one), any app's, a menu bar app's too (its minimized window): an app's own hidden windows belong to none (a chat app's hidden helper, which no position test told apart: it moved onto the screen). SkyLight's answer, read-only, looked up at run time.
addhumal
added a commit
to addhumal/sill
that referenced
this pull request
Oct 19, 2026
…f canImport(UIKit) Mac client targets (Saffsanity#66, Saffsanity#128) compile StreamClient.swift without UIKit's protectedDataWillBecomeUnavailableNotification.
Saffsanity
added a commit
to addhumal/sill
that referenced
this pull request
Oct 19, 2026
CLAUDE.md: the Current step entry for this pull request (what was kept, what replaced the re-pick, kind 44, the rule, the verification, the note for Saffsanity#107 and Saffsanity#66, and what is left for a device); the Layout (kind 44, the coordinator's answer, WindowSizer's CommandOutcome, KeyframeRecovery, HEVCDisplayView's checkLayer, SILL_TEST_STILL_AFTER, the fixture's Slow Close window and the close step, sillclient's --keyframe@T, the check); Build and run (the still host, --keyframe@T, -SillLayerFail); and kind 44 among the compatibility floor's kinds, empty and skipped by a host from before. docs/DEVELOPMENT.md: the picture's recovery under "If the picture freezes" (the host's line, the rule, an older Mac), the lights and Close's timeout after "Windows sized for a device", the menus' test app's Slow Close window and the close step (and that menu-check needs an unlocked screen), -SillLayerFail in the debug harness, the still host among the CLI's test variables, and --keyframe@T beside the test client's flags. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Saffsanity
added a commit
to addhumal/sill
that referenced
this pull request
Oct 19, 2026
…ished): kept as above 9b59e0d and 34872a5 arrived on addhumal:goodbye-sleep while the commits above were made from 485149e. Merged, not rebased, and every conflict resolved to the commits above, whose tree this is, because each change in them is done there in the way the maintainer's triage and task ask: - 9b59e0d guards the protected-data observer for the Mac clients (Saffsanity#66, Saffsanity#128): that observer left with the background wording (7425327), so nothing needs the guard. - 34872a5 closes each connection after the sleep and shutdown goodbyes and keeps a "quit" from following a "shutdown" (saidPowerOffGoodbye, set by willPowerOff): here every connection gets one goodbye, the first (StreamServer's `told`), the sleep's closes the connections, and the Quit takes restart or shut down from the quit's own Apple event, so a log out another app cancels sends nothing early and leaves no flag behind. - It gives "sleep" and "shutdown" the quit's 3 s before a row listed at the goodbye is dialled, and a note on the Reconnecting line: here a Mac that said it went to sleep counts as asleep (Saffsanity#77's rule, its first dial 20 s later, Saffsanity#69's lines from then on, the goodbye's words until it), and after a shut down or a restart a row listed at the goodbye is never dialled, only one listed afresh, so a sleep proxy's record is not dialled 3 s after the Mac went to sleep. The note grew from the background wording's, which is out of this pull request. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Saffsanity
added a commit
that referenced
this pull request
Oct 19, 2026
…aming one (#91) WindowList gains displays and desktopDisplay in pull request #66's shape (DisplayInfo: id, name, width and height in points as Double, main), so whichever lands second reconciles code alone. SourcePick is kind 6's pick with the Desktop's display inside its object, {"desktop":{"display":N}}; without one it writes StreamSource's own bytes, and v0.5.1's StreamProtocol reads it as the Desktop. No new kind. The compatibility check covers both keys, the pick both ways and #66's list, writes them for H4 (v0.5.1 read both), and gains seven mutants; the two mutant patterns that matched the old initializer follow it (here and in the clipboard check). The protocol check says kind 6 stays the pick. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HunterAhlquist
pushed a commit
to HunterAhlquist/sill
that referenced
this pull request
Oct 19, 2026
Kind 29 was free on 985a733, where the pen was built; since, main's comment and checks keep 29 to 31 free (29 the audio plan's, 30 and 31 Saffsanity#66's side streams), 32 is the clipboard, and open pull requests hold 33 to 44 (Saffsanity#66's 33 to 42, Saffsanity#94's 43, Saffsanity#93's keyframeRequest 44, checked against every open head today). So `case pen = 45`, and everything that names the number follows: SessionLink.isInput (8 or 45), the comments of Pen.swift, Switcher, PenTablet, InputInjector and StreamServer (which match the enum, not the number), sillclient.py's --raw45 and its sends, and the docs (DEVELOPMENT.md, CLAUDE.md's entry, Layout and Build and run, the plan's design and gates, with a note that its results before the rebase say 29). The compatibility floor's kinds are 0–28, 32 and 45 again with 29 to 31 free as main has them; kind 26's `seen` counts kinds 8 and 45. The checks: compatibility names kind 45 and 46 unknown (157 checks; its mutant "kind 45 as 46"), protocol keeps main's two kind checks and has the pen at 45 with 46 unknown (193 + 8), pointer-control's table and sweep say 45 (158), fence's input and its two C10 mutants say 45; menus' line and mutant and pointer-control's two renumbering mutants are main's again, 29 being free once more. clipboard passes unchanged. Run here: compatibility 157, protocol 193 and the 8 cross-checks, pointer-control 158, menus 309, clipboard 259, pen 197 and 5,000 random sessions, all passing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HunterAhlquist
added a commit
to HunterAhlquist/sill
that referenced
this pull request
Oct 19, 2026
While a Pencil hovers as a pen, the device's arrow shows in the Mac cursor's shape only when that cursor is an app's own (a drawing app's brush outline), never over a standard arrow, I-beam or hand. Kind 46 `cursorKind` (CursorKind), host → device, just before each kind 14; the host's CursorClass decides. Re-applied on pencil-pressure-rebased: its kind moved from 30, which Saffsanity#66 holds, to 46, the first number no open pull request holds after the pen's 45. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Saffsanity
added a commit
to Code-Tomato/sill
that referenced
this pull request
Oct 20, 2026
Any device, from home or away (and at home any device on the network while Require pairing is off), could turn Send Audio on for the whole Mac: every app's sound for the Desktop, calls included, played unmuted on every connected device. Share Clipboard is the Mac's alone for a like reason, and the PR's own author proposed the same for Saffsanity#66's mute-host. For the beta, only the Mac's menu and Settings, and the CLI's --audio, set it: - DeviceSettings.accepted refuses a device's sendAudio, on or off, from either door, and names it ("Settings from ‹device› refused: send audio (set on this Mac only)"); the rest of the change applies. - The device's Send Audio row shows the Mac's value, read-only (a disabled switch, as Direct Wireless's from afar), its footnote saying where it is set: "Turn on Send Audio in Sill on ‹Mac› …". - kind 17's sendAudio stays on the wire (never reused; a later decision may let devices set it again, as Saffsanity#66's Mac viewer asked). - away-quality: 12 new cases (refused on and off, home and away, named, the rest applied; a device still sees it; its log words) and five mutants (AQ20-24), 24 of 24 caught. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Saffsanity
added a commit
that referenced
this pull request
Oct 20, 2026
…exactly The plan's #173 section now says how the two differ (the pick's key beside or inside the Desktop's object, "Desktop 2" against "1 of 2", a pick without a display, a display gone, a pick during a restart, two quick taps), what backs each, and which should land: this branch, which does what Noah decided on #91 ("when there is another one it says so") and keeps devices from before on the main display. Rule 3 says a pick that named no display keeps meaning the main one through a change of the main display; the wire section says why desktopDisplay sits beside the list rather than as a flag on each entry (#66's shape). The checks README says which change brought the check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
westsoever
added a commit
to westsoever/sill
that referenced
this pull request
Oct 20, 2026
…nto main Merged locally from Saffsanity/sill pull/66/head (fe6c001) onto main (4d990e9). Conflicts: - WindowCatalog.findInstalledApps: main's InstalledApps.find (Saffsanity#76, Saffsanity#135) kept, with the PR's Finder from CoreServices added after it. - ci.yml: the mutants matrix is the union of both lists. - Tests/checks/README.md: main's key-strokes, installed-apps and remote-rules rows; the PR's protocol row. Fix after the merge: VirtualDisplay.resize called Runtime.applySettings, which Saffsanity#87 replaced with VirtualDisplay.applySettings(_:to:). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 20, 2026
Valdex
pushed a commit
to Valdex/sill
that referenced
this pull request
Nov 1, 2026
docs/coherence-plan.md: the decisions (PR Saffsanity#66's side streams per window, a native Windows frame, local moves, sill-data/1, clipboard generations 2 and 3, SMB with path mapping, pcm-s16le sound), milestones M0 to M6 with what done means, the kill criteria, and what a Linux container can and cannot verify. docs/coherence-protocol.md: every kind and field added, additive under Compatibility.swift's rules, Saffsanity#66's kinds 29-31 and 34-42 kept, 45 and up new. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AuUWHgK35sbA9hSxuzMiZV
Valdex
pushed a commit
to Valdex/sill
that referenced
this pull request
Nov 1, 2026
docs/coherence-protocol.md as types, for the milestones after M0;
nothing here sends or acts yet. Protocol/Coherence.cs: the capability
generations; kind 29's SideStreamRequest with the coherence fields;
kinds 30 and 31's window-ID prefix; kind 44's optional {"window"}
(empty unless the host says coherence 1); kinds 45 and 46 (WindowState
with its windows, roles and traffic lights; PlaceWindow); kind 15's
activate and restore; kind 41; kinds 34 to 38 as upstream PR Saffsanity#66 has
them (FileChunk's binary layout, the name cut at a character's edge
rather than a byte); kinds 48 and 49 and the sill-data/1 ALPN; kind 32's
generation 3 files; kind 8's window beside the event. Protocol/Audio.cs
mirrors Audio.swift (AudioFormat, AudioPackets, AudioMessage.Parse with
its caps) and names pcm-s16le. Payloads.cs gains the window list's
keyRepeat and the contract's capabilities, the hello's audio, coherence
and iconSize, kind 32's accepts, image, html and rtf, LaunchApp's
documents, and Send Audio's fields of kinds 16 and 17. Wire.cs names
kinds 29 to 49. The new types spell every JSON name out; nulls are left
out, and strings are written as Swift writes them (UTF-8 unescaped), so
an HTML clipboard does not pay six bytes for each angle bracket.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AuUWHgK35sbA9hSxuzMiZV
Valdex
pushed a commit
to Valdex/sill
that referenced
this pull request
Nov 1, 2026
A console program on plain net10.0 (not -windows, no packages) that compiles the client's files needing neither WinForms nor Win32 exactly as they are (Protocol/*.cs, Video/KeyframeRecovery.cs) and runs assertions in the spirit of upstream's Tests/checks: a line per check, the count, exit 1 on a failure. `dotnet run --project WindowsClient/Sill.Checks [group]`. Its groups: the 14-byte header against hand-made bytes and the kinds' numbers and caps; every payload docs/coherence-protocol.md adds, decoded from the contract's own examples and encoded back to the same JSON (names, types, nothing null written), and kind 2's capabilities with an older host's list and Saffsanity#66's Bool flags; kinds 30/31's window-ID prefix and kind 34's part against hand-made bytes; kind 43's packets against Audio.swift's layout and each of its refusals; KeyframeRecovery with the iOS check's cases and 2,000 random sessions. 194 checks pass. The first run caught one: WindowState.Window's RoleOrWindow was written into the JSON. It is [JsonIgnore] now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AuUWHgK35sbA9hSxuzMiZV
Valdex
pushed a commit
to Valdex/sill
that referenced
this pull request
Nov 1, 2026
DragDropHost takes a window's handle, a mapper to fractions of its picture and the Mac window's ID, so the Coherence windows can host it as the stream window does (ID null, the picked window). In: an IDropTarget on the handle; this PC's files go as kind 34 parts as soon as the drag comes over (an app's virtual files at the drop), the drag followed live as kind 38 (start, moves at most 60 a second and every 2 s at rest, files, drop or cancel) to a host that says liveDrag 1, else one kind 35 after both the drop and the last part; DROPEFFECT_COPY. Out: kind 36 while the left button is held here; when the pointer leaves the window, DoDragDrop with the Mac's files as MacFilesDataObject; dropped elsewhere, Esc and the button up on the Mac (Saffsanity#66's viewer); back over a Sill window of the session, the Mac's pointer follows and its own drop lands there; Esc with the button held lets the Mac's drag go on. LiveDragMachine and DragOutMachine are pure. The stream window wires the helper, attaches the session to ClipboardSync and FileService, and its activation is paste intent for copied files; its old activation-time clipboard code goes. Checks: live-drag (every phase order, older hosts' kind 35, 3,000 random drags), drag-out (each way a drag out ends). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AuUWHgK35sbA9hSxuzMiZV
Valdex
pushed a commit
to Valdex/sill
that referenced
this pull request
Nov 1, 2026
…he richer clipboard TEST ONLY. Its list says data 1, clipboard 3, fileDrop 1, liveDrag 1 and documents 1 (--no-data, --no-live-drag, --clipboard N for a host without them). To a hello that says coherence 1 it sends kind 48 and admits a sill-data/1 connection (its ALPN at a TLS door, its first message at a plain one) only with that ticket, the session's key, one per session, closed with it. Kind 34 parts are saved in %TEMP%\SillTestHost and printed with their SHA-256, in order or dropped as a Mac does; kinds 35 and 38 printed; kind 7's documents printed as OPEN. Keys from the client: F6 arms a kind 36 offer of generated files (one with a ':' and a '?', a folder zipped as ditto does, size 0 as Saffsanity#66 offers folders) for the next drag in the picture, with id 0 when the button comes up there; F7 offers them on the clipboard (kind 32's files), F8 a picture with HTML and RTF, F9 a concealed text. Kind 37 is answered with 512 KB parts on the bulk connection when it is up. A copy from the PC is echoed back by what its subscription takes: text, HTML, RTF, the image, and received files offered back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AuUWHgK35sbA9hSxuzMiZV
Valdex
pushed a commit
to Valdex/sill
that referenced
this pull request
Nov 1, 2026
…ntract The wire docs/coherence-protocol.md describes, which a C# client is written against: - SideStreams.swift, ported from pull request Saffsanity#66 (fe6c001): SideStreamRequest (kind 29), FileChunk (34), FileDrop (35), DragOffer (36), FileRequest (37), LiveDrag (38), DockMenu (39, for the wire only), WindowInput (kind 8 with a window), TakenBack (41), FolderZip (macOS only). Left out: Saffsanity#66's kind 33 sound (AudioPacket, the request's audio fields; main's sound is kind 43) and its clipboard images (ClipboardImage, ClipboardPasteboard). - StreamMessageKind: 29-31 and 34-42 with Saffsanity#66's meanings, 45, 46, 48 and 49 for coherence; 33 and 47 stay free. - Switcher.swift: Saffsanity#66's WindowList additions as Int generations (middleButton, sideStreams, fileDrop, liveDrag, dockMenu, windowInput, badges, displays, desktopDisplay, rebuildDesk) and coherence's (coherence, data, documents, iconSize), set after the initializer as keyRepeat is; DisplayInfo, PickDisplay, DesktopWindow; LaunchApp.documents; WindowCommand's activate and restore (a host before drops such a kind 15 whole). - Input.swift: the middle button and isButtonDown. - Coherence.swift: KeyframeRequest (kind 44's window), WindowStateMessage and its entries (45), PlaceWindow (46), DataTicket (48), DataHello (49), ClipboardFiles, the generations. Hello's coherence and iconSize, Clipboard's accepts, html, rtf and files, AudioCodec.pcmS16LE. Every new field is optional and left out when nil, so the old payloads' bytes are unchanged. The host's exhaustive switches take the new cases: the middle button is posted (InputInjector, DroppedInput and RevokedInput let go of it as of the others), activate and restore are refused until the host says coherence. Checks: coherence-wire (new, Foundation-only, runs on Linux) 103 checks and 21 of 21 mutants; protocol, compatibility, menus, pointer-control and clipboard follow the kinds now known, their renumbering mutants moved to free numbers; key-strokes gains the middle button (509 checks on Linux, the two new mutants caught). Type-checked with Swift 6.4 on Linux; compatibility (152), clipboard (340) and key-strokes run there with the Foundation-only files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AuUWHgK35sbA9hSxuzMiZV
Valdex
pushed a commit
to Valdex/sill
that referenced
this pull request
Nov 1, 2026
…ke-back Pull request Saffsanity#66's host side (fe6c001, against 2ecb639), ported onto main rather than merged: - New: SideStreams (kind 29 to 30/31, at most 6 windows, each its own capture and encoder at the knob held to its display's scale, CaptureScale, as main's main stream now is; a per-window SideWindowPipeline with a frame gate, the seam for coherence's focus rate and hidden windows), AppCrop and SharePill (a window with its menus and popovers), DeskStages (a virtual display per window, kept displays, identities by slot), DeskPicture, DisplayStreamProbe, FileDrops and DragOut (kinds 34-37), LiveDrags (38). - SideStreamPolicy.swift (pure, new): SidePlan, SideFrameGate, SideSend, DeskGeometry and DropFileName, the rules SideStreams, StreamServer, DeskStages, VirtualStage and FileDrops follow. - Hooks: StreamCoordinator (the requests, the desk and crop in select, the announced source, input over a window beside the streamed one, drops and drags, pick display, the desk's refresh, the picture check, display streams stopped, take-back and Stop Sharing, the exit), StreamServer (side frames by SideSend's budget, revoked clients skipped), VirtualStage (desk stages: identities, 1x, full screen fitted, spares, keyed emergency snapshots), VirtualDisplay (live resize, identities resting, removal in turn), WindowCapture, WindowCatalog (this Mac's own displays, the picked Desktop, windows in other Spaces), WindowSizer, HostShutdown, HostStatus, the menu bar. - Advertised: middleButton 1, sideStreams 6, windowInput 1, displays and desktopDisplay; fileDrop 1, liveDrag 1 and rebuildDesk 1 on a host that captures this Mac. An iPhone or iPad that asks for no side streams is served as before: the desk's waits, fresh listings, un-minimizing and full-screen teardown run only on desk stages (VirtualStage.forDesk); other Spaces are listed, the source announced, drags watched, Dock presses swallowed and a viewport kept across a pick only while a device asks; the exit's capture stops, display waits and longer watchdog only while the desk holds displays; WindowSizer's size-only match only for a window not drawn; the Desktop keeps preferMainDisplay. Left out: AppAudio (Saffsanity#66's kind 33; main's kind 43 serves), DockMenus (kind 39) with the badges and the Dock-icon reopen, Finder in the app list, HostClipboard's images (the clipboard generations come next), the menu's virtual display note. Hardened: a kind 34 part past its announced size dropped, a full disk no crash, a dropped name held to 200 bytes with no control characters, a live drag's events from a device no longer served dropped. Uncompiled here (no macOS SDK): every host file; all parse with swiftc -parse. Checks, on Linux: side-streams (new) 45 checks and 27 of 27 mutants, desk-picture (Saffsanity#66's) 27 checks and 12 of 12 mutants (its 3 grid mutants need CoreVideo), capture-scale 175 (its imports made conditional). The pacing and audio harnesses compile the new files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AuUWHgK35sbA9hSxuzMiZV
Valdex
pushed a commit
to Valdex/sill
that referenced
this pull request
Nov 1, 2026
DisplayWake's check enumerates every kind the build knows and wants each decided: PR Saffsanity#66's 29-31 and 34-42 and coherence's 45, 46, 48 and 49 are none of a person's (their device's clicks and keys are kind 8, which is), so the rule is unchanged and the count is 49. 212 checks, run on Linux. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AuUWHgK35sbA9hSxuzMiZV
Valdex
pushed a commit
to Valdex/sill
that referenced
this pull request
Nov 1, 2026
docs/coherence-plan.md gains "Port of PR Saffsanity#66": what was ported, what was left out and why (AppAudio, DockMenus and its badges, the clipboard's images, Finder in the app list, the menu's new note), how an iPhone or iPad stays served as before, and the riskiest uncompiled spots for the first Mac build. CLAUDE.md's Current step opens with Coherence M1a: what changed, what was type-checked and run on Linux, what is uncompiled. Tests/checks/README.md: the renumbering mutants' new numbers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AuUWHgK35sbA9hSxuzMiZV
Valdex
pushed a commit
to Valdex/sill
that referenced
this pull request
Nov 1, 2026
…side streams, desk, files) # Conflicts: # CLAUDE.md
Valdex
pushed a commit
to Valdex/sill
that referenced
this pull request
Nov 1, 2026
…eworks A harness that type-checks StreamProtocol, SillHostCore (package access, -package-name sill), SillHostCLI and SillMenuBar with Linux's Swift 6.4 in Swift 5 mode, against Swift stub modules of every Apple framework the host imports (AppKit, ScreenCaptureKit, CoreGraphics, CoreMedia, CoreVideo, VideoToolbox, AudioToolbox, Network, Security, CryptoKit, Accessibility, IOKit, SwiftUI, …), written from the frameworks' real signatures and checked against how upstream's compiled code (e4d3034) and PR Saffsanity#66 use them. C structs handed to C callbacks are a C module (stubs/c), so @convention(c) sees them as on a Mac. Darwin's Foundation-only API (CoreFoundation's types, the ObjC runtime, URLSession, DistributedNotificationCenter, ProcessInfo's activities) is a module imported into every file. stage.py copies each target with only what Linux cannot follow changed (os(macOS), submodule imports, @objc and #selector), lines and columns kept; selectors.py checks what @objc's removal hides. known.txt lists the 10 errors the stubs themselves cause (C-representability of Linux's CGSize and the stubs' CF classes, NSObject.init through a metatype, two FoundationNetworking and NSAttributedString differences). Everything else type-checks: 0 errors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AuUWHgK35sbA9hSxuzMiZV
Valdex
pushed a commit
to Valdex/sill
that referenced
this pull request
Nov 1, 2026
The README says what the run proves (the host's code consistent with itself and with the API it declares), how the stubs are held to compiled code (e4d3034 and PR Saffsanity#66's host type-check against them but for the listed limitations; Saffsanity#66's DockMenus wanted kAXURLAttribute), and what it cannot check: the real SDK, availability, Objective-C's own rules, C callbacks with class parameters, behaviour. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AuUWHgK35sbA9hSxuzMiZV
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Sill streams Mac windows to an iPhone or iPad. This adds Sill Viewer, a Mac client for the same host, so one Mac can use another Mac's apps as if they were its own: each remote window becomes a window here, with its menus, sound, clipboard and file drag and drop. It is built from Apple frameworks only and reuses the iOS app's session code (pairing, TLS, discovery, the reconnect) rather than a copy of it.
The design, the wire table and what was verified are in
docs/mac-client-plan.md.What a person gets
The wire
Everything is additive. Each new message is sent only to a host whose window list says it supports it, so the compatibility floor is unchanged:
middleButton)sideStreams)audio); its payload can be AAC-ELD for a device that asks (SideStreamRequest.audioCodec)fileDrop)liveDrag)dockMenu)displaysin the window list); a display goes where a window ID does with its top bit set (DesktopWindow)TakenBack), host → device, to Mac viewers onlyrebuildDesk), device → host, no payloadwindowkey: a scroll or hover over a window in the background (windowInput)badges: each listed app's Dock badgeThe iOS app sends none of them.
Host changes worth reviewing
DeskStages/VirtualStage: one hidden display per window, as many as fit in a quarter of the host's memory. Each display gets its own identity, because two displays that share a vendor and product ID count as one monitor to macOS, which then takes the older one offline. Each display is sized once to the viewer's largest screen at 2× (1× beyond 3792×2160 pt) and resized live for full screen; a full-screen window leaves full screen before its display goes. A display a window leaves is kept for the next window, and all are removed together once the desk is empty: every display made and removed leaves macOS's ColorSync service more work for the rest of the login session (enough churn saturated it and WindowServer's watchdog restarted it), and on macOS 27.0.1 a display made after another was removed can stream a third display's picture. No display is made for a window Accessibility cannot find, and a window that cannot be placed after all hands its display to the next; one the desk could not take is not tried again for 30 s when picked, 5 minutes beside it (DeskStages.mayTry).DeskPicture: the main stream's frames are compared with a small screenshot of the window alone, at start and every 10 s; a picture that is another display's is replaced by capturing the window alone. The pill's refresh button (kind 42) makes every display again from none, the one thing that mends that macOS bug.AppAudio: a CoreAudio process tap, AAC-ELD encoding, all off the main actor. Its calls wait on coreaudiod and on the audio-recording permission prompt, which once froze the host.FileDrops,DragOut,LiveDrags: offers are scoped to the device they were made to, and files received for a drag that never landed are removed.StreamCoordinator.takeBack(the host menu's Stop Sharing, an app brought forward on the host) andappActivatedHere, which ignores activations within 2 s of a device acting (a pick counts as acting when its app comes forward) and follows an app a device's action opened. Both only while every device is a Mac viewer.OutputSwitch: a retired pipeline's frames stop at the switch. The old encoder is drained off the main actor, and its last keyframe could otherwise reach devices after the next stream began.DockProxies,SillDockProxy): what an icon shows (window titles, the Dock menu) is in a 0600 file, never in a notification; a request is signed with an HMAC under the key Sill Viewer writes there at launch, with a counter that must rise, so it can be neither forged nor replayed.DockMenus: an app's own Dock menu is read through Accessibility, which only exposes it while the Dock shows it, so it flashes in the host's Dock for about half a second when read; only items it offered are pressed. Badges are read from the Dock items (AXStatusLabel) every 2 s while a Mac viewer is connected, nothing shown.MainRouter: the main stream goes to the window of the source the host last listed, decided on the network queue in stream order, never by what is focused here; a side stream is drawn only after its own parameter sets.InputInjectorand the viewer'sStreamInputView: a button's release that the viewer's stream view would miss (it moved to another window while the button was held) is sent from the viewer, and the host lets go of a button still held at a new press.Checks
Tests/checks/run-all.sh: every check passes.mac-keys(695 checks, 7 mutants),main-router(32 checks, 15 mutants),desk-picture(26 checks, 9 mutants).protocol: 240 checks, every new payload included, all 33 mutants caught.compatibility(111 checks, 25 of 25 mutants),menus,pointer-control(170 checks, 33 of 33 mutants),key-strokes.swift build -c releasepasses, and the iOS app builds on Xcode 27.0 with-D SILL_NO_DUO_SDK. Without that flag it fails only on the 27.1 Duo APIs, as main does; CI's 27.1 is the real check.Verified by hand, between two Macs on one LAN, each as host in turn
Not yet confirmed by hand
Known limits