[dr-apple]

A suggestion: the Desktop in full screen, with the Mac giving it a display of the device's own size, as Screen Sharing's High Performance mode does: the Mac's own screens dark, its sound heard on the device alone, and the picture in step with it.

What it does

On the device

• While the Desktop streams, a second tap on Desktop shows it full screen: no bars, thumbnails, key row or trackpad; the picture takes the whole screen. Two small buttons halfway down the trailing edge bring up the keyboard or leave full screen. iPadOS's status bar is hidden and its edge gestures are deferred (defersSystemGestures(on: .all)), so a touch at the top reaches the Mac's menu bar. Anything else streaming ends full screen. The full-screen picture is the layouts' (ZoomedStreamView and the pinch zoom, the Mac's key repeat, the locked Mac's cover).
• Smooth, in step with the sound (full screen only): each frame shows at its own time instead of as it decodes (PictureClock in HEVCDisplayView.swift, a host-clock timebase on the display layer): its stamp plus the offset at which the Mac's sound of the same stamp is heard (AudioPlayout.pictureOffset, published by AudioOutput); with no sound, its stamp plus the link's floor and jitter cover of the last 10 s (20 to 150 ms). On Wi-Fi a 25 fps video showed 21 to 23 frames a second with the sound about 165 ms behind; now the frames keep their cadence and picture and sound go together, for 100 to 170 ms more latency in full screen alone. Out of full screen every frame shows as it decodes, as before.

On the Mac

• The viewport carries a new optional Viewport.deviceDesktop. With it, the host makes a virtual display at the device's size (÷ Aa's scale, 2x HiDPI, at least 320×240 pt) and makes it the main display while the Desktop streams: menu bar and Dock move there. A rotation or an Aa step makes it again at the new size. Another source, leaving full screen, the last device leaving, or Quit gives the main role back and removes it (DeviceDesktop.swift, the same private API and vendor ID as VirtualStage, its own product ID). WindowCatalog never takes one of Sill's virtual displays for the real screen.
• The curtain: the windows on the Mac's own screens are moved onto the device's display (through WindowSizer, read before the arrangement changes, each back to its frame afterwards), and each of the Mac's own screens gets a black window of Sill's at the shielding level, "This Mac is in use on a device.". No mirroring: mirroring the Mac's screens onto the virtual one, tried first, cost the stream its rate (0.7 Mbps carried with a 2560×1440 HiDPI screen among them) and ignored its gamma.
• The sound on the device alone: while a device plays the Mac's sound the Mac's own output is muted (MacOutputMute), and unmuted when none does, at Quit, or at the next start after a crash; only a mute Sill set is undone. ScreenCaptureKit takes each app's sound before the output device: a probe here caught the same peak with the output muted and not. The Desktop's sound is captured on the device's display while it lives, and AudioPipeline.renew starts it again when the Desktop's display changes (a capture on the other one ended with SCStreamErrorDomain -3815).

Compatibility

Additive only: deviceDesktop is optional. An older host ignores it and the device shows the Mac's real screen full screen (fitted); an older device never sends it. The device desktop needs the AppKit loop (Sill.app, or the CLI with --virtual-display) and is never made on a synthetic host, which also never mutes the Mac. On the hiding-the-bar decision in #92 (a Settings toggle with a shake back): the device half here is a second tap; it can move to that shape, or stay out and leave the host half to #312.

Tested

• swift build -c release, Sill.app via make-app.sh, the iOS app Debug and Release for the simulator and Debug for a device (Xcode 27.1), only the known warnings; Tests/checks/run-all.sh, all passing
• A synthetic host streaming the Desktop to sillclient.py, as before; the harness on an iPad simulator, in and out of full screen
• Live on an iPad Pro 12.9" (3rd gen) with Sill.app from this branch, a MacBook with a 2560×1440 screen: the device's display main, the Mac's menu bar usable from the top edge; 2 windows moved onto it and both Mac screens dark with the curtain, and back where they were after full screen; the Mac silent, the sound on the iPad; a 25 fps YouTube video smooth and in sync

Known, not done

• Before the curtain's fix, removing the device's display (leaving full screen) was followed by the Mac's screen locking and unlocking 2 to 3 s later, three times in the log (unlocked by Apple Watch, it seems); not seen since (the last removal, with the windows going back, locked nothing), and not looked into.
• No pure check for DeviceDesktop or PictureClock; no docs or plan file; nothing on an iPhone or with two devices at once.

Happy to adjust anything.

🤖 Generated with Claude Code

[lepeos]

Thanks for this. Your host side (DeviceDesktop, the StreamCoordinator and WindowCatalog changes, and Viewport.deviceDesktop) is carried in #161, unchanged, as a separate commit under your name. #161 uses it for Immersive on the iPad: with a hardware keyboard and the Desktop streaming, the viewport asks for the device's own display, so the Desktop fills the screen with no letterbox. #161 doesn't take the device-side second-tap full-screen mode. If #156 lands first, that commit rebases away and #161 just depends on yours.

[dr-apple]

Thanks for the heads-up, and for carrying the host side unchanged. That works for me. I've split my commit into a host-side one and a device-side one (the second-tap full screen), so the device side rebases cleanly onto #161 once it lands. I'll keep #156 open until then and, if #161 goes in first, rebase it down to just the device-side commit.

[Saffsanity]

From a pass over all the open pull requests (nothing is decided yet): #161 carries this PR's host half unchanged and replaces the second tap on Desktop with its Immersive mode, conflicting in StreamScreen.swift, so the device desktop should land once, from whichever goes first, with this PR reduced to its device half as you and #161's author already agreed. #66 (and #72 and #73, built on it) replaces the catalog's display rule (its realDisplays and desktopDisplayID against this PR's Sill-display filter and the display it makes main) and picks the Desktop's display by a new message rather than by moving the main display, so the second to land reconciles the two rules; #158 gives each independent session its own coordinator (each would own a DeviceDesktop), #150 wraps the landscape pane in a zoom that would not reach this PR's full-screen pane, and #155's iOS 16 target would reject the new two-parameter onChange.

[dr-apple]

Thanks for going through all of them. Agreed on landing the device desktop once: #156 is now split into two commits, the host side (same as in #161) and the device side (second tap), so whichever goes first, I'll rebase this PR down to the device commit.

On #155: the new onChange(of:) line in StreamScreen.swift is now a .task(id:) keyed on the flag, so it needs no two-parameter closure and doesn't depend on iOS 17. I haven't built it locally yet, so I'd treat it as untested until CI or you run it.

For #66, #72, #73, #158 and #150, I'll reconcile with whichever lands first rather than guess now. Happy to adjust the Sill-display filter or the full-screen pane if you tell me which direction you prefer.

[dr-apple]

Update on the build: I have now built it. Debug and Release for the iOS Simulator and Debug for a device (unsigned, Xcode 27.1) all succeed, with only the known StreamClient.swift capture warning, and Tests/checks/run-all.sh passes in full. I have not run it on a physical iPad or iPhone yet, so the viewport behaviour on entering and leaving full screen is untested on a device.

[Saffsanity]

The decision on hiding the bar is on #92: a toggle in Settings on the device, with a shake of the device bringing the bar back. A second tap on Desktop does not fit that shape, so the device half here would change to it, or stay out.

[lepeos]

A heads-up, since you were holding #156 for #161: I've closed #161 in favour of #197 for hiding the bar, per Noah's decision on #92. Your host commit is now carried unchanged, under your name, in #312, which sits on top of #197. The one conflict with main is in StreamCoordinator, where the Desktop's capture moved into desktopCapture(). I resolved it by giving desktopCapture an optional display (on:), and your device display is passed through it. Please check that the resolution reads right to you. Whichever of #156 and #312 lands first, the other rebases down as we agreed.

[dr-apple]

Thanks, I checked #312. DeviceDesktop, WindowCatalog and Viewport match my commit line for line, and passing the device display through desktopCapture(on:) reads right to me: Sill's windows stay out, the size and scale come from the virtual display, and skipping desktopLine for it makes sense.

One thing in the retry: startDesktopAgain calls desktopCapture() without the device display. If the first start fails while the device desktop is already live and main, the retry streams the Mac's own screen, but the menu bar and Dock sit on the virtual display, and currentSourceRect/pointerGeometry still map input onto the device desktop's frame, so touches land off the picture. Passing it through there too keeps capture and input on the same display:

guard !shuttingDown, let next = await desktopCapture(on: deviceDesktop.isLive ? deviceDesktop.scDisplay : nil) else {

(or release it before the retry, if you'd rather fall back to the real screen).

I've merged main into #156 (43f38f6) with the same resolution as yours plus that line, so whichever lands first carries it. The merge also brings the full-screen picture in line with main's (ZoomedStreamView, the Mac's key repeat, the locked Mac's cover). It builds (the package, the iOS app Debug and Release) and Tests/checks/run-all.sh passes; I'm testing it on the iPad next.