[addhumal]

Closes #91.

Reworked from main (e4d3034), and merged with main at c4a331f (0.6.0, the locked Mac). This replaces the earlier version of this branch.

What it does

On a Mac with two displays or more, the Desktop button reads "Desktop 1", "Desktop 2"…, and tapping it while the Desktop streams moves it to the next display. The order is left to right, then top to bottom, as System Settings › Displays arranges them, wrapping round. Sill's virtual display is never one of them. With one display, or against an older Mac, the button looks and behaves as before. This is the tapping method Noah decided on in #91.

• Wire (additive): kind 47 nextDisplay (empty payload), sent only to a host whose window list says desktopDisplays 2 or more. WindowList.desktopDisplays and desktopDisplay (1-based). Kinds 45 and 46 are held by open pull requests (the microphone, the pen, the camera). Older hosts skip kind 47; older devices skip the fields.
• Host (WindowCatalog, StreamCoordinator):
  • The host works out the next display from its own newest look and keeps the choice by the display's UUID, so a new display ID after a sleep is still the same display. The device never names a display, so a stale ID can't reach the host.
  • Input and the pointer land on the display the capture runs on, not the one just chosen, so a tap made during the switch can't land on a screen the device isn't showing.
  • A Desktop whose display moves under it (the chosen display plugged in again, a new ID) restarts there. With nothing chosen, the Desktop stays on the display it is on.
  • If the new display doesn't stream (its start and the one more try fail), the Desktop goes back to the display it was on.
  • The Mac's pointer moves to the new display's centre.
  • One log line: "Desktop: display 2 of 3 (DELL U2723QE), asked by ‹device›."
  • A locked Mac (main's LockPolicy) lists no displays and refuses kind 47 as it refuses a window pick. The host checks the lock again before it steps and before it moves the pointer. lock-policy: 540 checks, 141 of 141 mutants.
• Device (StreamClient.tapDesktop(), StreamScreen, PortraitStreamScreen): only the Desktop buttons send kind 47. The device's own Desktop requests and gestures still pick the Desktop, so nothing steps by itself. VoiceOver hears "Display 2 of 3" and the hint "Shows the next display."
• DEBUG: -SillDesktopDisplays 3,2 puts the harness's mock on display 2 of 3.

Checks

• Tests/checks/desktop-displays (new, in CI's mutants matrix): 56 checks and 31 of 31 mutants. It covers the order, the UUID choice, the next display, where input lands, falling back after a failed step, the button's words and tap, and the pointer, with 5,000 random arrangements.
• compatibility: 159 checks and 36 of 36 mutants (the new fields and kind 47). display-wake: 199 checks and 62 of 62 mutants (kind 47 is not a person at the device, as a pick isn't). pointer-control: 160 checks and 33 of 33 mutants.
• run-all.sh: all 64 checks pass. swift build -c release: only the CaptureProbe warning. The three iOS builds (Debug and Release for the simulator, Debug for a device) show only the known StreamClient capture warning.
• Harness photos at 1000×710, 710×500, 710×1000, 466×678 and 320×568 (phone): "Desktop 2" fits on one line in every bar.

Tried by others

• @xiaofeng-han, on a Mac with three external monitors: "working well" (Desktop: cycle through the Mac's displays with the Desktop button (#91) #173 (comment)). A list to choose from, which they asked for too, is Hold the Desktop button for the Mac's displays by name (#273, on top of #173) #296.

Untested

• On a Mac with two or three monitors: stepping and wrapping, the label, the pointer moving, input landing on the shown display right after a tap, and a monitor unplugged and plugged in again.
• VoiceOver reading the value and the hint.
• A mirrored display (whether it shows up as another Desktop).

Problems found, and how they were fixed

1. A tap right after switching could click the wrong monitor. Choosing the next display moved Sill's idea of the Desktop at once, while the stream still showed the old monitor until it restarted. Fix: input and the pointer land on the display the capture runs on (DesktopDisplays.landing).
2. A monitor Sill can't capture would trap the Desktop. Nothing streamed, the choice was kept, and the button can only step while the Desktop streams. Fix: if the new display doesn't stream, the Desktop goes back to the one it was on (DesktopDisplays.afterStep).
3. Display IDs change across sleeps (2026-10-28: 21 became 53 and 52), so a choice kept by ID would be lost. Fix: kept by the display's UUID.
4. A choice could flip when macOS lists the displays in another order, restarting the Desktop on the other monitor. Fix: with nothing chosen, the Desktop stays on the display it's on.
5. A locked Mac would have shown its screens and moved its pointer. This came up merging main's locked Mac. Fix: a locked Mac lists no displays and refuses kind 47 as a window pick, and the host checks the lock again before it steps and before it moves the pointer.
6. The earlier version's findings (marliechorgan's, Desktop displays: follow the streamed display, number displays left to right (Saffsanity/sill#91) addhumal/sill#2). A pick that landed mid-switch was dropped, and monitors were numbered by ID. Fix: a second quick tap is ignored while a switch is under way, and the numbers follow the arrangement, left to right.

[marliechorgan]

Thanks for picking up #91. I ran this on a Mac with three displays: a MacBook Pro's built-in screen (main) under two identical BenQ MA270U monitors. Two things came up, and both are fixed in addhumal#2, which goes into this branch.

1. A pick that lands mid-switch is dropped, but the device is told it happened. The Desktop started on display 4, display 2 was picked, then display 1 50 ms later. The host streamed two BenQ pipelines (3840×2160) and never the built-in screen, yet its window list said desktopDisplay=1. Input and the pointer read the same catalog.display, so taps land on a screen the device isn't showing. It happened in 4 of 4 runs. The cause is that pickDesktop moves catalog.display at once (WindowCatalog.swift:99), while select's defer drops the queued .desktop because pick != active is false (StreamCoordinator.swift:1237).

2. The numbers follow display IDs, not positions. Here the right-hand BenQ (macOS's "(2)") is ID 2 and the left-hand one ("(1)") is ID 4, so the button calls the left monitor Desktop 3, and tapping steps right, then left (WindowCatalog.swift:278).

The linked PR has the repro (a desktop:ID pick for sillclient.py), a fix for both, and checks built on this layout. Under the ID order those checks fail. With #173 and the fix merged onto main, all 46 pure checks pass.

Two smaller points about the description:

• The default Desktop changes for everyone with more than one display, from displays.first to the main display. I think that's better, but users will notice it, so it deserves a line.
• preferMainDisplay is now written but never read, and its comment no longer holds.

[addhumal]

Thanks @marliechorgan for testing this on a 3-display setup and putting together addhumal#2!

Merged addhumal#2 into desktop-displays, removed the unused preferMainDisplay property (c15dd2b), merged latest main (4d990e9), and updated the PR description to note the main-display default and the new check/mutant counts (compatibility: 112 checks, 29 of 29 mutants).

[addhumal]

@Saffsanity this is ready for your review: tap Desktop again for the next monitor (#91, your tapping method). It's reworked from main, and merged with main's locked Mac.

Decisions to confirm:

• Monitors are numbered left to right, then top to bottom, as System Settings arranges them.
• The choice is kept by display UUID while Sill runs.
• The Mac's pointer jumps to the new monitor's centre.
• The virtual display isn't a target (your words on Feature Request: Select which Mac display Desktop mode streams #91 named it; left out for now).

Untested, for you: on a Mac with two or three monitors:

• Stepping and wrapping, and the "Desktop 2" label.
• A tap right after switching landing on the shown monitor.
• A monitor unplugged and plugged back in.
• VoiceOver's "Display 2 of 3".

The pure rules are checked (desktop-displays: 56 checks, 31 of 31 mutants), and CI is green. Whichever of this and #200 lands second needs WindowCatalog's display choice reconciled.

[addhumal]

Merging this beside #173 / #200: they conflict only in how the Desktop picks its display. A tested merge of the two is ready on addhumal:reconcile-173-200, for whichever lands second.

• WindowCatalog: Desktop: cycle through the Mac's displays with the Desktop button (#91) #173 picks among the displays of Host: virtual Desktop display fallback for headless Macs (#61) #200's HeadlessDesktopPolicy.pool (a real display, else Sill's, else macOS's stand-in).
• StreamCoordinator.desktopCapture: keeps both lines.
• Result: with a real display, Desktop: cycle through the Mac's displays with the Desktop button (#91) #173 steps through the real ones. With the lid closed, the button reads Desktop and Sill's Retina display streams.
• Checks: run-all.sh all 64; mutants desktop-displays 31/31, virtual-display-api 28/28, lock-policy 141/141.
• Known, minor: opening the lid restarts the Desktop twice in a row, once for each change.

[xiaofeng-han]

please merge this, I tried it on my machine with 3 external monitors, working well. It would be great if it could provide a list to choose from, but I could work with current solution

[addhumal]

Thanks @xiaofeng-han for testing it on three monitors! A list to choose from is in #296: hold the Desktop button and pick a display by name.