[addhumal]

Why

Two problems from addhumal's audit (#101):

• Closing a window could stall the Mac's host for seconds. The Close button kAXCloseButtonAttribute returns is a new Accessibility element on the system's default messaging timeout, and the host presses it on its main actor. An app whose close asks about unsaved changes in an alert of its own, or a hung one, held up input, window picks and settings for every connected device for that long.
• A still window could stay blank or frozen after an interruption. When iOS stops the display layer (Sill in the background, another app taking the decoder) or a frame fails to decode, HEVCDisplayView noticed only status == .failed, only when a frame arrived, and never told the Mac. A window that is not changing sends no frame, so nothing brought the picture back until it did.

What

The first version of this pull request picked the source again (kind 6) every 2 s while the layer read interrupted. That restarts every device's stream, raises the window on the Mac in regular mode, and repeats for as long as a layer keeps failing. It now asks for one keyframe instead, through a new message.

• Close (host; the contributor's change, kept). The button is type-checked (CFGetTypeID, then the cast, as MenuReader does) and given a 1 s timeout. WindowSizer.perform now says what came of a window's light (CommandOutcome): "Window 104 close: done", "refused", or after 1 s "no answer in time (the app is busy, or asking about unsaved changes); it may still act". A minimize or full screen with no answer says only that the app is busy. Scripts/menu-check/run.sh close builds again (since A window Sill sized goes back to its own frame when it stops streaming #75 WindowSizer needs WindowRestore.swift) and presses Close on the menus' test app's Slow Close window, whose close holds its main thread for 4 s.
• Kind 44, keyframeRequest (device → host, empty payload; the number Noah gave this pull request on Apple Pencil Features (pressure, tool swap, etc) #81). The host answers with HEVCEncoder.requestKeyframe, which encodes a still window's last frame again; every device gets that keyframe, as when one joins. At most one a second per connection ("Keyframe for ‹device›: its picture was interrupted."; the rest one line a minute), nothing while nothing streams, never a restart or a raise. A Mac from before (Sill for Mac 0.5.1) skips the kind without a line, and the picture there waits for the stream's next keyframe, as it always has.
• The device. HEVCDisplayView.checkLayer flushes a layer that failed, could not decode a frame (failedToDecodeNotification) or needs a flush to decode again (requiresFlushToResumeDecoding): at each frame, at the layer's notifications (a still window sends no frame that would show it) and when the picture's window becomes active. It says so (onNeedsKeyframe), and a keyframe that ends the wait says the picture is back (onPictureBack). A keyframe the flush throws away, because the layer still read failed when it came, counts as the picture back and lost again.
• When to ask: KeyframeRecovery (pure, Tests/checks/keyframe-recovery, new):
  • once per interruption while its keyframe is due, and again when none came within max(4 s, 4 × the worst round trip), so a Mac that never answers is asked after 4, 8 and 16 s, then every 32 s;
  • only while the picture's window is foreground active (each Sill window has its own client) and something streams;
  • at least 2 s apart; 4, 8, 16, then 32 s for a picture that fails again within 10 s of its keyframe, so a decoder that cannot keep up never costs every device a keyframe every 2 s;
  • a trip to the background owes an ask that was out again and starts the spacing at 2 s again, and a keyframe shown while Sill is away never makes the next interruption a repeat.
• One mechanism (Noah on this pull request, 2026-10-15: "One recovery mechanism should serve all of them"): Settings panel: Picture Smoothing (Automatic, None, Low, Medium, High) #107's renderer already calls the same onNeedsKeyframe; it can drop PictureSmoothing.DecoderRecovery and its re-pick and call onPictureBack where its renderer takes a keyframe.
• Test hooks. DEBUG: -SillLayerFail <s>[,<s>…] (a frame that cannot decode goes onto the layer at those seconds) and -SillKeyframeLost <n> (the first n keyframes that end a wait are thrown away by the flush). TEST ONLY, synthetic hosts only: SILL_TEST_STILL_AFTER=<s> (the Desktop still from then on) and SILL_TEST_KEYFRAME_IGNORE=<n> (the first n keyframe requests ignored, as an older Mac ignores every one); sillclient.py --keyframe@T.
• Docs. CLAUDE.md (Current step, Layout, Build and run, the compatibility floor's kinds now 0–28, 32 and 44), docs/DEVELOPMENT.md ("If the picture freezes", the window lights' outcomes, the harness and test hooks), the checks' README, and site/privacy.html's list of what a device sends its Mac, between <!-- Keyframe and <!-- /Keyframe --> with a perl line in docs/release-checklist.md part 1 §3 to cut it for a republish before the release that ships it. No release words yet (What's New, README, support): they wait for a device run of the trip to the background.

The review, fixed

A verifier re-ran everything on 09549b3 and found six problems. Each is fixed in a commit of its own:

1. A keyframe the flush threw away left the picture frozen (80e982f). If the keyframe an ask brought arrived while the layer still read failed, the flush discarded it, the interruption stayed open with its ask counted as out, and nothing was asked again while Sill stayed active. Now that keyframe counts as the picture back and lost again, and every ask has a deadline (max(4 s, 4 round trips)), after which it is made again under the backoff.
2. A trip to the background kept the backoff (80e982f). The verifier's probe held an ask 24 s after the return. Now a trip starts the spacing at 2 s again, and a keyframe shown while Sill is away keeps the trip, so keyframes iOS may refuse in the background never build a streak.
3. DEVELOPMENT.md on older Macs. A Mac from before kind 44 brings the picture back with the stream's next keyframe (the encoder makes one every fps × 4 encoded frames, none while the window stays still), not when the window next changes.
4. Minimize and full screen said "asking about unsaved changes" (a45a523). Each action now has its own words.
5. Noah's steps. TextEdit asks about unsaved changes in a sheet, which returns at once, so its Close most likely logs "done"; either outcome is fine. Against Sill for Mac 0.5.1 the device does send kind 44, which 0.5.1 skips with no line.
6. Merge notes (CLAUDE.md): The Mac's sound on the device: Send Audio, kind 43, AAC-ELD (#83) #94's compatibility check asserts StreamMessageKind(rawValue: 44) == nil, and Sill Viewer: a Mac controlling another Mac #66's, Sill Viewer: Dock badges, bounces and minimised windows in the remote apps' Dock icons; full speed in the background (builds on #66) #72's and Background windows paused while hidden and at 1×; stream rate capped by size (builds on #72 and #71) #73's pointer-control mutant "kind 26 numbered 44" stops compiling once 44 exists (their mutants.py then counts it as not caught); whichever lands second changes the check, never kind 44. Sill Viewer for Mac, and Rust hosts for Windows and Linux #128's Mac viewer compiles StreamClient.swift without HEVCDisplayView's new hooks, the scene observers or KeyframeRecovery.swift.

Verified

On the merge with main at 4d990e9, on a Mac shared with other agents (load average in the hundreds, so times are slow):

• Tests/checks/keyframe-recovery: 95 checks and 20,000 random sessions against a model of the event history (never stuck while open, active and streaming; a re-ask only once the last ask's keyframe was overdue; no two asks less than 2 s apart); 39 of 39 mutants, three of them each finding put back.
• Tests/checks/pointer-control (its kind table has 44): 158 checks, 33 of 33 mutants.
• Tests/checks/run-all.sh: all 47 passed (28 min under that load).
• swift build -c release on the merged tree: complete, no new warning.
• The iOS app for the simulator with CI's command (Xcode 27.1 27A9275, clean derived data): succeeded, only the known StreamClient.swift:3819 capture warning.
• Scripts/menu-check/run.sh close: 9 checks, back in 1.01 s as no answer, the close delivered to the window, every action's words, the front app unchanged.
• The host alone (sillclient.py, loopback, software encoder): with SILL_TEST_KEYFRAME_IGNORE=1, requests at 4 and 6 s: the first ignored with its line, the second answered (its keyframe 0.09 s later).
• Live on a private iPad simulator (iOS 27.0) against loopback synthetic hosts on the software encoder, the Desktop still from 3 s:
  • -SillLayerFail 6: the layer's own -11821, one ask, one "Keyframe for …" on the host, the picture back, and one "Synthetic capture" line (no re-pick).
  • With SILL_TEST_KEYFRAME_IGNORE=1: the first ask ignored, "keyframe: no keyframe 4.0 s after the last ask; asked the Mac for a keyframe (spacing 4 s)" 4.2 s later, the host's line, the picture back 0.15 s after it.
  • -SillKeyframeLost 1: the keyframe that ended the wait thrown away ("flushed with the keyframe it waited for"), the next ask 4 s after the first (a repeat's spacing), the picture back.
  • -SillLayerFail 6,9,13,21.5,37.5: asks 4.05, 8.1 and 16.0 s apart; the fifth interruption held 28.5 s by a 32 s spacing; then Settings in front for 2 s and Sill back: asked at once ("spacing 2 s"), 21 s before the hold would have let it.
• This build against Sill for Mac 0.5.1's host (git archive v0.5.1, its Desktop moving) with -SillLayerFail 6: asks at +0, +4.1 and +12.3 s, no line on the host and no disconnect, pongs and frames throughout, the picture back 15.2 s after the interruption with the stream's periodic keyframe.

Not run here: anything on a device. The simulator's display layer never stops for a trip to the background, so the issue's own case (requiresFlushToResumeDecoding and a failed status after Sill was away) has run only through the pure check.

Untested, for Noah

On the iPad and the iPhone with this build and a Sill.app from this branch:

1. A still window (TextEdit, nothing changing), Sill in the background for 2 s and back: the console's "display: the picture was interrupted …", one "Keyframe for …" in Sill.log, the picture back at once, no "Capture started", the window not raised.
2. Ten quick trips: a line each, each ask at once, never one held for 4 s or more after a trip.
3. A video playing in Split View beside Sill.
4. Two Sill windows, one swapped out and back.
5. Close on a TextEdit window with unsaved changes: its sheet on the Mac, input and picks unaffected meanwhile; Sill.log most likely says "Window N close: done" (a document's sheet returns at once). An app that asks in an alert of its own, or a hung one, gives "no answer in time …" after 1 s. Either is fine.
6. This iPad against Sill for Mac 0.5.1: the device sends kind 44, which 0.5.1 skips with no line and no disconnect; the console's asks again after 4, 8 and 16 s, then every 32 s; the picture back only with the stream's next keyframe (a still window's never, as before).

Towards #101 (the trip to the background, the issue's own case, needs a device).

🤖 Generated with Claude Code

🤖 Generated with Claude Code

[addhumal]

Hey found few issues hope i can help. Got few more features planned.

[Saffsanity]

From a pass over all the open pull requests (nothing is decided yet): this edits the same decode path as #107, which adds its own keyframe re-pick (PictureSmoothing.DecoderRecovery) for its Metal renderer, and as #66 (with #72 and #73), which moves this code into HEVCSampleFeed.swift. One recovery mechanism should serve all of them, with whichever lands second reusing the first. Thanks for the AC11/FC11 renumbering; the remaining overlaps with #94, #95, #99 and #117 are the ci.yml mutants line and neighbouring project-file entries.

[addhumal]

Thanks. Merged main in (d821e1a, after #78, #79, #80 and #98). In #107 (7c7651e), HEVCDisplayView now exposes the same var onNeedsKeyframe: (() -> Void)? hook and routes MetalStreamRenderer's keyframe request through it in StreamClient.swift, so whichever of #93 and #107 lands second connects both paths to one recovery rule.

[Saffsanity]

Thanks for the audit and for both fixes. Your branch now has commits on top of yours; none of yours were rewritten, and main (4d990e9) is merged in.

Close: kept as you wrote it (the type check, then the 1 s timeout on the button). Added:

• The harness builds again: since A window Sill sized goes back to its own frame when it stops streaming #75 WindowSizer needs WindowRestore.swift, so Scripts/menu-check/build.sh compiles it too. run.sh close passes: against Slow Close the press comes back in 1.01 s, the close reaches the window, and the front app never changes.
• WindowSizer.perform says what came of a press (CommandOutcome). A close that times out logs "no answer in time (the app is busy, or asking about unsaved changes); it may still act" instead of "refused", since the press did reach the window. A minimize or full screen with no answer says only that the app is busy.

The interrupted picture: picking the source again restarts every device's stream, raises the window on the Mac in regular mode, and with a layer that keeps failing it repeated every 2 s. The device now asks for a keyframe instead:

• Kind 44, keyframeRequest (device → host, empty payload; the number Noah gave this pull request on Apple Pencil Features (pressure, tool swap, etc) #81). The host answers with HEVCEncoder.requestKeyframe, which encodes a still window's last frame again, at most once a second per connection, and logs "Keyframe for ‹device›: its picture was interrupted." A Mac from before kind 44 skips it without a line; the picture there waits for the stream's next keyframe, as before.
• checkLayer also listens to the layer's failed-to-decode and needs-a-flush notifications, so a still window recovers without waiting for a frame. A keyframe the flush throws away (the layer still read failed when it came) counts as the picture back and lost again.
• KeyframeRecovery decides when to ask: once per interruption while its keyframe is due, again when none came within max(4 s, 4 × the worst round trip) (so an older Mac is asked after 4, 8 and 16 s, then every 32 s, which costs it nothing); only while the picture's window is active and something streams; at least 2 s apart, and 4, 8, 16, then 32 s for a picture that fails again right after its keyframe; a trip to the background starts the spacing at 2 s again. Its check has 95 cases plus 20,000 random sessions against a model, and catches 39 of 39 mutants.

For #107: its renderer already calls onNeedsKeyframe, so it can drop PictureSmoothing.DecoderRecovery and its re-pick and call onPictureBack where its renderer takes a keyframe, so one rule serves both.

For testing: DEBUG -SillLayerFail <s>[,<s>…] and -SillKeyframeLost <n>, and on a synthetic host SILL_TEST_STILL_AFTER (a still Desktop) and SILL_TEST_KEYFRAME_IGNORE (asks ignored, as an older Mac ignores them), reproduce each path in a simulator. Live there, each failure gave one ask and one host line with the source never picked again; an ignored ask was made again 4 s later; and a trip to the background ended a 32 s hold at once. The simulator's display layer never stops for a trip to the background, though, so the issue's own case still needs a device run, and the description now says "Towards #101". It also still describes the first version (the kind 6 re-pick); a replacement is drafted.