Skip to content

Settings panel: Capture Screen - #96

Open
addhumal wants to merge 5 commits into
Saffsanity:mainfrom
addhumal:capture-screen
Open

addhumal wants to merge 5 commits into
Saffsanity:mainfrom
addhumal:capture-screen

Conversation

@addhumal

@addhumal addhumal commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Closes #103.

What it does

Adds a Capture Screen… row in the device's Settings panel, above Take the Tour, that captures the current stream picture and presents the standard iOS/iPadOS share sheet (Save Image, Copy, Messages…). Device only: no wire change, nothing sent to the Mac.

HEVCDisplayView.snapshot(maxWidth:includePointer:) draws the display view cropped to videoRect (no letterbox bars) at the screen's scale, with the pointer sprite when visible (and optional maxWidth / includePointer parameters shared with #117's tile previews). The row is disabled while videoSize == .zero. On an iPad with multiple windows, the share sheet presents from the window that owns the session's displayView.

Why

Taking a device screenshot captures the top bar, the status bar or the portrait trackpad and key row around the stream, and scales the window to whatever size the pane happens to have on screen. Cropping HEVCDisplayView to videoRect shares or saves only the streamed Mac window or Desktop.

Save Image writes to Photos, so NSPhotoLibraryAddUsageDescription is added to Info.plist and Info-Debug.plist, and site/privacy.html names Photos (adding only).

Checks

  • Tests/checks/run-all.sh and swift build -c release pass with only the known warnings; the iOS app builds for the simulator.
  • Verified on an iPad Air simulator (iOS 27.0) against SillHost --synthetic with DEBUG -SillCaptureAt <s>: 1608×1009 px, 100 % not black, cropped to the video rectangle.

Windows App's Capture Screen, from the roadmap: a row above Take the Tour shares a picture of
what streams through the share sheet (Save Image, Copy, Messages…). `HEVCDisplayView.snapshot()`
draws the display view, cropped to the video (no letterbox), at the device's scale or the Mac's
pixels if more, with the pointer sprite if it shows. Disabled while no picture has arrived.
Sends nothing to the Mac.

Save Image needs NSPhotoLibraryAddUsageDescription (Info.plist and Info-Debug.plist: "Sill saves
a picture of your Mac's screen to Photos when you ask it to."), and the privacy policy names
Photos, adding only. Its "Last updated" date is left for the day the site is republished, as
the release checklist says.

DEBUG `-SillCaptureAt <s>` writes the picture to the app's tmp and says how much of it is not
black. Checked on an iPad Air simulator (iOS 27.0) against SillHost --synthetic: 1608×1009 px,
100 % not black, the test pattern cropped to the video; the row in the panel. Not seen: the share
sheet itself (no tap here), and a device's video layer, which could snapshot otherwise.
Review finding 3. The picture was drawn at the screen's scale or the Mac's pixel width, whichever
was more, but drawHierarchy draws what the screen shows, so the larger scale only enlarged it: a
Retina stream (3024 px wide) shown 804 pt wide on an iPad came out 3024×1898 from 1608 pixels of
detail. It is drawn at the screen's scale now.

Checked on an iPad Air simulator against SillHost --synthetic on the hardware encoder (no device
connected): 3024×1898 px before, 1608×1010 px after, 100 % not black both times.
Found in the review before this push: the sheet was presented from the first scene in front, and
on an iPad with two Sill windows side by side both are, so Capture Screen in one could open the
sheet over the other. It is presented from the window this session's picture is in
(displayView.window), which also drops the scene lookup.
@Saffsanity

Copy link
Copy Markdown
Owner

From a pass over all the open pull requests (nothing is decided yet): this overlaps #117, which adds a nearly identical drawHierarchy snapshot right after videoRect in HEVCDisplayView.swift; #95, which inserts its group at the same spot in HostSettingsPanel.swift; and #107 in StreamScreen.swift. One snapshot helper shared with #117 and one agreed order with #95 would let these land in any order, and whichever lands after #107 should check the capture with its Metal layer.

addhumal added a commit to addhumal/sill that referenced this pull request Oct 15, 2026
@addhumal

addhumal commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor Author

Thanks. Pushed 29d4910 (and merged main in 152bc5b): HEVCDisplayView.snapshot(maxWidth: CGFloat? = nil, includePointer: Bool = true) -> UIImage? is now identical between #96 and #117 (a700cca), so the two merge without a conflict in either order. In #107 (7c7651e), MetalStreamRenderer also sets layer.framebufferOnly = false so drawHierarchy can read the Metal drawable.

Saffsanity added a commit to addhumal/sill that referenced this pull request Oct 19, 2026
The helper shared with Saffsanity#96 (HEVCDisplayView.snapshot) gave a picture whatever the
layer showed, and its pointer hide did not reach the screen in time. On a private
simulator against a synthetic host whose scripted pointer moves (the Mac's arrow
over the stream), the stored includePointer: false picture carried the arrow;
flushing the hide's transaction first still left it in 1 picture of 3; holding
the sprite back and drawing after the screen update left it in none of 8, while a
picture with the pointer taken in the same tearDown had it in all 8.

- Nil while no keyframe has gone onto the layer since it lost, or may have lost,
  its picture (showsPicture: apply's flush for a new format, a failed layer's
  flush in enqueue, clear), while the layer is failed or needs a flush to decode
  again (iOS takes the decoder from an app in the background), and when
  drawHierarchy says the drawing is incomplete (a window not on screen).
- Without the pointer: the sprite is held back (pointerHeldBack, which
  placeCursor respects, so a layout pass during the drawing cannot show it) and
  drawHierarchy waits for the screen update (afterScreenUpdates: true only then;
  Capture Screen's picture with the pointer is drawn as before).
- The size is whole points rounded down: rounded to the nearest, a picture took
  in a hairline of the letterbox's black (the last row of every 640×402 JPEG,
  now 640×400).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Capture Screen button to share or save a snapshot of the stream

2 participants