Skip to content

Clipboard: images both ways (generation 2) - #300

Open
addhumal wants to merge 1 commit into
Saffsanity:mainfrom
addhumal:clipboard-images
Open

addhumal wants to merge 1 commit into
Saffsanity:mainfrom
addhumal:clipboard-images

Conversation

@addhumal

@addhumal addhumal commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

The clipboard now carries images both ways, as decided on 2026-11-02: up to 8 MB as they are, bigger ones re-encoded to fit, with the cap raised for the clipboard's messages alone.

What changed

  • Copy on the Mac, paste on the iPad. A screenshot or an image copied in any app is ready to paste on the device. A PNG or JPEG within 8 MB goes as it is. macOS's own TIFF copies go as PNG when that fits; anything bigger becomes a JPEG, scaled down until it fits.
  • Paste on the Mac. The chip and the Paste button offer an image too. Photos' HEIC goes the same way, and text still wins when there's both.
  • Wire (additive):
    • The window list says clipboard 2.
    • A device subscribes with {"accepts":2} and only then gets images. {} stays text alone, so every iPhone and iPad from before is unchanged.
    • image is the field Sill Viewer: a Mac controlling another Mac #66 already used.
    • The clipboard's message may be 12 MiB either way; every other kind's cap is as before.
  • Mixed versions. An older Mac ignores accepts, and this device offers it no images. An older device subscribes with {} and gets text alone.
  • Words. The Mac's footnote, the Paste hint, a "Too big to send" note, the README, privacy.html and support.html, which said "text only".

Problems found, and how they were fixed

  • Finder copies would have sent an icon. A copy of files in the Finder carries the files' icons as images, so with no text it would have gone as a picture. Copies with a file URL are skipped, and the icon is never even read. A mutant that dropped this rule had survived; the check now covers it.
  • Promises that were no longer true. The privacy page, the support page and the Mac's footnote all promised text only. They now say text or an image.
  • Shuffled text order. The Paste button's text items load in parallel, so their order could come back shuffled, and "the first that isn't empty" depends on it. They're kept by index now.
  • Mutant patterns. Seven pinned lines that moved, and one was inverted: "generation 2" was the mutant before and is the build now, so it became "generation 1".

Checks

  • clipboard: 368 checks, including real PNG, JPEG and TIFF images and a noise PNG over 8 MB fitted to a JPEG within it. Its mutants: 13 new, 7 repointed, all caught.
  • compatibility: 152 checks. message-reader passes.
  • run-all.sh: all passed.
  • swift build -c release: only the known CaptureProbe warning.
  • The iOS Debug simulator build passes.
  • Live on a synthetic host's own pasteboard, never this Mac's clipboard:
    • A copied PNG reached the image-taking client as it was (1,636 bytes).
    • A 15 MB TIFF arrived as a 4.4 MB JPEG.
    • The {} client got neither.
    • A client's PNG was written to the host's pasteboard byte for byte and forwarded only to the other image-taking client.

Not checked: a real iPad: the Paste button with an image, a photo's HEIC, writing the Mac's image to the iPad's clipboard. The Mac's real pasteboard. A copy of over 8 MB from the device: the check covers it, but a command line can't carry it.

Risk: a device's image is checked only by its first bytes (PNG or JPEG) before it goes on the Mac's pasteboard, the same trust a device's text gets today. Re-encoding it on the Mac would parse it inside Sill instead. That's your call.

@Saffsanity: on the iPad with this build, take a screenshot on the Mac (⌃⇧⌘4) and paste it in Notes on the iPad. Then copy a photo on the iPad, tap Paste in Sill, and paste it in a Mac app.

Noah's call (2026-11-02): images up to 8 MB as they are, bigger ones
re-encoded to fit; the cap raised for kind 32 alone.

- Wire (additive): a host's list says clipboard 2; a device subscribes with
  {"accepts":2} and gets images only then ({} stays text alone, so a device
  from before is unchanged); `image` is Saffsanity#66's field, PNG or JPEG, at most
  8 MiB (Clipboard.maxImageBytes). Kind 32 may be 12 MiB either way
  (StreamMessage.maxClientPayload(for:) and maxHostPayload(for:)); every other
  kind's cap is as before.
- ClipboardImage.fitted (ImageIO, both ends): a PNG or JPEG within 8 MB as it
  is; a TIFF (macOS's own copies) or HEIC as PNG when that fits; else JPEG
  at 0.85, scaled down until it fits.
- Host: with no text on the pasteboard, its PNG, JPEG or TIFF goes to the
  subscribers that take images, never for a copy of files (their icons); a
  device's image is written to the pasteboard (marked as this launch's) and
  forwarded to the other image subscribers.
- Device: the Mac's image is written to the clipboard (10 minutes, local
  only); the chip offers an image too while the Mac takes them; the Paste
  button takes images then (text still wins), fitted off the main thread.
- Words: the Mac's footnote, the Paste hint, a too-big note, README,
  privacy.html and support.html.
- sillclient: --clipboard-images.

Checks: clipboard 368 checks; its mutants, 13 new and 7 repointed, caught
(the I4 hole found and closed); compatibility 152; message-reader; run-all.sh
all passed; swift build -c release; the iOS Debug simulator build. Live on a
synthetic host's own pasteboard: a PNG to an image subscriber as is (1,636
bytes), a 15 MB TIFF as a 4.4 MB JPEG, nothing to a {} subscriber; a device's
PNG written byte for byte and forwarded to the image subscriber alone.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant