Repository navigation
Apple Pencil pressure and tilt - #175
Open
HunterAhlquist wants to merge 20 commits into
Open
HunterAhlquist wants to merge 20 commits into
HunterAhlquist wants to merge 20 commits into
Conversation
It came in with the pen proof of concept; main never tracked it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
As the Pencil lifts, iPadOS 27.0.1 on an iPad Pro M4 sends one more hover update about 1 ms before touchesEnded, carrying where the Pencil hovered before it landed. handleHover sent it as a move while the Mac's button was still down, so every stroke in a drawing app ended with a straight line back to where it began (six strokes of six, main's build too). Hunter Ahlquist found it and fixed it in his pencil-pressure branch (issue Saffsanity#81), where the rule is PenStrokes.hover's and covers pen mode too; this is the same rule on today's Pencil-as-mouse path alone, so it can land ahead of the rest: a Pencil touch sets pencilTouching from touchesBegan to touchesEnded or touchesCancelled (and leaving the window clears it), and a hover update meanwhile sends nothing. The iPad's own pointer is untouched: only a Pencil touch sets the flag. Co-authored-by: Hunter Ahlquist <36090375+HunterAhlquist@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… as a pen Hunter Ahlquist's five commits (issue Saffsanity#81), from main at 985a733, merged as they are onto main at d1fa97e and the stale-hover commit before this one. Eighteen files conflicted; every resolution keeps both sides: - StreamMessage: main's clipboard (32), then the pen (still 29 here). - Switcher: WindowList keeps `clipboard` and gains `pencil`; one init with both parameters, and the coordinator's list sends both. - InputOverlay: main moved the finger pan onto the scroll path it shares with the iPad's trackpad and a wheel (Saffsanity#80); the palm guard stays in handlePan, before scroll(gesture, .finger), so it silences fingers only. The view's leave as it goes sits beside main's keyboard proxy. The stale-hover flag of the commit before gives way to PenStrokes.hover, the same rule in both modes (gone() resets it as the view leaves the window). - StreamClient, MockCatalog, HostSettingsPanel, ContentView, sillclient.py: the clipboard's lines and the Pencil's side by side (the This iPad group has Larger Picture, Share Clipboard, the gestures, then Pencil Pressure; -SillInputScript has Saffsanity#80's steps and the pen's; --copy and --pen). - The checks: compatibility runs main's clipboard cases and the pen's, its mutants follow the new init; protocol and pointer-control list both; the README's rows and ci.yml's mutants matrix gain `pen`. - docs/DEVELOPMENT.md: main's hardware keyboard and clipboard sections, then Apple Pencil; site/privacy.html and CLAUDE.md likewise. Left for the commits after this one: the pen's kind 29 is a number main's clipboard, compatibility, protocol and pointer-control checks keep free (they fail on it here), PenStrokes.swift's project IDs AC01/FC01 are HardwareKeyboardPolicy.swift's since Saffsanity#80 (the iOS app does not build here), and the workspace file comes back. swift build -c release passes here. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Kind 29 was free on 985a733, where the pen was built; since, main's comment and checks keep 29 to 31 free (29 the audio plan's, 30 and 31 Saffsanity#66's side streams), 32 is the clipboard, and open pull requests hold 33 to 44 (Saffsanity#66's 33 to 42, Saffsanity#94's 43, Saffsanity#93's keyframeRequest 44, checked against every open head today). So `case pen = 45`, and everything that names the number follows: SessionLink.isInput (8 or 45), the comments of Pen.swift, Switcher, PenTablet, InputInjector and StreamServer (which match the enum, not the number), sillclient.py's --raw45 and its sends, and the docs (DEVELOPMENT.md, CLAUDE.md's entry, Layout and Build and run, the plan's design and gates, with a note that its results before the rebase say 29). The compatibility floor's kinds are 0–28, 32 and 45 again with 29 to 31 free as main has them; kind 26's `seen` counts kinds 8 and 45. The checks: compatibility names kind 45 and 46 unknown (157 checks; its mutant "kind 45 as 46"), protocol keeps main's two kind checks and has the pen at 45 with 46 unknown (193 + 8), pointer-control's table and sweep say 45 (158), fence's input and its two C10 mutants say 45; menus' line and mutant and pointer-control's two renumbering mutants are main's again, 29 being free once more. clipboard passes unchanged. Run here: compatibility 157, protocol 193 and the 8 cross-checks, pointer-control 158, menus 309, clipboard 259, pen 197 and 5,000 random sessions, all passing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fork gave PenStrokes.swift A1000001000000000000AC01 and …FC01, free on 985a733; Saffsanity#80 has since given the same pair to HardwareKeyboardPolicy.swift on main. The project file merges without a conflict but then holds each ID twice, xcodebuild keeps one ("Skipping duplicate build file"), and the iOS build fails on HardwareKeyboardPolicy. A045/F045 (after the pen's kind) are in no project file on main or in any open pull request's head, checked against all of them today. CLAUDE.md's Layout line names the new pair. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
be8abf4 untracked iOSClient/Sill.xcodeproj/project.xcworkspace/ contents.xcworkspacedata, which Xcode writes when it opens the project and main has never tracked; a85fba7 brought it back with the edge cases. Out again, so this branch's project folder is main's plus PenStrokes.swift. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
privacy.html is on main ahead of every release, and a republish before the Pencil ships (another fix's words) would tell readers the app sends their Pencil's pressure and tilt and keeps a Pencil Pressure switch, which the builds out do not. As the clipboard's words are (PR Saffsanity#78), each Pencil item now stands alone between `<!-- Pencil: …` and `<!-- /Pencil -->`: "Your Apple Pencil's strokes, with how hard you press and how you tilt it." among what the device sends, and "On an iPad, whether Pencil Pressure is on." among what it keeps. docs/release-checklist.md part 1 §3 gains the perl line that cuts them before that release, and says which Pencil lines stay after the cut (index.html's intro, terms.html's trademarks). Checked on a copy: after the Pencil's cut site/ is main's byte for byte, and with the clipboard's too `grep -i pencil` prints only those two lines. The page's date is left for the release to set, as the plan says. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s a pen So a test can tell a host that takes kind 45 from one that skips it, as it already tells the clipboard's: against this branch's synthetic host the line reads "clipboard=1 pencil=1", against main's "clipboard=1" alone (the device then sends its Pencil as kind 8). The help says so beside --pen. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s people Tests/checks/pen/run.sh still listed "mouse input from any device" and "a second without a message with the tip up" among the pen's leaves; since a85fba7 (Sidecar's behaviour) mouse input never takes the pen out of range and the idle leave comes a second after an up with no hover since, as PenTablet, its check and the checks' README say. Now the header says so. docs/pencil-pressure-plan.md was written in Hunter's fork, where "the owner" meant him; in Sill's repository it reads as the repository's owner. Who ran, saw, drew or asked is now Hunter by name, and the calls the plan leaves open (its decisions' defaults, Q1) are Noah's, Sill's maintainer. Nothing else in the plan changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pencil-pressure-rebased Only ci.yml's mutants matrix conflicted: it keeps both new checks, `installed-apps` (Saffsanity#136) and `pen`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s rebase Hunter's branch carried a 56-line Current step entry written for his fork (its "the owner" meant him, and its numbers were the fork's); every other open pull request edits the top of that section too, so the entry is now a short one in the section's usual form: what the branch is, the wire, the host, the device, what the maintainer's commits after the merge did, what was verified, and what waits for a device. The details stay where they were already: the plan, the Layout lines and Build and run. PenProbe's line in Build and run says a person runs --post-to-this-mac by hand at the Mac, never an agent (it said "the owner's to run"). docs/pencil-pressure-plan.md gains "Rebased onto main (2026-10-20)": the order of the branch (the stale hover on main's mouse path first, Hunter's five commits merged as they are, the maintainer's commits, main again), the merge's resolutions (WindowList's one init with both `clipboard` and `pencil`, the palm guard in handlePan before Saffsanity#80's shared scroll(gesture, .finger), the stale-hover flag giving way to PenStrokes.hover), kind 45 and the checks it touched, the project IDs, the workspace file, the site's markers, and what was verified with its numbers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
While a Pencil hovers as a pen, the device's arrow shows in the Mac cursor's shape only when that cursor is an app's own (a drawing app's brush outline), never over a standard arrow, I-beam or hand. Kind 46 `cursorKind` (CursorKind), host → device, just before each kind 14; the host's CursorClass decides. Re-applied on pencil-pressure-rebased: its kind moved from 30, which Saffsanity#66 holds, to 46, the first number no open pull request holds after the pen's 45. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…to pencil-pressure Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ding) into pencil-pressure CI's mutants matrix keeps both new checks, pen and virtual-display-api. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HunterAhlquist
marked this pull request as ready for review
October 19, 2026 20:31
… parameter after clipboard The mutant "clipboard 1 by default" matched the init's line ending at clipboard, which now goes on to pencil, so it no longer applied (72 of 72 with it applying again). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
<3 |
Saffsanity
added a commit
that referenced
this pull request
Oct 20, 2026
…is run's checks Work in progress, paused for the night. The plan's "Rebased onto main" and the CLAUDE.md entry say what ran on f85aa49 (main at 4d990e9 merged as d8984a6, Hunter's clipboard-mutant fix 5c39211 cherry-picked from #175): swift build -c release from git archive and the three iOS builds from an empty derived-data folder (known warnings only), SillHost --pen-selftest 37 of 37, run-all.sh all 47, H4 (idle identical; client runs identical but for their last stats second), H5 and S1 on loopback (the pen's enter, tilt, pressure, drags, up and leave against this host; kind 8 alone against main's). What remains: clipboard/run.sh --mutants to the end (9 of 72 caught before the pause), and a decision on #175, which carries Hunter's hover cursor (kind 46, CursorClass) that this branch does not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main took kinds 43 (sound) and 44 (keyframe request); 45 and 46 stay the Pencil's and the cursor's kind. The pen now goes through main's new rules as kind 8 does: a revoked device and a locked Mac pass on only what lets its pen go (PenTablet.mayLetGo, stillGoes), the lock gate withholds kind 46 with kind 14, DisplayWake counts kind 45 as a person, and the pen's button carries its owner like main's buttons. The palm guard runs before main's picture zoom in handlePan. Checks: run-all.sh all passed but display-wake, now taught kinds 45 and 46 (200 checks, 64 of 64 mutants); lock-policy knows kind 46. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Apple Pencil pressure and tilt, as discussed in #81. Built on
pencil-pressure-rebased, @Saffsanity's merge of my branch into main, with one more commit of mine on top and main at 125be9d merged in.A Pencil on the stream now draws as a tablet pen in Mac apps that read pressure and tilt, and stays the mouse everywhere else. Double tap and barrel roll will come in a second pull request.
What's in it
pen;PenTabletandPenEventson the Mac;PenStrokeson the iPad, with palm rejection;cursorKind(CursorKind {standard}), host → device, sent just before each kind 14. Older devices skip it, and a device that gets none treats the cursor as standard.CursorClassdecides by comparing the cursor's pixels with AppKit's standard cursors. The arrow and I-beam are also recognised by the window server's 1×–10× images, because the CLI has no NSApplication to compare against.penandvirtual-display-api; nothing else conflicted.Checked
swift build -c releaseand the iOS app for the simulator and for a device: only the known warnings.Tests/checks/run-all.sh: all pass.pen97,pointer-presence48,compatibility43,protocol24,pointer-control33,clipboard72. Oneclipboardmutant no longer applied onceWindowList's init gainedpencil:afterclipboard:; its pattern is fixed in the last commit.PenProbe --canvas, run through Sill on this build: every stroke arrived as tablet points, pressure finer than 8 bits, and tilt from −0.67 to +0.59, the same sign as Sidecar's for the same grip.docs/pencil-pressure-plan.md§13).Still open
The plan's device tests P1–P10 beyond those above:
Known gaps:
Disclosure, as in #81: Claude Code wrote most of the code; I designed the behaviour with it and tested it on my devices.
🤖 Generated with Claude Code