fix(CommandPalette): cut search highlights on grapheme clusters, not code points - #371
Merged
Merged
Conversation
IgorShevchik
force-pushed
the
claude/search-mark-boundary
branch
from
August 13, 2026 06:37
8d757c7 to
e1b1968
Compare
IgorShevchik
force-pushed
the
claude/search-grapheme-clusters
branch
from
August 13, 2026 06:43
be7fa97 to
cb41285
Compare
8 tasks
IgorShevchik
force-pushed
the
claude/search-mark-boundary
branch
from
August 13, 2026 08:01
e1b1968 to
bd740b9
Compare
IgorShevchik
force-pushed
the
claude/search-grapheme-clusters
branch
2 times, most recently
from
August 13, 2026 08:23
fe8722e to
e1e10f0
Compare
IgorShevchik
force-pushed
the
claude/search-mark-boundary
branch
from
August 13, 2026 09:07
bd740b9 to
b0b8c74
Compare
IgorShevchik
force-pushed
the
claude/search-grapheme-clusters
branch
from
August 13, 2026 09:14
e1e10f0 to
b284e71
Compare
…code points Code points are not what a reader sees. A flag is two regional indicators, a family emoji several joined by ZWJ, `कि` a consonant plus a vowel sign. Cutting inside one produces a *different* character rather than a broken one — 🇺🇸 truncated by a single code point re-pairs into 🇸🇺, a different country, with nothing to signal the loss. Both boundaries snap off the straddled cluster: the mark insertion in `generateHighlightedText` and the truncation in `truncateHTMLFromStart`. Chosen on measurement. Segmenting the whole string costs 455 µs at 979 characters and 52 ms at 100k. A fixed ±64 window is constant-time but wrong: a run of flags is longer than the window, so segmentation starts mid-run and re-pairs the indicators — reproducing the exact bug, 28 disagreements in 1044 probes. `Segments.containing()` is correct, so it ships, behind three guards: a `< U+0300` screen, which keeps ASCII and Latin-1 off the segmenter but does nothing for Cyrillic or CJK, both above the floor; one segmenter view per value, built lazily; and an 8192-character ceiling, past which only the surrogate snap applies and every multi-code-point cluster loses protection. Two defects found in review are folded in. A region left empty by the ordering clamps emitted a bare `<mark></mark>` with the highlight lost. A region past the end of the value bypassed that guard entirely, because `substring()` clamps its own arguments while the comparison did not — it emitted the same empty mark, froze the cursor past every later region, and the trailing mark then drove truncation into eating the text before it. `indices` are sorted before use. That is a no-op for the search path, since Fuse merges and sorts them itself, but `highlight()` is a published export and `CommandPaletteGroup.postFilter` lets a caller supply its own matches; out-of-order boundaries make `containing()` re-walk a long run of regional indicators from its start, costing as much as the per-boundary segmenter this avoids. `truncateHTMLFromStart` counts retained code units and slices once, because the snap needs an index to move. Verified equivalent over 400,078 fuzzed inputs, including strings that themselves begin with `...`. One fixture trap is recorded in the tests: UAX #29 glues a run of bare emoji modifiers into a single cluster, so `'\u{1F3FF}'.repeat(20)` is one character rather than twenty. Closes #364 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWWrBHgfqGSbeU3V6UuMF8
IgorShevchik
force-pushed
the
claude/search-grapheme-clusters
branch
from
August 13, 2026 09:18
b284e71 to
db194cf
Compare
8 tasks
This was referenced Aug 13, 2026
IgorShevchik
pushed a commit
that referenced
this pull request
Aug 13, 2026
`GRAPHEME_SNAP_MAX_LENGTH` is documented, and was tested, as a bound on the
field text. It was not measured against it: `truncateHTMLFromStart` built its
snapper from `content` — the value after escaping and after `<mark>` insertion —
so the ceiling was reached early, by an amount nothing in the constant hints at.
Escaping has no fixed overhead: `&` expands to five characters, `"` to six.
'a'.repeat(8140) + FLAG.repeat(10) + 'match' // value.length 8185
'&'.repeat(1600) + FLAG.repeat(50) + 'match' // value.length 1805
Both degraded, and both rendered `🇸🇺` with an orphaned regional indicator where
the author wrote `🇺🇸` — a different country, silently. That is the defect #364
describes and #371 exists to prevent, reappearing above a threshold no reader
could predict.
`createClusterSnapper` now takes the length to weigh separately from the string
to segment. The parameter is required rather than defaulted: a default is what
would let the next derived-string caller inherit this bug without noticing.
The cost is real and is worth naming. The snapper builds a fresh `Segments` view
per call and the cut sits near the end of `html`, where `containing()` is
dearest: ~140µs at the ceiling, ~930µs on the six-fold expansion an escape-heavy
value produces there. Both are paid only when the boundary characters clear
`CLUSTER_CONTINUATION_FLOOR`, so plain text returns before a view is built and
escaped runs like `"` are themselves ASCII — the dear case needs a long
value *and* a cluster landing on the cut. A fraction of a millisecond, once per
truncated field, against rendering a flag as the wrong country.
The two ceiling tests moved off `value.length + MARKUP` onto `value.length`,
which makes `snaps clusters exactly at the ceiling` the plain-text repro — a
value of exactly the advertised length previously degraded. The new case covers
the unbounded half: every fixture in `degraded paths` was free of escapable
characters, so the expansion never reached the ceiling arithmetic.
`.sync/PORTING.md`'s guard entry said the threshold "is crossed 13 characters
earlier than a reading of `value.length` suggests". That described the defect,
not the design, and is corrected here rather than left for the documentation
follow-up — a porter reading it could otherwise restore the bug on purpose.
Closes #387.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LWWrBHgfqGSbeU3V6UuMF8
IgorShevchik
pushed a commit
that referenced
this pull request
Aug 13, 2026
`GRAPHEME_SNAP_MAX_LENGTH` is documented, and was tested, as a bound on the
field text. It was not measured against it: `truncateHTMLFromStart` built its
snapper from `content` — the value after escaping and after `<mark>` insertion —
so the ceiling was reached early, by an amount nothing in the constant hints at.
Escaping has no fixed overhead: `&` expands to five characters, `"` to six.
'a'.repeat(8140) + FLAG.repeat(10) + 'match' // value.length 8185
'&'.repeat(1600) + FLAG.repeat(50) + 'match' // value.length 1805
Both degraded, and both rendered `🇸🇺` with an orphaned regional indicator where
the author wrote `🇺🇸` — a different country, silently. That is the defect #364
describes and #371 exists to prevent, reappearing above a threshold no reader
could predict.
`createClusterSnapper` now takes the length to weigh separately from the string
to segment, named `fieldTextLength` for what it is. The parameter is required
rather than defaulted: a default is what would let the next derived-string
caller inherit this bug without noticing.
Segmenting a string the ceiling would have excluded is affordable, but not
because the call is cheap in isolation — on a long run of regional indicators it
is not. It is that `Array.from(html)` already paid the larger price on the same
string, unconditionally, on both sides of this fix. End to end the difference is
tens of microseconds either way, within noise. `test/bench/search.bench.ts`
gains a case that reaches the ceiling by escaping rather than by length, so the
claim has a durable measurement behind it rather than a throwaway script.
The two ceiling tests moved off `value.length + MARKUP` onto `value.length`,
which makes `snaps clusters exactly at the ceiling` the plain-text repro — a
value of exactly the advertised length previously degraded. The new case covers
the unbounded half: every fixture in `degraded paths` was free of escapable
characters, so the expansion never reached the ceiling arithmetic.
`.sync/PORTING.md`'s guard entry said the threshold "is crossed 13 characters
earlier than a reading of `value.length` suggests". That described the defect,
not the design, and is corrected here rather than left for the documentation
follow-up — a porter reading it could otherwise restore the bug on purpose.
Closes #387.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LWWrBHgfqGSbeU3V6UuMF8
IgorShevchik
added a commit
that referenced
this pull request
Aug 13, 2026
#388) `GRAPHEME_SNAP_MAX_LENGTH` is documented, and was tested, as a bound on the field text. It was not measured against it: `truncateHTMLFromStart` built its snapper from `content` — the value after escaping and after mark insertion — so the ceiling was reached early, by an amount nothing in the constant hints at. Escaping has no fixed overhead: `&` expands to five characters, `"` to six. A value of 8185 characters, and one of 1805 made largely of `&`, both degraded and rendered a flag as a different country, silently. That is the defect #364 describes and #371 exists to prevent, reappearing above a threshold no reader could predict. `createClusterSnapper` now takes the length to weigh separately from the string to segment, named `fieldTextLength`. The parameter is required rather than defaulted: a default is what would let the next derived-string caller inherit this bug without noticing. Segmenting a string the ceiling would have excluded is affordable, but not because the call is cheap in isolation. It is that `Array.from(html)` already paid the larger price on the same string, unconditionally, on both sides of this fix. End to end the difference is tens of microseconds either way, within noise. `test/bench/search.bench.ts` gains a case that reaches the ceiling by escaping rather than by length. `.sync/PORTING.md`'s guard entry said the threshold "is crossed 13 characters earlier than a reading of `value.length` suggests". That described the defect, not the design, and is corrected here — a porter reading it could otherwise restore the bug on purpose. Closes #387. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWWrBHgfqGSbeU3V6UuMF8
This was referenced Aug 13, 2026
IgorShevchik
pushed a commit
that referenced
this pull request
Aug 15, 2026
…nput
`sanitizeSnippet` preserved real `<mark>` tags by swapping them for
`\0markO\0`/`\0markC\0`, escaping everything, then swapping back. NUL is an
ordinary character a snippet can carry, so what decided whether markup was
emitted was a string the input could supply.
A whole sentinel forged a tag outright:
sanitizeSnippet('before \0markO\0 after')
// → 'before <mark> after' — one <mark> out, none in
sanitizeSnippet('\0markC\0\0markO\0')
// → '</mark><mark>' — a close ahead of its opener
Worse, and easier: **six** of the seven bytes, immediately before a *real* tag,
were enough. The placeholder this function inserted for that tag completed the
prefix, and the restore step then found a sentinel spanning the two:
sanitizeSnippet('a\0markO<mark>hit</mark>b')
// → 'a<mark>markO\0hit</mark>b'
The genuine highlight moved, and with text on both sides it landed on text it
was never meant to mark. No crafted sequence — one stray fragment ahead of any
highlight. The first description of this defect, in #391 and in this branch's
first revision, said the input had to carry the sentinel itself. That understated
both the trigger and the consequence; an independent fuzzing pass found the
prefix case.
The function's own doc says the preserved tag is hardcoded *"on purpose: taking
it as a parameter would let a caller pass any tag through to the `v-html` that
renders the result."* The intent was right; the mechanism did not hold it.
Not XSS, and the bound is worth stating because it is what keeps this a content
bug: `escapeHTML` ran before the swap back, so content inside a forged region
stayed escaped, and the emitted tag came from a fixed literal rather than from a
capture group — no attribute could land inside it. Forgeable surface: the two
strings, nothing else. The consequence is spoofed emphasis, relocated highlights
and unbalanced markup reaching `v-html`, in snippets that come from whatever
backend the host application passes to `<ContentSearch>`
(`useContentSearch.ts:141,144`).
Splitting on the real tags removes the guess, and with it the collision. Both
implementations were run over ~2.1M generated inputs by an independent pass:
every divergence traces to this family, and the new one never emitted a tag
other than the two literals nor leaked an unescaped character. The regex has no
quantifiers, and growth is linear to 61MB.
Five tests added, each mutation-checked: dropping the regex capture, dropping the
closing-tag half of the condition, and escaping the tags themselves are all
caught; restoring the old implementation fails exactly the four forgery cases.
One pins the *bound* rather than the bug — a forged tag could never carry an
attribute — so a future rewrite cannot widen that silently.
Pre-existing since `557a5178`, the original port, and untouched by #365, #371
and #388. The function came from upstream unchanged, so `nuxt/ui` very likely
carries it too.
Closes #391.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LWWrBHgfqGSbeU3V6UuMF8
IgorShevchik
pushed a commit
that referenced
this pull request
Aug 15, 2026
…nput
`sanitizeSnippet` preserved real `<mark>` tags by swapping them for
`\0markO\0`/`\0markC\0`, escaping everything, then swapping back. NUL is an
ordinary character a snippet can carry, so what decided whether markup was
emitted was a string the input could supply.
A whole sentinel forged a tag outright:
sanitizeSnippet('before \0markO\0 after')
// → 'before <mark> after' — one <mark> out, none in
sanitizeSnippet('\0markC\0\0markO\0')
// → '</mark><mark>' — a close ahead of its opener
Worse, and easier: **six** of the seven bytes, immediately before a *real* tag,
were enough. The placeholder this function inserted for that tag completed the
prefix, and the restore step then found a sentinel spanning the two:
sanitizeSnippet('a\0markO<mark>hit</mark>b')
// → 'a<mark>markO\0hit</mark>b'
The genuine highlight moved, and with text on both sides it landed on text it
was never meant to mark. No crafted sequence — one stray fragment ahead of any
highlight. The first description of this defect, in #391 and in this branch's
first revision, said the input had to carry the sentinel itself. That understated
both the trigger and the consequence; an independent fuzzing pass found the
prefix case.
The function's own doc says the preserved tag is hardcoded *"on purpose: taking
it as a parameter would let a caller pass any tag through to the `v-html` that
renders the result."* The intent was right; the mechanism did not hold it.
Not XSS, and the bound is worth stating because it is what keeps this a content
bug: `escapeHTML` ran before the swap back, so content inside a forged region
stayed escaped, and the emitted tag came from a fixed literal rather than from a
capture group — no attribute could land inside it. Forgeable surface: the two
strings, nothing else. The consequence is spoofed emphasis, relocated highlights
and unbalanced markup reaching `v-html`, in snippets that come from whatever
backend the host application passes to `<ContentSearch>`
(`useContentSearch.ts:141,144`).
Splitting on the real tags removes the guess, and with it the collision. Both
implementations were run over ~2.1M generated inputs by an independent pass:
every divergence traces to this family, and the new one never emitted a tag
other than the two literals nor leaked an unescaped character. The regex has no
quantifiers, and growth is linear to 61MB.
Five tests added, each mutation-checked: dropping the regex capture, dropping the
closing-tag half of the condition, and escaping the tags themselves are all
caught; restoring the old implementation fails exactly the four forgery cases.
One pins the *bound* rather than the bug — a forged tag could never carry an
attribute — so a future rewrite cannot widen that silently.
Pre-existing since `557a5178`, the original port, and untouched by #365, #371
and #388. The function came from upstream unchanged, so `nuxt/ui` very likely
carries it too.
Closes #391.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LWWrBHgfqGSbeU3V6UuMF8
IgorShevchik
added a commit
that referenced
this pull request
Aug 15, 2026
…nput (#405) `sanitizeSnippet` preserved real `<mark>` tags by swapping them for `\0markO\0`/`\0markC\0`, escaping, then swapping back. NUL is an ordinary character a snippet can carry, so what decided whether markup was emitted was a string the input could supply. A whole sentinel forged a tag outright. Worse and easier: six of its seven bytes immediately before a real tag were enough, because the placeholder inserted for that tag completed the prefix — so `a\0markO<mark>hit</mark>b` came back as `a<mark>markO\0hit</mark>b`, the genuine highlight landing on text it was never meant to mark. No crafted sequence, one stray fragment ahead of any highlight. Not XSS: escaping ran before the swap back, so content inside a forged region stayed escaped, and the emitted tag came from a fixed literal rather than a capture group — no attribute could land inside it. Forgeable surface was the two strings and nothing else. The consequence is spoofed emphasis, relocated highlights and unbalanced markup reaching `v-html`, in snippets that come from whatever backend the host application passes to `<ContentSearch>`. Splitting on the real tags removes the guess, and with it the collision. Independently fuzzed at ~2.1M inputs against both the old implementation and a reference scanner using neither `split` nor `replaceAll`: no bypass, no unescaped character, no tag other than the two literals. The regex has no quantifiers; growth is linear to 61MB and costs 2.15µs against the old 2.09µs at realistic snippet size. Five tests added, each mutation-checked by running it: dropping the regex capture, dropping either half of the condition, escaping the tags themselves and returning the input untouched are all caught, and restoring the old implementation fails exactly the forgery cases. Pre-existing since `557a5178`, the original port; untouched by #365, #371 and #388. Closes #391. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWWrBHgfqGSbeU3V6UuMF8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linked issue
Closes #364.
Type of change
Description
Code points are not what a reader sees. A flag is two regional indicators, a family emoji several joined by ZWJ,
किa consonant plus a vowel sign. Cutting inside one produces a different character rather than a broken one — 🇺🇸 truncated by a single code point re-pairs into 🇸🇺, a different country, with nothing to signal the loss. That is worse than a�, which at least looks wrong.Both boundaries snap off the straddled cluster: mark insertion in
generateHighlightedText, truncation intruncateHTMLFromStart.Chosen on measurement.
Segments.containing()The window is worth knowing about, because it is the obvious optimisation and looks ideal. A run of flags is longer than the window, so segmentation starts mid-run and re-pairs the indicators — reproducing the exact bug being fixed, 28 disagreements in 1044 probes.
So
containing()ships, behind three guards:< U+0300screen, keeping ASCII and Latin-1 off the segmenter. It does nothing for Cyrillic (U+0430) or CJK — both above the floor. For a product localised into Russian this covers markup and Latin identifiers, not body text. perf(search): the grapheme-snap fast path skips Cyrillic and CJK — every boundary reaches the segmenter #376 tracks widening it.value.lengthsuggests.pnpm run bench(test/bench/search.bench.ts): ASCII baseline, Cyrillic and CJK 2.3×, emoji 2.0×, an all-flag run 6.1×.The snap was the easy half
Every defect review found was in the bookkeeping around it, not in the segmentation — and each one reached the user as duplicated or vanished text rather than as an error, because
substring()swaps a range it finds reversed and clamps one it finds out of range instead of throwing.Four, in the order they were found:
<mark></mark>, highlight lostsubstring()did[[0,5],[1,2]]onabcdefghgave<mark>abcdef</mark>defdefgh— the overlap emitted three timesNaNcompares false against every guard includingend > start, then lands in the cursor, wheresubstring(NaN)reads assubstring(0)and repeats the whole valueAll four are guarded, each by a test verified to fail with its guard removed. Both round-3 defects are reachable through
CommandPaletteGroup.postFilter, which lets a caller supply its ownmatches— Fuse itself always emits sorted, merged, integer-bounded regions.indicesare now filtered to integer bounds and sorted before use. The sort's tie-break puts the longest of an equal-start pair first, so the outer region is marked whole instead of being split across two<mark>s.One clamp went the other way. Mutation testing showed
Math.min(…, value.length)onstartwas unreachable —startcan only exceed the value by exceedingend, which is checked — so it was removed rather than left as an untested guard.Coverage
Every constant and branch was verified by removing it and watching a named test fail. That is what closed the two gaps this PR previously listed as known-and-unfixed:
Intl.Segmenternever sees, because the< U+0300screen answers that boundary first — so all three of its mutations passed the whole file.highlight()through any label built from a byte-truncated field. All four widening mutations passed.One mutation still survives by design: removing the
< U+0300screen entirely. It is a performance guard and changes no output — the bench is what measures it.Remaining caveat
The bench is advisory: no assertions, not in CI.
Notes
truncateHTMLFromStartcounts retained code units and slices once, because the snap needs an index to move. Verified equivalent over 400,078 fuzzed inputs, including strings beginning with....A fixture trap is recorded in the tests: UAX #29 glues a run of bare emoji modifiers into a single cluster, so
'\u{1F3FF}'.repeat(20)is one character, not twenty.Checklist
Full suite green on the rebase — 6304 passed, 6 skipped, 270 files. Each new case was verified failing against the unpatched version first.
eslint .,vue-tsc --noEmitandpnpm buildclean.