Skip to content

[interpreter/spec] Fix a couple of UTF-8 issues - #2295

Merged
rossberg merged 2 commits into
mainfrom
fux.unicode
Oct 21, 2026
Merged

rossberg merged 2 commits into
mainfrom
fux.unicode

Conversation

@rossberg

Copy link
Copy Markdown
Member

Fixes #2284 and fixes #2287.

@rossberg
rossberg requested a review from tlively October 21, 2026 13:33
0xc0 lor (n lsr 6) :: con n :: encode' ns
| n::ns when n >= 0xd800 && n < 0xe000 ->
raise Utf8
| n::ns when n < 0x10000 ->

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a duplicate of the next case.

Comment thread interpreter/text/lexer.mll Outdated
@@ -37,7 +37,10 @@ let string s =
let j = !i + 2 in
i := String.index_from s j '}';
let n = int_of_string ("0x" ^ String.sub s j (!i - j)) in

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gemini tells me this will raise Failure "int_of_string" for overly long hex sequences, e.g. \u{ffffffffffff}.

(assert_malformed
(module quote "(func (export \"\\u{D800}\"))")
"malformed UTF-8 codepoint"
)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to add a negative test for \u{DFFF} and a positive test for \u{E000} as well. Might as well do \u{D7FF} as well.

@rossberg

Copy link
Copy Markdown
Member Author

Done, PTAL.

@rossberg
rossberg merged commit 2b34c73 into main Oct 21, 2026
13 checks passed
@rossberg
rossberg deleted the fux.unicode branch October 21, 2026 22:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[interpreter] Text lexer accepts surrogate escapes and crashes on escapes past U+10FFFF [spectec] Tstringchar excludes \u{E000}..\u{E7FF}

2 participants