Repository navigation
Conversation
Motivation: When a header value byte looks like the lead byte of a 2-, 3- or 4-byte UTF-8 sequence but the bytes after it do not complete one, the header parser copied the value as-is but still skipped the full sequence length. At the end of a value that skipped the CR/LF, so the next header line was read as part of this value (and a bare CR could end up being consumed in the 2-byte case). Modification: On a failed decode, take only the lead byte and the continuation bytes (10xxxxxx) directly after it as the sequence and copy them one char per byte, as before. Any other byte, including CR and LF, is left for the next step of the scan. The three per-length branches are folded into one. Result: A header value ending in a stray non-ASCII byte keeps its line end, and the following header is parsed as its own header. Valid UTF-8 and invalid sequences made only of continuation bytes decode as before. Tests: - http-core/testOnly HttpHeaderParserCRLFSpec HttpHeaderParserLFSpec: passed; the two new tests fail without the fix References: None - found while looking at HTTP/2 header value decoding
Motivation: HPACK decodes string literals as ISO-8859-1, one char per octet. RequestParsing.parseHeaderPair then encoded the header line as UTF-8 before handing it to the HTTP/1.1 header parser, which encoded every octet above 0x7F as a char of its own. A UTF-8 value such as "café" came out as "café" over HTTP/2, while HTTP/1.1 gives "café". Modification: Encode the line as ISO-8859-1, so that the parser sees the octets that were on the wire and applies its usual UTF-8 decoding. Octets that are not valid UTF-8 are still copied one char per byte. This relies on the parser keeping the line end after an invalid sequence. Result: Regular header values (request headers, response headers, trailers and the joined cookie header) are decoded the same way over HTTP/2 and HTTP/1.1. Tests: - http2-tests/testOnly RequestParsingSpec client.ResponseParsingSpec: passed References: None - builds on the HTTP/1.1 header value fix
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation:
HPACK decodes string literals as ISO-8859-1, one char per octet.
RequestParsing.parseHeaderPair then encoded the header line as UTF-8
before handing it to the HTTP/1.1 header parser, which encoded every
octet above 0x7F as a char of its own. A UTF-8 value such as "café"
came out as "café" over HTTP/2, while HTTP/1.1 gives "café".
Modification:
Encode the line as ISO-8859-1, so that the parser sees the octets that
were on the wire and applies its usual UTF-8 decoding. Octets that are
not valid UTF-8 are still copied one char per byte. This relies on the
parser keeping the line end after an invalid sequence.
Result:
Regular header values (request headers, response headers, trailers and
the joined cookie header) are decoded the same way over HTTP/2 and
HTTP/1.1.
Tests:
passed
References:
Depends on #1499; this branch contains its commit and should be rebased once that merges. Not covered:
content-type,:pathand:authority, which HeaderDecompression parses directly from the ISO-8859-1 string.🤖 Generated with Claude Code