Skip to content

Avoid rescanning unescaped JSON blobs - #730

Open
dantrapp wants to merge 1 commit into
aws:mainfrom
dantrapp:perf/json-blob-decoding
Open

dantrapp wants to merge 1 commit into
aws:mainfrom
dantrapp:perf/json-blob-decoding

Conversation

@dantrapp

Copy link
Copy Markdown

Description of changes:

ReadBlob currently unquotes every JSON string and copies it into a temporary string before base64 decoding. The parser has already checked for escapes, so ordinary unescaped blobs incur a redundant scan and copy.

Decode unescaped base64 directly from the token bytes with AppendDecode. Escaped strings still pass through UnquoteBytes. The decoded result owns its storage, empty strings produce non-nil empty slices, and decoding failures leave the destination unchanged.

The Kinesis GetRecords benchmark exercises the actual SDK client, middleware, response buffering, JSON parsing, and blob decoding through a mock HTTP client. Each fixture's decoded bytes are verified before timing. Median results from six alternating baseline/candidate trials:

Response Before After Allocated bytes
1 × 64 B 8.946 µs 9.081 µs −0.55%
100 × 1 KiB 264.7 µs 129.8 µs −18.30%
500 × 4 KiB 3.891 ms 1.370 ms −24.51%
100 × 64 KiB 10.833 ms 3.902 ms −19.66%
100 × 4 KiB, escaped slashes 1.485 ms 1.318 ms −17.84%

Go 1.26.2, Apple M4, darwin/arm64, GOMAXPROCS=4, 500 ms per sub-benchmark. The small-response timing difference is not statistically significant. These generated workloads exclude network latency and AWS service processing; timing variability is included in the reproduction instructions, benchmark source, and raw results.

Validation:

  • GOMAXPROCS=4 make unit-race: all module vet and race checks passed.
  • FuzzReadBlob: 2,448,545 executions over 30 seconds, comparing decoded bytes, nil/empty results, and base64 error offsets with encoding/json.
  • Unit tests cover padding, escapes, invalid input, unchanged destinations on failure, and output ownership. Decoder benchmarks cover escaped and unescaped blobs.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@dantrapp
dantrapp requested review from a team as code owners October 10, 2026 23:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant