fix(pdf): align OCR text layer with the scanned image - #708
Merged
Conversation
Run the OCR at line level (tesseract RIL_TEXTLINE) instead of paragraph level: a paragraph box holds many lines, so the text layer could never match the raster by construction. Draw each block as a single run placed on its baseline and horizontally scaled to cover the detected box, on both platforms. This drops the device density factor (text_scale), the findFittingFontSize search and the iOS StaticLayout reflow, which all made the placement depend on the exporting phone. Scale the boxes against the ocr image size so rotated pages land right too. Blocks holding several lines (documents scanned before this change) share the box height between their lines instead of rendering as a single oversized run. Refs #63 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The invisible text layer of exported PDFs did not sit on the raster words: selecting or searching text highlighted the wrong area. Four independent causes, all on our side rather than in Tesseract:
RIL_PARA), so one block held many lines under a single box and font size — the text layer could never match the raster by construction. It now runs at line level (RIL_TEXTLINE).text_scale = Screen.mainScreen.scale * 1.4, then a fit-by-shrinking search on android and aStaticLayoutreflow on iOS). Same document, different phone, different text layer.Both renderers now draw each block as a single run placed on its baseline and horizontally scaled to cover exactly the detected box: no reflow, no wrapping, no magic constants.
findFittingFontSize/drawTextInBoxare gone.Documents scanned before this change keep their paragraph-level data; a block holding several lines shares the box height between them, so they degrade sensibly instead of rendering as one oversized run. A re-OCR is needed for exact placement on those.
Testing
npx vitest run— 261 passednpx eslinton the changed files — clean;svelte-checkshows no new errors (2 pre-existing ones inapp/utils/pkpass.ts)debugflag on (it fills each box and draws the text visible), checking that the boxes hug the lines and the text sits on the baselineStill worth a look before this leaves draft:
FontCacheRoboto branch) — the original report in OCR text positioning is not accurate #63 / [BUG] OCR for German language quite inaccurate #459 is Russian textDocumentOCR.cppnow apply per line instead of per paragraph, so a short low-confidence line (a page number,A-5) can be dropped from the text where it previously survived inside its paragraphRefs #63