Skip to content

fix(pdf): align OCR text layer with the scanned image - #708

Merged
farfromrefug merged 4 commits into
mainfrom
fix/63-ocr-text-positioning
Aug 11, 2026
Merged

fix(pdf): align OCR text layer with the scanned image#708
farfromrefug merged 4 commits into
mainfrom
fix/63-ocr-text-positioning

Conversation

@farfromrefug

Copy link
Copy Markdown
Collaborator

Summary

The invisible text layer of exported PDFs did not sit on the raster words: selecting or searching text highlighted the wrong area. Four independent causes, all on our side rather than in Tesseract:

  • OCR ran at paragraph level (RIL_PARA), so one block held many lines under a single box and font size — the text layer could never match the raster by construction. It now runs at line level (RIL_TEXTLINE).
  • Every block was drawn 10pt too high on android: the rect was inflated by a fixed margin and the text aligned to its top.
  • Font size depended on the exporting phone's screen density (text_scale = Screen.mainScreen.scale * 1.4, then a fit-by-shrinking search on android and a StaticLayout reflow on iOS). Same document, different phone, different text layer.
  • Rotated pages used the wrong scale basis — boxes live in the post-rotation OCR image space, but the scale came from the unrotated page size.

Both renderers now draw each block as a single run placed on its baseline and horizontally scaled to cover exactly the detected box: no reflow, no wrapping, no magic constants. findFittingFontSize / drawTextInBox are gone.

Documents scanned before this change keep their paragraph-level data; a block holding several lines shares the box height between them, so they degrade sensibly instead of rendering as one oversized run. A re-OCR is needed for exact placement on those.

Testing

  • npx vitest run — 261 passed
  • npx eslint on the changed files — clean; svelte-check shows no new errors (2 pre-existing ones in app/utils/pkpass.ts)
  • Verified on device by exporting with the renderer's debug flag on (it fills each box and draws the text visible), checking that the boxes hug the lines and the text sits on the baseline

Still worth a look before this leaves draft:

  • the Cyrillic path (the FontCache Roboto branch) — the original report in OCR text positioning is not accurate #63 / [BUG] OCR for German language quite inaccurate #459 is Russian text
  • iOS, which shares the geometry but not the renderer
  • extracted text on pages with very short lines: the confidence/length filters in DocumentOCR.cpp now apply per line instead of per paragraph, so a short low-confidence line (a page number, A-5) can be dropped from the text where it previously survived inside its paragraph

Refs #63

farfromrefug and others added 3 commits August 3, 2026 21:08
Run the OCR at line level (tesseract RIL_TEXTLINE) instead of paragraph level: a paragraph
box holds many lines, so the text layer could never match the raster by construction.

Draw each block as a single run placed on its baseline and horizontally scaled to cover the
detected box, on both platforms. This drops the device density factor (text_scale), the
findFittingFontSize search and the iOS StaticLayout reflow, which all made the placement
depend on the exporting phone. Scale the boxes against the ocr image size so rotated pages
land right too.

Blocks holding several lines (documents scanned before this change) share the box height
between their lines instead of rendering as a single oversized run.

Refs #63

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@farfromrefug farfromrefug added the bug Something isn't working label Aug 11, 2026
@farfromrefug
farfromrefug marked this pull request as ready for review August 11, 2026 13:19
@farfromrefug
farfromrefug merged commit a89eb13 into main Aug 11, 2026
1 check passed
@farfromrefug
farfromrefug deleted the fix/63-ocr-text-positioning branch August 11, 2026 13:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant