Skip to content

fix: keep layout superscripts aligned with baseline numbers - #1392

Open
BetterAndBetterII wants to merge 1 commit into
jsvine:stablefrom
BetterAndBetterII:fix/1380-layout-asterisk-xpos
Open

BetterAndBetterII wants to merge 1 commit into
jsvine:stablefrom
BetterAndBetterII:fix/1380-layout-asterisk-xpos

Conversation

@BetterAndBetterII

Copy link
Copy Markdown

Summary

With extract_text(layout=True), a superscript * sitting just above a baseline number (common footnote / citation markers) was clustered onto its own layout line. That line kept raw PDF x-columns while the baseline line drifted right, so the markers appeared shifted left of 328 / 329 (issue PDF: Against Celsus line).

Two small layout-only adjustments:

  1. Widen vertical line clustering for layout=True with max(y_tolerance, line_density / 2) so near-baseline superscripts share a line with the text they annotate.
  2. X-sort words on layout lines even when presorted=True, so extraction order cannot leave * before the number it belongs to. use_text_flow is unchanged.

Test plan

  • pytest tests/test_utils.py (37 passed), including new synthetic superscript layout regression
  • Reproduced on the issue's page826.pdf: one line Against Celsus (Origen), 327 , 328 * , 329 *

Fixes #1380

layout=True clustered superscript '*' just above a baseline number onto
its own line, leaving raw x-columns while the baseline drifted right so
the markers appeared shifted left. Widen line clustering for layout and
x-sort layout lines so annotated markers stay after their numbers.

Fixes jsvine#1380
@perigot827

perigot827 commented Sep 14, 2026

Copy link
Copy Markdown

I reproduced a regression on head d2fd6de017953297fb965622130f23d28b65d7fa: the increased line tolerance merges two separate, nonoverlapping rows, even with explicit y_tolerance=0.

The test below uses 4-point characters on rows 5 points apart, leaving a 1-point vertical gap. It returns ['Alpha', 'Beta'] on base 4c64b92d5caccd71c645e98e0fabb0c4dba7ff45, but ['Alpha Beta'] on this head. All four parameter cases pass on the base and fail on the head. The same result reproduces through Page.extract_text with an original PDF generated using ReportLab, so it is not limited to synthetic character input.

At default y_density=13, max(y_tolerance, line_density / 2) forces a 6.5-point tolerance. This also changes output when use_text_flow=True. With layout=False or y_density=4, both revisions preserve the rows in this example.

Could the superscript handling preserve the caller's ability to keep distinct rows separate? This regression test needs no additional dependency beyond pytest and the library:

"""Regression coverage for pdfplumber PR 1392; no additional dependency."""

import pytest

from pdfplumber import utils


def make_char(text, x0, top):
    return {
        "text": text,
        "x0": x0,
        "x1": x0 + 2,
        "top": top,
        "doctop": top,
        "bottom": top + 4,
        "width": 2,
        "height": 4,
        "size": 4,
        "upright": True,
    }


@pytest.mark.parametrize("use_text_flow", [False, True])
@pytest.mark.parametrize("y_tolerance", [0, 3])
def test_layout_preserves_separate_nonoverlapping_rows(use_text_flow, y_tolerance):
    chars = [
        make_char(letter, 30 + 2 * index, top)
        for word, top in [("Alpha", 100), ("Beta", 105)]
        for index, letter in enumerate(word)
    ]
    text = utils.extract_text(
        chars,
        layout=True,
        layout_bbox=(0, 0, 200, 200),
        layout_width=200,
        layout_height=200,
        use_text_flow=use_text_flow,
        y_tolerance=y_tolerance,
    )
    assert [line.strip() for line in text.splitlines() if line.strip()] == [
        "Alpha",
        "Beta",
    ]

I also ran the existing tests/test_utils.py: 36 tests pass on the base and 37 on this head, including the new superscript case. I did not run the entire suite. This report was prepared with AI assistance and its reproduction was executed locally on both revisions.

Candidate amendment: patch and reproduction notes. This experimental amendment applies to head d2fd6de017953297fb965622130f23d28b65d7fa. In local checks it preserves the issue PDF's superscript placement and passes 18 extended distinct-row regression cases. The candidate's full suite passed 190 tests with Ghostscript 10.08.0, and project lint passed. These additional results concern the candidate amendment; the base/head review results above are unchanged. The approach uses conservative geometry checks and is not a general semantic-line guarantee.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

page.extract_text(layout=True) incorrectly reproducing text

2 participants