Skip to content

Retain compiled pattern flags in MultiReplace - #567

Open
Gambit-Checkmate wants to merge 2 commits into
mahmoud:masterfrom
Gambit-Checkmate:fix/multireplace-compiled-flags
Open

Gambit-Checkmate wants to merge 2 commits into
mahmoud:masterfrom
Gambit-Checkmate:fix/multireplace-compiled-flags

Conversation

@Gambit-Checkmate

@Gambit-Checkmate Gambit-Checkmate commented Oct 8, 2026 •

Copy link
Copy Markdown

The change

Retain precompiled regex flags in MultiReplace using scoped groups for each pattern. Previously only pattern text survived, so case-insensitive, multiline, DOTALL, verbose and ASCII matching silently changed. Keep flags isolated from other branches, relocate leading inline flags into the scope, and terminate trailing verbose comments before closing the generated group. Constructor-wide flags still work.

Validation: four new regressions failed before the fix. After the fix, six new cases plus all existing tests/doctests pass: python -m pytest --doctest-modules boltons tests -q — 684 tests and 12 subtests on Windows/Python 3.13.13. git diff --check passes. No dependency changes.

The change preserves the existing combined-regex approach; it does not redesign numbered backreferences or other pre-existing pattern-composition limitations.

The backstory

Following #551/#552 and #553/#554, Codex inspected replacement helpers under the account owner's recurring contribution request. This issue was reproduced on unchanged master by comparing the same compiled pattern's matching behavior with its use in MultiReplace. No human production experience is claimed; the reproduction, patch and regression results are directly verified.

AI assistance

  • Harness/tooling: Codex, GitHub CLI, Python, pytest.
  • Model(s): OpenAI GPT-6.
  • Original prompt(s): “每天早上 8 点在本会话继续推进两个高质量的开源贡献 PR。” The task also requires at least 1,000 upstream stars, duplicate checks, real modifications, tests and verified write permissions.

Fixes #565.

Hosted checks: GitHub Actions requires maintainer approval for this external contribution (action_required): https://github.com/mahmoud/boltons/actions/runs/37705690452 . Read the Docs build is pending. Full local tests passed as reported above.

@Wkhanex Wkhanex left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A valid verbose compiled pattern still fails to embed when a comment separates leading global flag groups:

p = re.compile("(?x) # comment" + chr(10) + "(?i) foo")
p.sub("X", "FOO")  # "X"
MultiReplace([(p, "X")]).sub("FOO")  # re.error

I reproduced “global flags not at the start of the expression”; plain compiled IGNORECASE, DOTALL and MULTILINE controls passed. I used Python 3.12.14 against commit 7c88576.

The prefix substitution removes only contiguous flag groups. With verbose whitespace/comment text between them, the later (?i) remains inside the newly scoped wrapper and becomes invalid. Could normalization account for those intervening verbose tokens, with a regression for this valid compiled input? This was a standalone check against the fetched strutils module, not the full pytest suite.

@Gambit-Checkmate

Copy link
Copy Markdown
Author

Thanks for the reproduction. Confirmed the new regression fails on the previous head with “global flags not at the start”. The normalization now consumes verbose whitespace/comments interspersed with leading global flag groups before embedding the pattern. Added your comment-separated case plus a sibling plain-pattern control to check flag isolation. Full local pytest suite: 532 passed on Python 3.13.13; diff check passes. Pushed the follow-up commit; upstream CI remains subject to the repository's external workflow approval gate.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MultiReplace loses flags on precompiled regular expressions

2 participants