Skip to content

Various pcre-printing fixups - #547

Merged
katef merged 22 commits into
mainfrom
kate/print-pcre-fixups
Nov 17, 2026
Merged

katef merged 22 commits into
mainfrom
kate/print-pcre-fixups

Conversation

@katef

@katef katef commented Nov 16, 2026

Copy link
Copy Markdown
Owner

re -l pcre renders out the AST to pcre syntax.

Previously that was only a rough approximation. This PR is a pass at making it correct.
It's probably still not correct. But still.

Several essentially unrelated improvements here, especially around the bitmap construction.
As good an example as any:

; bmake -r CC=gcc DEBUG=1 && ./build/bin/re -br pcre -l pcre '(?:(?i)[M-Za-z]|(?i)[M-Za-z]|[^\x00-\xff])'
(?i)[a-z]

katef added 22 commits November 15, 2026 01:22
This is implicit in the pcre syntax. But exists none the less.
`a{1}{2}` would be a syntax error, and `a*?` or whatever other kind of bewildering syntax pcre has is not what we mean.
The goal here is to emit pcre syntax, this is pcre syntax. Don't look at me like that.
This avoids the syntax "leak" problem around local flags within a group. But does give us a possibly separate bug, where I think we might drop flags at the top level. I'm not sure. I'll think about that separately.

I like that this means I don't need to pass group0 all the way through the recursion though.
`x(?![\x00-\xff])a|b|cy` is strictly worse than the input of `x[^abc]y` but these are independent issues.
Always with the off by ones. This and fencepost errors.

This was visible e.g. before:
```
; ./build/bin/re -bpl dot '[A-z]' | grep A-Z
	S0  -> S1  [ weight = 3, label = <[A-Z\-z]> ];
```
where `[` should be present after `Z` but is missing.

That's two ranges, that the `\` is not escaped is due to the dot syntax for `<...>` labels.
Don't be distracted by the `[...]` syntax for the class, that's coincidental.

after:
```
; ./build/bin/re -bpl dot '[A-z]' | grep A-Z
	S0  -> S1  [ weight = 3, label = <[A-Z[-z]> ];
```
…e where possible.

I'm surprised I didn't make this already. But then I try to avoid things that sometimes succeed. I'm not sure. Maybe we could use this during AST -> NFA compilation and skip some work there by recurring through the tree here instead, sort of like lookahead, but recursively. for literals.
This is roughly the same idea as for the labels on the FSM dot output, except instead of populating a bitmap from a set of edges, here we're recuring through subtrees. And here we also have the possibility that some things are not expressible, so we have to fall through to those too.
This was a harmless bug, we just over-allocated for every bitmap. Sigh.
… of `ast_expr_charset()`.

I figure it has to be here for sake of AST_EXPR_SUBTRACT, otherwise I end up implementing the same thing twice when recurring in the caller.

While I'm here I've made an attempt to fill out the other cases for expr nodes, even though we don't construct them. These are best effort; it's always okay to decline to construct a bitmap here.

There's some complication around nullable children of alts here, I have tried to explain it in comments. I'm pretty sure we need both places.
This is one of those pay-off commits where the prior run of work all comes together. I love this stuff.
This gives stuff like:
```
; ./build/bin/re -br pcre -l pcre -i '[M-Za-z]'
[a-z]
```
which is lovely. but I have not attempted to address nested flags yet:
```
; ./build/bin/re -br pcre -l pcre '(?:(?i)[M-Za-z])'
[A-Za-z]
```
This is by demogran's theorem for the same conditions with `continue;` above. This really isn't clear, because the `||` in the top loop is spelled out as two consecutive blocks, and the structure for both doesn't correspond. I'm trying to fix the bug here, and I'll see if I can rephrase it later. This is what my "awful" comment is about...
… for charset output.

This was just too offensive to stand. It's still more awkward than I'd like, but at least it doesn't have the internal structural mismatch between two loops now.
…literal children.

This gets us:
```
; re -br pcre -l pcre '(?i)(?:x|(?-i)((?i)[ab]))'
(?i)[abx]
```

which I think is pretty cool.

@katef katef left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good job kate 👍

@katef
katef merged commit 7dfecd2 into main Nov 17, 2026
346 checks passed
@katef
katef deleted the kate/print-pcre-fixups branch November 17, 2026 20:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant