The regex literal lexer (`engine/src/rhs_types/regex/mod.rs:71-111`) tracks a one-bit `in_char_class`
flag to let an unescaped `"` appear inside a `[...]` character class without terminating the literal:
a `[` not already in a class sets it, a `]` while in one clears it, and the closing `"` is only
recognized when the flag is clear. Escaping (`\"`, `\[`) is handled correctly regardless of the
flag's state, and genuinely running out of input while still inside an unclosed class already
produces a proper parse error (`MissingEndingQuote`).
The gap: a `[` that starts one intended character class, followed later by an unrelated `]`
elsewhere in what was meant to be separate syntax, clears the flag early — so the lexer's next
unescaped `"` (not the one meant to close the first literal) is accepted as the terminator, with no
error, while the text in between silently becomes part of one regex pattern.
Note the first literal isn't valid regex syntax on its own — `host ~ "[abc"` alone correctly fails to
parse with `MissingEndingQuote`. The bug is that adding well-formed text after it converts a clean
parse error into a silently different, successfully-compiled filter:
```
host ~ "[abc" or uri matches "d]e"
```
parses without error into a single comparison on `host` whose pattern is literally
`[abc" or uri matches "d]e` — the intended, separate `or uri matches "d]e"` clause never becomes its
own AST node.
The obvious fix (error on end-of-input while still inside a character class) doesn't catch this
specific case, since by the time the lexer reaches the real final `"`, `in_char_class` has already
been cleared by the incidental `]`. This looks like it needs the literal-scanning pass to be properly
aware of character-class grammar (including things like `]` as a literal first character) rather than
approximated with one bit of state — happy to hear if there's a simpler angle I'm missing, or if this
ambiguity is considered acceptable and just needs documenting.
Found with the rust-in-peace pipeline.
The regex literal lexer (`engine/src/rhs_types/regex/mod.rs:71-111`) tracks a one-bit `in_char_class`
flag to let an unescaped `"` appear inside a `[...]` character class without terminating the literal:
a `[` not already in a class sets it, a `]` while in one clears it, and the closing `"` is only
recognized when the flag is clear. Escaping (`\"`, `\[`) is handled correctly regardless of the
flag's state, and genuinely running out of input while still inside an unclosed class already
produces a proper parse error (`MissingEndingQuote`).
The gap: a `[` that starts one intended character class, followed later by an unrelated `]`
elsewhere in what was meant to be separate syntax, clears the flag early — so the lexer's next
unescaped `"` (not the one meant to close the first literal) is accepted as the terminator, with no
error, while the text in between silently becomes part of one regex pattern.
Note the first literal isn't valid regex syntax on its own — `host ~ "[abc"` alone correctly fails to
parse with `MissingEndingQuote`. The bug is that adding well-formed text after it converts a clean
parse error into a silently different, successfully-compiled filter:
```
host ~ "[abc" or uri matches "d]e"
```
parses without error into a single comparison on `host` whose pattern is literally
`[abc" or uri matches "d]e` — the intended, separate `or uri matches "d]e"` clause never becomes its
own AST node.
The obvious fix (error on end-of-input while still inside a character class) doesn't catch this
specific case, since by the time the lexer reaches the real final `"`, `in_char_class` has already
been cleared by the incidental `]`. This looks like it needs the literal-scanning pass to be properly
aware of character-class grammar (including things like `]` as a literal first character) rather than
approximated with one bit of state — happy to hear if there's a simpler angle I'm missing, or if this
ambiguity is considered acceptable and just needs documenting.
Found with the rust-in-peace pipeline.