Skip to content

Stop greedy choice after full input match#26

Merged
joente merged 1 commit into
cesbit:masterfrom
cdamours:stop-greedy-choice-after-full-input-match
Jul 6, 2026
Merged

Stop greedy choice after full input match#26
joente merged 1 commit into
cesbit:masterfrom
cdamours:stop-greedy-choice-after-full-input-match

Conversation

@cdamours

@cdamours cdamours commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Background

While investigating parser performance in an internal application, we noticed that a successful parse can still perform significant unnecessary work. Even when an early Choice element matches and consumes the entire input, the parser continues to walk the remaining elements. Our profiling confirmed numerous _walk calls occurring at the same input position after a complete match had already been established.

Issue

A most-greedy Choice selects the valid element that advances furthest through the input. Once the current best match consumes the full input, no later element can advance further. Continuing the scan in these cases causes additional computation with no effect on the final result.

Proposed Change

I suggest modifying Choice._most_greedy_result to stop walking elements once the current best match reaches end-of-input. This preserves existing behavior while avoiding redundant work. In our internal tests, this change successfully reduced the fixed cost of parsing short, complete statements.

Steps to Reproduce

You can reproduce this behavior with a grammar containing a large most-greedy choice where an early element matches the full string:

from pyleri import Choice, Grammar, Keyword, Regex, Sequence, Token

class ManyAlternativesGrammar(Grammar):
    comparison = Choice(
        Sequence(Keyword("field_a"), Token("="), Regex(r"\d+")),
        Sequence(Keyword("field_b"), Token("="), Regex(r"\d+")),
        Sequence(Keyword("field_c"), Token("="), Regex(r"\d+")),
        # ... additional elements ...
    )
    START = comparison

grammar = ManyAlternativesGrammar()
grammar.parse("field_a=123")

Currently, the parser evaluates every element even after "field_a=123" is matched. I have also verified that this change continues to correctly select the longest choice when multiple valid parsing solutions exist.

A most-greedy Choice selects the valid alternative that advances furthest through the input. Once an alternative reaches the end of the input, no later alternative can produce a longer match.

This improves grammars with large Choice elements where an early alternative fully matches a complete statement. The parser can avoid walking the remaining alternatives while preserving most-greedy semantics.
@joente

joente commented Jul 3, 2026

Copy link
Copy Markdown
Member

Thanks for the pull request! I agree that it makes sense to break once _len_string is reached. I'll take a closer look next week and merge it once I've verified it doesn't break anything.

@joente joente merged commit 85829f8 into cesbit:master Jul 6, 2026
5 checks passed
joente added a commit that referenced this pull request Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants