Stop greedy choice after full input match#26
Merged
joente merged 1 commit intoJul 6, 2026
Conversation
A most-greedy Choice selects the valid alternative that advances furthest through the input. Once an alternative reaches the end of the input, no later alternative can produce a longer match. This improves grammars with large Choice elements where an early alternative fully matches a complete statement. The parser can avoid walking the remaining alternatives while preserving most-greedy semantics.
Member
|
Thanks for the pull request! I agree that it makes sense to break once |
joente
added a commit
that referenced
this pull request
Jul 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
While investigating parser performance in an internal application, we noticed that a successful parse can still perform significant unnecessary work. Even when an early
Choiceelement matches and consumes the entire input, the parser continues to walk the remaining elements. Our profiling confirmed numerous_walkcalls occurring at the same input position after a complete match had already been established.Issue
A most-greedy
Choiceselects the valid element that advances furthest through the input. Once the current best match consumes the full input, no later element can advance further. Continuing the scan in these cases causes additional computation with no effect on the final result.Proposed Change
I suggest modifying
Choice._most_greedy_resultto stop walking elements once the current best match reachesend-of-input. This preserves existing behavior while avoiding redundant work. In our internal tests, this change successfully reduced the fixed cost of parsing short, complete statements.Steps to Reproduce
You can reproduce this behavior with a grammar containing a large most-greedy choice where an early element matches the full string:
Currently, the parser evaluates every element even after "field_a=123" is matched. I have also verified that this change continues to correctly select the longest choice when multiple valid parsing solutions exist.