[JuliaSyntax] Remove separate syntax heads for each operator#62000
Merged
Conversation
Keno
force-pushed
the
kf/syntax-rmopkinds
branch
4 times, most recently
from
June 4, 2026 19:51
e75229b to
2f7fab2
Compare
This replaces all the specialized operator heads by a single K"Operator" head that encodes the precedence level in its flags (except for operators that are also used for non-operator purposes). The operators are already K"Identifier" in the final parse tree. There is very little reason to spend all of the extra effort separating them into separate heads only to undo this later. Moreover, I think it's actively misleading, because it makes people think that they can query things about an operator by looking at the head, which doesn't work for suffixed operators. Additionally, this removes the `op=` token, replacing it by two tokens, one K"Operator" with a special precedence level and one `=`. An operator only forms a compound assignment when followed by the single token `=`; when followed by `==`, `===` or `=>` it is emitted as a plain operator token. This then removes the last use of `bump_split` (since this PR is on top of JuliaLang/JuliaSyntax.jl#573). As a free bonus this prepares us for having compound assignment syntax for suffixed operators, which was infeasible in the flips parser. That syntax change is not part of this PR but would be trivial (this PR makes it an explicit error). Because most operators no longer have a dedicated kind, the in-tree consumers of the syntax tree are updated to match: * `_green_to_est` now lowers the one-argument `K"op="`/`K".op="` forms (the quoted compound-assignment operators such as `:(+=)`) to the corresponding identifier name. * JuliaLowering's `is_dotted_operator` no longer looks up an operator by `Kind` name (which no longer exists for most operators); it uses the operator-name test `Base.isoperator`, as already done for `op=` in `est_to_dst`. * JuliaSyntaxHighlighting is updated similarly (detect `===` by text, drop the removed `is_syntactic_assignment`, and recognize the now-kindless operators) and its stdlib version pin is bumped accordingly. Fixes JuliaLang/JuliaSyntax.jl#334 (cherry picked from commit 9c41dfa55e460ff50906644f21050d1352f01f20) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Keno
added a commit
to KenoAIStaging/julia
that referenced
this pull request
Jun 10, 2026
`a .≔ b` (likewise `.⩴` and `.≕`) crashed the parser with an internal "Unexpected dotted operator" error since the dots were split into separate tokens (JuliaLang#61992), because `dotted()` only handled the heads `.= .&& .|| .op=`. Add dedicated `K".≔"`/`K".⩴"`/`K".≕"` heads - matching `K".="` - so these parse to `(.≔ a b)` and convert to `Expr(:.≔, a, b)` as in the reference parser. Dotted `:=` is not valid syntax (the reference parser rejects it too), so emit a proper parse error for it instead of crashing. Also a few smaller items from review of JuliaLang#62000: * The new heads also fix `Expr(:.≔, a, b)` conversion into a `SyntaxTree`, which previously went through JuliaLowering's compound assignment `unknown_head` path. That path now additionally requires a trailing `=` in the head name, so operator-named heads such as a hand-written `Expr(:⊕, a, b)` produce a lowering error rather than being mangled into an `op=` form. * Add `K"Operator"` to `_nonunique_kind_names`, since the kind name doesn't determine the token text. * Remove duplicated `·` (U+00B7) and `·` (U+0387) entries from the `PREC_TIMES` operator table (they are listed - with comments - at the end of the list). * Fix an asymmetry in `Expr` conversion of one-argument `K".op="` vs `K"op="` heads. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Keno
added a commit
to KenoAIStaging/julia
that referenced
this pull request
Jun 10, 2026
`a .≔ b` (likewise `.⩴` and `.≕`) crashed the parser with an internal "Unexpected dotted operator" error since the dots were split into separate tokens (JuliaLang#61992), because `dotted()` only handled the heads `.= .&& .|| .op=`. Add dedicated `K".≔"`/`K".⩴"`/`K".≕"` heads - matching `K".="` - so these parse to `(.≔ a b)` and convert to `Expr(:.≔, a, b)` as in the reference parser. Dotted `:=` is not valid syntax (the reference parser rejects it too), so emit a proper parse error for it instead of crashing. Also a few smaller items from review of JuliaLang#62000: * The new heads also fix `Expr(:.≔, a, b)` conversion into a `SyntaxTree`, which previously went through JuliaLowering's compound assignment `unknown_head` path. That path now additionally requires a trailing `=` in the head name, so operator-named heads such as a hand-written `Expr(:⊕, a, b)` produce a lowering error rather than being mangled into an `op=` form. * Add `K"Operator"` to `_nonunique_kind_names`, since the kind name doesn't determine the token text. * Remove duplicated `·` (U+00B7) and `·` (U+0387) entries from the `PREC_TIMES` operator table (they are listed - with comments - at the end of the list). * Fix an asymmetry in `Expr` conversion of one-argument `K".op="` vs `K"op="` heads. * Bump the JuliaSyntaxHighlighting pin to drop a stray local-path `[sources]` section (left over from testing) from its Project.toml. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[Rebase of https://github.com/JuliaLang/JuliaSyntax.jl/pull/575 plus Claude's updates to JuliaLowering]
This replaces all the specialized operator heads by a single K"Operator" head that encodes the precedence level in its flags (except for operators that are also used for non-operator purposes). The operators are already K"Identifier" in the final parse tree. There is very little reason to spend all of the extra effort separating them into separate heads only to undo this later. Moreover, I think it's actively misleading, because it makes people think that they can query things about an operator by looking at the head, which doesn't work for suffixed operators.
Additionally, this removes the
op=token, replacing it by two tokens, one K"Operator" with a special precendence level and one=. This then removes the last use ofbump_split(since this PR is on top of JuliaLang/JuliaSyntax.jl#573).As a free bonus this prepares us for having compound assignment syntax for suffixed operators, which was infeasible in the flips parser. That syntax change is not part of this PR but would be trivial (this PR makes it an explicit error).
Because most operators no longer have a dedicated kind, the in-tree consumers of the syntax tree are updated to match:
_green_to_estnow lowers the one-argumentK"op="/K".op="forms (the quoted compound-assignment operators such as:(+=)) to the corresponding identifier name.is_dotted_operatorno longer looks up an operator byKindname (which no longer exists for most operators); it uses the operator-name testBase.isoperator, as already done forop=inest_to_dst.Fixes JuliaLang/JuliaSyntax.jl#334
(cherry picked from commit 9c41dfa55e460ff50906644f21050d1352f01f20)