parser: apply the offside rule to computation-expression items - #77
Merged
Conversation
Inside a CE's braces, an item's expression now owns the rest of its line plus any following lines indented past the item; the first line that is not is examined as the next item. A bare statement line after a binding is rejected with the existing "expected `let!`, `let`, `do!`, `return`, or `yield`" message on that line, instead of being parsed as application arguments and surfacing as a type error on the line above. Tokens record their line-opening column and bracket depth, so the parser can judge the boundary only directly inside the CE's braces; brackets nested within an item keep treating line breaks as continuations, and lines led by operators still continue the item.
The indented spelling of the issue #64 repro still produced the misleading type error: a statement line indented past its item is a continuation under the item-boundary rule, so it was read as application arguments. Inside a CE, at the CE's item depth, an application argument that opens a source line must now start at or right of the function expression it attaches to (F#'s FS0058 offside rule, applied to application). The error points at the offending token and names the column that continues the expression and the column that starts a new item. Measured first: the rejected shape appears nowhere in the repo (tests, examples, lessons, doc code blocks, playground samples, tree-sitter fixtures).
The message named two columns; the expression column landed on a token that could never take an argument, and the suggested un-indent produced a second error, since an atom start can never begin a CE item. The error now names the item being continued by keyword and source line (line-opening tokens carry their line number, counted incrementally in the lexer) and offers the two accurate fixes: align the line with the item and bind its value (`let _ = ...`), or indent it past the start of the expression to continue it.
…rors CompileError::message() now returns the bare message field for the Lex and Parse variants, as its doc always claimed; their Display impls keep the (at N..M) suffix for the paths that print an error with no source to render against, and the one such path (a project module whose file cannot be re-read) prints the Display form. to_type_error carries the bare message too, so the LSP publishes it to the editor alongside the range, and the REPL and kernel render it without the stage prefix and offsets.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A bare statement is not a CE item, and the parser said so in first position only. Anywhere after a binding, the binding's value expression ran through the newline and took the next line as application arguments, so the failure surfaced as a type error on the wrong line naming a type nobody wrote (
('a ->{io} unit) -> 'bisprintpartially applied).The cause is that the CE's opening brace turns off the lexer's layout rule for the whole block, so no separator token ever hands control back to the item loop.
What changed
Tokens now record their line-opening position (line and column) and the bracket depth in force where they start (
src/lexer/token.rs, patched on in the lexer's main loop). This is the line information that survives into a bracketed region.The parser gives each CE item an offside frame: the item's expression owns the rest of its line plus any following lines indented past the item, and the application-argument loop stops at the first line that is not (
ce_item_ends_hereinsrc/parser/mod.rs). The item loop then examines that line and rejects a bare statement with the existing "expectedlet!,let,do!,return, oryield" message, pointing at the offending line. The repro now reports:A continuation line that does supply an application argument must in turn start at or right of the function expression it attaches to (
check_ce_argument_alignment). This is F#'s FS0058 offside rule applied to application, and it closes the residual: indenting the statement past the item made it a continuation again, and the misleading type error came back. That spelling now reports, at the offending token:The message names the item being continued rather than columns, and its step fix includes the binding, because an atom start can never begin a CE item, so un-indenting alone would trade one error for another. It is an error, not a warning: the parse phase returns a
Resultwith no warning channel, and the shape is close to certainly unintended, since the argument reading resurfaces as the wrong-line type error this PR removes. Measured before implementing: the rejected shape appears nowhere in the repo. The full test suite, every example, all 23 lessons, all 156 fenced code blocks across the tracked markdown (README, DESIGN, INTERNALS, docs site), the playground samples, and the tree-sitter fixtures parse identically under the old and new binaries, and a second measurement over a private 1,867-line dogfooded application across 9 modules found zero behavior differences.Both rules are judged only directly inside the CE's braces, at the recorded bracket depth. Brackets nested within an item keep treating line breaks as continuations, lines led by operators (
|>and friends) continue the item as they do at top level, and single-line CE blocks are untouched. All four built-in builders and user-defined builders go through the same item loop, so the rules cover them uniformly.Rendered lex and parse errors no longer append the raw
(at N..M)byte span.CompileError::message()returns the bare message for theLexandParsevariants, as its doc always claimed; the--> line:colheader and the caret already carry the location. The bare message also flows throughto_type_error, so the LSP publishes it to the editor alongside the range, and the REPL and kernel render it without the stage prefix and offsets. TheDisplayimpls keep the span for paths with no source to render against, and the one such path (a project module whose file cannot be re-read) prints that form.DESIGN.mddocuments both halves of the item-level offside rule in section 8.1 and cross-references it from the layout section.Tests
a_statement_line_after_a_ce_binding_is_rejected_as_an_item: the issue repro, asserting the message and that the span lands on theprintline.an_offside_argument_line_inside_a_ce_names_both_readings: the indented spelling of the repro and the bracket-opening shape, asserting the message names the item's keyword and line, offers thelet _ = ...fix, contains no column numbers, and spans the offending token.every_builder_rejects_a_statement_line_after_any_item: the same shape afterletinseq,do!inasync,let!inoption,yieldinseq, and in a user-defined builder.a_ce_item_expression_may_span_lines_indented_past_the_item: a value continued on deeper lines, a pipeline broken across lines, a multi-linematch, and a nested CE all keep parsing.a_line_opening_argument_at_its_functions_column_keeps_parsing: pins the alignment boundary, an argument at exactly the function's column is a continuation.brackets_inside_a_ce_item_still_continue_across_lines: a line break inside parentheses never separates, whatever its column, under both rules.tokens_carry_line_column_and_bracket_depth: pins the new token fields, line numbers included.cargo test(1,044 tests),cargo clippy --all-targets, andcargo fmt --checkare clean; every example type-checks anddocs/verify_lessons.pypasses all 23 lessons against the fixed compiler.Closes #64