Skip to content

parser: apply the offside rule to computation-expression items - #77

Merged
simontreanor merged 4 commits into
mainfrom
fix/ce-item-offside
Aug 29, 2026
Merged

parser: apply the offside rule to computation-expression items#77
simontreanor merged 4 commits into
mainfrom
fix/ce-item-offside

Conversation

@simontreanor

@simontreanor simontreanor commented Aug 29, 2026

Copy link
Copy Markdown
Owner

A bare statement is not a CE item, and the parser said so in first position only. Anywhere after a binding, the binding's value expression ran through the newline and took the next line as application arguments, so the failure surfaced as a type error on the wrong line naming a type nobody wrote (('a ->{io} unit) -> 'b is print partially applied).

The cause is that the CE's opening brace turns off the lexer's layout rule for the whole block, so no separator token ever hands control back to the item loop.

What changed

  • Tokens now record their line-opening position (line and column) and the bracket depth in force where they start (src/lexer/token.rs, patched on in the lexer's main loop). This is the line information that survives into a bracketed region.

  • The parser gives each CE item an offside frame: the item's expression owns the rest of its line plus any following lines indented past the item, and the application-argument loop stops at the first line that is not (ce_item_ends_here in src/parser/mod.rs). The item loop then examines that line and rejects a bare statement with the existing "expected let!, let, do!, return, or yield" message, pointing at the offending line. The repro now reports:

    error: expected `let!`, `let`, `do!`, `return`, or `yield`, found identifier `print`
     --> 5:5
      |
    5 |     print f"y is {y}"
      |     ^^^^^
    
  • A continuation line that does supply an application argument must in turn start at or right of the function expression it attaches to (check_ce_argument_alignment). This is F#'s FS0058 offside rule applied to application, and it closes the residual: indenting the statement past the item made it a continuation again, and the misleading type error came back. That spelling now reports, at the offending token:

    error: identifier `print` is indented as a continuation of the `let` item on line 4, where it can only be an argument to that item's expression; to make it a new item, align it with that item and bind its value (`let _ = ...`), or indent it past the start of the expression to continue it
     --> 5:7
      |
    5 |       print f"y is {y}"
      |       ^^^^^
    

    The message names the item being continued rather than columns, and its step fix includes the binding, because an atom start can never begin a CE item, so un-indenting alone would trade one error for another. It is an error, not a warning: the parse phase returns a Result with no warning channel, and the shape is close to certainly unintended, since the argument reading resurfaces as the wrong-line type error this PR removes. Measured before implementing: the rejected shape appears nowhere in the repo. The full test suite, every example, all 23 lessons, all 156 fenced code blocks across the tracked markdown (README, DESIGN, INTERNALS, docs site), the playground samples, and the tree-sitter fixtures parse identically under the old and new binaries, and a second measurement over a private 1,867-line dogfooded application across 9 modules found zero behavior differences.

  • Both rules are judged only directly inside the CE's braces, at the recorded bracket depth. Brackets nested within an item keep treating line breaks as continuations, lines led by operators (|> and friends) continue the item as they do at top level, and single-line CE blocks are untouched. All four built-in builders and user-defined builders go through the same item loop, so the rules cover them uniformly.

  • Rendered lex and parse errors no longer append the raw (at N..M) byte span. CompileError::message() returns the bare message for the Lex and Parse variants, as its doc always claimed; the --> line:col header and the caret already carry the location. The bare message also flows through to_type_error, so the LSP publishes it to the editor alongside the range, and the REPL and kernel render it without the stage prefix and offsets. The Display impls keep the span for paths with no source to render against, and the one such path (a project module whose file cannot be re-read) prints that form.

  • DESIGN.md documents both halves of the item-level offside rule in section 8.1 and cross-references it from the layout section.

Tests

  • a_statement_line_after_a_ce_binding_is_rejected_as_an_item: the issue repro, asserting the message and that the span lands on the print line.
  • an_offside_argument_line_inside_a_ce_names_both_readings: the indented spelling of the repro and the bracket-opening shape, asserting the message names the item's keyword and line, offers the let _ = ... fix, contains no column numbers, and spans the offending token.
  • every_builder_rejects_a_statement_line_after_any_item: the same shape after let in seq, do! in async, let! in option, yield in seq, and in a user-defined builder.
  • a_ce_item_expression_may_span_lines_indented_past_the_item: a value continued on deeper lines, a pipeline broken across lines, a multi-line match, and a nested CE all keep parsing.
  • a_line_opening_argument_at_its_functions_column_keeps_parsing: pins the alignment boundary, an argument at exactly the function's column is a continuation.
  • brackets_inside_a_ce_item_still_continue_across_lines: a line break inside parentheses never separates, whatever its column, under both rules.
  • tokens_carry_line_column_and_bracket_depth: pins the new token fields, line numbers included.

cargo test (1,044 tests), cargo clippy --all-targets, and cargo fmt --check are clean; every example type-checks and docs/verify_lessons.py passes all 23 lessons against the fixed compiler.

Closes #64

Inside a CE's braces, an item's expression now owns the rest of its line
plus any following lines indented past the item; the first line that is
not is examined as the next item. A bare statement line after a binding
is rejected with the existing "expected `let!`, `let`, `do!`,
`return`, or `yield`" message on that line, instead of being parsed
as application arguments and surfacing as a type error on the line
above. Tokens record their line-opening column and bracket depth, so
the parser can judge the boundary only directly inside the CE's braces;
brackets nested within an item keep treating line breaks as
continuations, and lines led by operators still continue the item.
The indented spelling of the issue #64 repro still produced the
misleading type error: a statement line indented past its item is a
continuation under the item-boundary rule, so it was read as
application arguments. Inside a CE, at the CE's item depth, an
application argument that opens a source line must now start at or
right of the function expression it attaches to (F#'s FS0058 offside
rule, applied to application). The error points at the offending token
and names the column that continues the expression and the column that
starts a new item. Measured first: the rejected shape appears nowhere
in the repo (tests, examples, lessons, doc code blocks, playground
samples, tree-sitter fixtures).
The message named two columns; the expression column landed on a token
that could never take an argument, and the suggested un-indent produced
a second error, since an atom start can never begin a CE item. The
error now names the item being continued by keyword and source line
(line-opening tokens carry their line number, counted incrementally in
the lexer) and offers the two accurate fixes: align the line with the
item and bind its value (`let _ = ...`), or indent it past the start
of the expression to continue it.
…rors

CompileError::message() now returns the bare message field for the Lex
and Parse variants, as its doc always claimed; their Display impls keep
the (at N..M) suffix for the paths that print an error with no source
to render against, and the one such path (a project module whose file
cannot be re-read) prints the Display form. to_type_error carries the
bare message too, so the LSP publishes it to the editor alongside the
range, and the REPL and kernel render it without the stage prefix and
offsets.
@simontreanor
simontreanor merged commit 0923115 into main Aug 29, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A statement after a binding in a CE block is silently parsed as application arguments

1 participant