Skip to content

extern: caller-supplied keyword slots (kw = ...) - #21

Merged
simontreanor merged 2 commits into
mainfrom
extern-kwarg-slots
Jul 28, 2026
Merged

extern: caller-supplied keyword slots (kw = ...)#21
simontreanor merged 2 commits into
mainfrom
extern-kwarg-slots

Conversation

@simontreanor

Copy link
Copy Markdown
Owner

Closes the Caller-varying keyword arguments backlog item added in #20.

The gap

Pinned kwargs (DESIGN.md §6) accepted only literals fixed at the declaration, so requests.get(url, timeout = t) with a caller-supplied t needed a separate extern per call shape. Python's API culture is optional-kwargs-with-defaults, which made this the boundary friction a real user was likeliest to meet first.

The feature

A ... in place of the literal takes the keyword's value from the caller:

extern parseInt : string -> int -> int = int(base = ...)
extern openText : string -> string -> Seq string = builtins.open(mode = "rt", encoding = ...)
extern writeText : Path -> string -> string -> int = .write_text(encoding = ...)
parseInt "ff" 16int("ff", base=16)
openText p encbuiltins.open(p, mode="rt", encoding=enc)
writeText p t encp.write_text(t, encoding=enc)

The spelling is Python's own stub-file placeholder (def get(url, timeout=...)), lexed as one token so . . . stays three dots.

Binding rule. Positional, mirroring a Python call: the target takes the leading arguments positionally and the slots take the trailing ones in the order the keywords are written. Pinned literals consume no argument, so the two mix freely: m.f(a=1, b=..., c="x", d=...) at arity 3 emits m.f(s, a=1, b=i, c="x", d=b).

Rejections. A slot claims one argument of the declared arrow, so the type must have one to spare. A receiver takes the first argument and a nullary extern's only argument is the unit lowering drops, so both are diagnosed rather than silently mis-lowered.

Under-application. functools.partial cannot carry a keyword whose value has not arrived, so an under-applied slot extern becomes a lambda over the remaining arguments:

parseInt "ff"lambda _pf_k0: int("ff", base=_pf_k0)
parseIntlambda _pf_k0, _pf_k1: int(_pf_k0, base=_pf_k1)

Already-supplied arguments, and a method extern's receiver, bind to temporaries first, so they evaluate at application time exactly as functools.partial would have evaluated them rather than once per later call.

Scope

The type is untouched. A slot changes only where an argument lands in the emitted call, so inference, effects and arity are unchanged, and src/types/ needed no edit. src/python_emitter/ needed none either: PyExpr::CallKw already held arbitrary expressions. Strictly additive, so every existing extern behaves as before.

Also updated the tree-sitter grammar (extern_slot), since editors using it would otherwise fail to parse a slot extern. The TextMate grammars only highlight the extern keyword, so they need nothing.

Tests

7 new in tests/compile.rs (placement, mixed literal/slot ordering, receiver form, partial-to-lambda, eager evaluation of supplied args and receiver, the three rejections, one e2e), 4 new roundtrip cases, 1 new lexer test, 1 new tree-sitter corpus case.

856 Rust tests pass, 42/42 tree-sitter corpus, cargo fmt --check and cargo clippy --all-targets clean. Verified cross-module too: a slot extern binds to a lambda at module top level and a dependent module calls it n-ary.

A practical 17-section guide to Rust using examples from the Pyfun
compiler, for readers new to Rust who want to understand the codebase.
Covers ownership, impl blocks, pattern matching, Result types, traits,
generics, lifetimes, memory layout, closures, error handling, modules,
deriving traits, smart pointers, iterators, and syntax fundamentals.

Placed in docs/src/internals as a prerequisite to the numbered compiler
tour chapters, since the compiler is written in Rust and readers may
need a language primer alongside the architecture tour.
Pinned kwargs took only literals fixed at the declaration, so a call whose
keyword value comes from the caller needed a separate extern per call shape.
A `...` in place of the literal now takes the value from the caller, spelled
as in a Python stub file and lexed as one token.

The target takes the leading arguments positionally and the slots take the
trailing ones in written order; pinned literals consume no argument, so the
two mix freely on all three target forms. A slot claims one argument of the
declared arrow, so a receiver-only or nullary extern has none to spare and is
rejected with a diagnostic.

Under-application cannot use functools.partial, which has no way to carry a
keyword whose value has not arrived, so it becomes a lambda over the missing
arguments. Already-supplied arguments (and a method extern's receiver) bind to
temporaries first, keeping the evaluation timing functools.partial had.

The type is untouched: a slot changes only where an argument lands in the
emitted call, so inference, effects and arity are unchanged.
@simontreanor
simontreanor merged commit c1d5f1f into main Jul 28, 2026
11 checks passed
@simontreanor
simontreanor deleted the extern-kwarg-slots branch July 28, 2026 18:03
This was referenced Jul 28, 2026
simontreanor added a commit that referenced this pull request Jul 31, 2026
Two user-visible additions since 0.2.0 -- the `input` prelude builtin
(`string ->{io} string`, #19) and caller-supplied `extern` keyword slots
(`kw = ...`, #21) -- plus five fixes, four of them from the first program
written in Pyfun in anger:

* imported schemes reused the exporting module's type-variable ids, silently
  blocking generalization in the importing module and cascading to its
  consumers (#26 / #27)
* a recursive function of arity >= 2 could not perform any effect -- the shape
  of every loop in a language with no `while` (#24 / #28)
* a module that only matched an imported `Option` emitted `None_` without
  importing it (#25 / #29)
* user identifiers collided with Python keywords, builtins the emitter calls,
  and modules an `extern` imports (#23 / #30)
* the local-binder module-alias shadowing residual (#18)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant