Skip to content

Commit fb60735

Browse files
committed
lowering: rename a root-level let that shadows an enclosing-scope name read earlier in the function
A root-level let in a nested function that reuses a name the function had already read from an enclosing scope (a parameter of the outer function, a module-level binding, a builtin) made that name local to the whole Python def, so the earlier read raised UnboundLocalError. The capture census now tells a reference to a binding outside the frame from one to the frame's own root (a parameter or an earlier root-level let), and a root-level let renames to _name exactly when some read of the name in the frame is outside. Rebinding a parameter or an earlier let in sequence stays as Python does it, and top-level lets are globals and never rename.
1 parent 6848e84 commit fb60735

6 files changed

Lines changed: 295 additions & 46 deletions

File tree

DESIGN.md

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -210,9 +210,14 @@ sibling arms of one match or arms of two matches in sequence, so the common `cas
210210
Error why: …` keeps its names however many matches in a function spell it. The one exception is a
211211
reference inside a nested closure, which always counts, since a closure can outlive its arm: a
212212
lambda made in one arm that reads `why` forces a later arm capturing `why` to rename. A `let` at
213-
the root of a function body is never renamed, since rebinding a name in sequence is exactly what
214-
Python does too; a nested `let x = x + 1` reads the outer `x` in its value and binds the fresh
215-
name, as `_x = x + 1`. A renamed nested function is defined under its fresh name and recurses under
213+
the root of a function body has its own rule, because Python makes a name local to the whole `def`
214+
the moment anything in the `def` assigns it: such a `let` is renamed only when some read of the
215+
name in that function means a binding *outside* it (an enclosing function's, a module-level one, a
216+
builtin) and sits before the `let` or in its own value or in a closure made before it, where Python
217+
would raise `UnboundLocalError` or read the wrong slot. Rebinding a parameter or an earlier `let` of
218+
the same function in sequence is exactly what Python does too, so that is emitted as itself, and a
219+
top-level `let` is a global and never renames. A nested `let x = x + 1` reads the outer `x` in its
220+
value and binds the fresh name, as `_x = x + 1`. A renamed nested function is defined under its fresh name and recurses under
216221
it, and a renamed `let mut` is assigned under it, including from a closure's `nonlocal`. Mechanics
217222
in `INTERNALS.md`.
218223

INTERNALS.md

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -234,7 +234,7 @@ the runtime alongside `Some`/`None_` unless it binds that name itself.
234234
### Arm-scoped captures and nested-block `let`s — implements DESIGN §5
235235

236236
A match arm's capture, and a `let` in a nested block, both become function-wide Python locals, so one
237-
that another binding of the same name is live across is renamed (issues #92, #96 and #97). The
237+
that another binding of the same name is live across is renamed (issues #92, #96, #97 and #99). The
238238
rule, in `src/lowering/captures.rs`, is liveness: binder `B` of name `n` (an arm capture or a
239239
nested-block `let`) is freshened when some reference to `n` outside `B` resolves, under Pyfun
240240
scoping, to a binding that is live across `B`: the frame root (a parameter, a root-level `let`, a
@@ -247,8 +247,10 @@ three arms under a root-level `why` read at the end all rename, and a lambda mad
247247
reads `why` forces a later arm capturing `why` to rename.
248248

249249
The census is one walk of the frame's body (`Census`) that resolves every `Var` reference and `<-`
250-
target to what it reads at that point, recorded as an `Occurrence { binder, in_closure }`: `binder`
251-
is `None` for the frame root, or the identity of a registered arm/nested-block binder. Identity is
250+
target to what it reads at that point, recorded as an `Occurrence { target, in_closure }`: the
251+
`target` is `Outside` (a binding outside the frame: an enclosing function's, a module-level one, a
252+
builtin), `Root` (a parameter or a root-level `let` of this frame), or `Binder` (a registered
253+
arm/nested-block binder); `must_rename` counts the first two alike. Identity is
252254
the address of the AST node (`binder_key`, the `MatchArm` or the `LetBinding`), which is what the
253255
lowering holds when it decides that binder, so no spans are involved and a desugared or cloned tree
254256
can never be mistaken for the original. Each registered binder also records its `ancestors`, the
@@ -298,7 +300,14 @@ are not a block, so any block inside a CE is nested; the module frame treats a b
298300
binding as root, since a collision with a module-level binding is already isolated by
299301
`lower_module`'s frame wrap). In a block that is not the root (`Frame::block_is_nested`, reported by
300302
`enter_block`), `lower_block_let` decides each bound name by `Frame::must_rename(n, key(let))`. A
301-
root-level `let` never renames: rebinding in sequence is what Python does too. Ordering: a value `let` lowers its value first (references there mean the outer binding), then
303+
root-level `let` asks `Frame::must_rename_root(n)` instead (issue #99): rename iff any occurrence of
304+
`n` in the frame is `Outside`. Python makes `n` local to the whole `def` as soon as anything in it
305+
assigns `n`, so a read of an outer `n` before the `let` (or in its value, or in a closure made
306+
before it) would raise `UnboundLocalError`; a read after the `let` resolves to `Root(let)` and a
307+
read of a parameter or an earlier root-level `let` resolves to `Root(that)`, neither of which is
308+
`Outside`, so rebinding in sequence stays as Python does it. Parameters are bound as `Root` when
309+
the frame is built (`Frame::of_body` takes them), and the module frame never renames a root-level
310+
`let` (`Frame::module`), since those are globals. Ordering: a value `let` lowers its value first (references there mean the outer binding), then
302311
installs `n → _n` in `renames` for the rest of the block, where it dies with the block's
303312
`restore_local_scope`; a parameterised `let` installs it before its body so a recursive call resolves
304313
to the renamed def, and the def is emitted under the fresh name. The emitted target is passed down

ROADMAP.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -231,7 +231,10 @@ this entry.
231231
nested-block `let`s"). Downstream review found the first rule too eager (#97, closed the same
232232
day): it renamed a name reused across sequential matches, where nothing is live across the
233233
rename, so the rule was tightened to liveness and `case Error why:` in three matches of one
234-
function keeps its plain name.
234+
function keeps its plain name. The same review turned up the last member of the family (#99,
235+
closed the same day): a root-level `let x` in a function that had already read an enclosing
236+
`x` made `x` local to the whole `def` and the earlier read raised `UnboundLocalError`; such a
237+
`let` now renames exactly when a read in the function means a binding outside it.
235238

236239
11. ~~**A destructuring folder went quadratic**~~ **CLOSED 2026-08-30** (#85 in #90). The in-place
237240
fold pass rejected `fun m (p, l) -> Map.add p l m` because the element parameter had no single

src/lowering/captures.rs

Lines changed: 87 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -32,19 +32,30 @@
3232
3333
use std::collections::{HashMap, HashSet};
3434

35-
use crate::parser::ast::{BlockStmt, CeItem, Expr, ExprKind, InterpPart, Item, LetBinding};
35+
use crate::parser::ast::{BlockStmt, CeItem, Expr, ExprKind, InterpPart, Item, LetBinding, Param};
3636

3737
/// The identity of an arm or nested-block binder: the address of its AST node.
3838
pub(super) fn binder_key<T>(node: &T) -> usize {
3939
node as *const T as usize
4040
}
4141

42-
/// One occurrence of a name: the binder it resolves to (`None` for the frame
43-
/// root or anything outside the frame) and whether it sits inside a nested
44-
/// closure of the frame.
42+
/// What an occurrence of a name reads, under Pyfun scoping.
43+
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
44+
enum Target {
45+
/// A binding outside the frame: an enclosing function's, a module-level
46+
/// binding, a builtin or prelude name.
47+
Outside,
48+
/// A root binding of the frame: a parameter or a root-level `let`.
49+
Root(usize),
50+
/// A registered arm or nested-block binder of the frame.
51+
Binder(usize),
52+
}
53+
54+
/// One occurrence of a name: what it resolves to and whether it sits inside a
55+
/// nested closure of the frame.
4556
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
4657
struct Occurrence {
47-
binder: Option<usize>,
58+
target: Target,
4859
in_closure: bool,
4960
}
5061

@@ -64,14 +75,21 @@ pub(super) struct Frame {
6475
/// function whose body is a block). Its `let`s are root-level and never
6576
/// renamed; any other block is nested.
6677
pub root_is_body: bool,
78+
/// Whether this is the module frame, whose root-level `let`s are globals and
79+
/// never rename.
80+
pub module: bool,
6781
}
6882

6983
impl Frame {
70-
/// The frame for a function whose body is `body` (parameters are root
71-
/// bindings of the frame).
72-
pub fn of_body(body: &Expr) -> Frame {
84+
/// The frame for a function whose body is `body`; `params` are its
85+
/// parameters, root bindings of the frame.
86+
pub fn of_body(params: &[Param], body: &Expr) -> Frame {
7387
let root_is_body = matches!(body.kind, ExprKind::Block { .. });
7488
let mut census = Census::new(root_is_body);
89+
for p in params {
90+
let names = super::pattern_bindings(&p.pattern);
91+
census.bind_root(binder_key(p), &names);
92+
}
7593
census.expr(body);
7694
census.finish(root_is_body)
7795
}
@@ -93,7 +111,9 @@ impl Frame {
93111
// as root here.
94112
let mut census = Census::new(true);
95113
census.items(items);
96-
census.finish(true)
114+
let mut frame = census.finish(true);
115+
frame.module = true;
116+
frame
97117
}
98118

99119
/// This frame's census with `body`'s occurrences added on top, `body` being
@@ -108,6 +128,7 @@ impl Frame {
108128
census.expr(body);
109129
let mut frame = census.finish(false);
110130
frame.fresh = self.fresh.clone();
131+
frame.module = self.module;
111132
frame
112133
}
113134

@@ -132,17 +153,37 @@ impl Frame {
132153
return false;
133154
};
134155
let ancestors = self.ancestors.get(&key);
135-
occurrences.iter().any(|o| match o.binder {
136-
Some(b) if b == key => false,
137-
None => true,
138-
Some(b) => o.in_closure || ancestors.is_none_or(|a| a.contains(&b)),
156+
occurrences.iter().any(|o| match o.target {
157+
Target::Binder(b) if b == key => false,
158+
Target::Outside | Target::Root(_) => true,
159+
Target::Binder(b) => o.in_closure || ancestors.is_none_or(|a| a.contains(&b)),
139160
})
140161
}
162+
163+
/// The root-level rule (issue #99): a root-level `let` of `name` must be
164+
/// renamed when any occurrence of `name` in the frame reads a binding
165+
/// *outside* it. Python makes a name local to the whole `def` as soon as
166+
/// anything in the `def` assigns it, so such a read (before the `let`, in
167+
/// its own value, or in a closure made before it) would fail or read the
168+
/// wrong slot. A reference after the `let` resolves to the `let` itself and
169+
/// never counts; one that reads a parameter or an earlier root-level `let`
170+
/// is a same-frame rebinding, which is what Python does too. The module
171+
/// frame's root-level `let`s are globals and never rename.
172+
pub fn must_rename_root(&self, name: &str) -> bool {
173+
if self.module {
174+
return false;
175+
}
176+
self.occurrences
177+
.get(name)
178+
.is_some_and(|os| os.iter().any(|o| o.target == Target::Outside))
179+
}
141180
}
142181

143182
/// What a name resolves to at a point of the walk.
144183
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
145184
enum Resolution {
185+
/// A root binding of this frame: a parameter or a root-level `let`.
186+
Root(usize),
146187
/// A registered arm/nested-block binder of this frame.
147188
Binder(usize),
148189
/// A binder belonging to a nested Python scope: its own slot, never an
@@ -183,24 +224,39 @@ impl Census {
183224
fresh: HashSet::new(),
184225
depth: 0,
185226
root_is_body,
227+
module: false,
186228
}
187229
}
188230

189231
fn occurrence(&mut self, name: &str) {
190-
let binder = match self.env.get(name) {
232+
let target = match self.env.get(name) {
191233
Some(Resolution::Inner) => return,
192-
Some(Resolution::Binder(b)) => Some(*b),
193-
None => None,
234+
Some(Resolution::Binder(b)) => Target::Binder(*b),
235+
Some(Resolution::Root(r)) => Target::Root(*r),
236+
None => Target::Outside,
194237
};
195238
self.out
196239
.entry(name.to_string())
197240
.or_default()
198241
.push(Occurrence {
199-
binder,
242+
target,
200243
in_closure: self.in_closure,
201244
});
202245
}
203246

247+
/// Bind `names` as root bindings of the frame from here on (a parameter, a
248+
/// root-level `let`): a reference after this point reads the frame's own slot.
249+
fn bind_root(&mut self, key: usize, names: &[String]) {
250+
for n in names {
251+
let r = if self.in_closure {
252+
Resolution::Inner
253+
} else {
254+
Resolution::Root(key)
255+
};
256+
self.env.insert(n.clone(), r);
257+
}
258+
}
259+
204260
/// Bind `names` for the extent of `f`: to a registered frame binder in the
205261
/// frame proper, to a nested scope's own slot inside a closure.
206262
fn scoped<F: FnOnce(&mut Census)>(&mut self, key: Option<usize>, names: &[String], f: F) {
@@ -220,8 +276,8 @@ impl Census {
220276
self.env = saved;
221277
}
222278

223-
/// Bind `names` from here on (no restore): a block `let` for the rest of its
224-
/// block. `key` is `None` for a root-level `let` (a root binding).
279+
/// Bind `names` from here on (no restore): a nested-block `let` for the rest
280+
/// of its block, or an arm capture for the arm's extent.
225281
fn bind(&mut self, key: Option<usize>, names: &[String]) {
226282
for n in names {
227283
match (self.in_closure, key) {
@@ -366,13 +422,20 @@ impl Census {
366422
/// and binds after; a parameterised `let` is a nested def whose own name is
367423
/// bound before its body, so a recursive reference resolves to it.
368424
fn block_let(&mut self, b: &LetBinding, nested: bool) {
369-
let key = nested.then(|| binder_key(b));
425+
let key = binder_key(b);
370426
let names = b.bound_names();
427+
let bind = |c: &mut Census| {
428+
if nested {
429+
c.bind(Some(key), &names);
430+
} else {
431+
c.bind_root(key, &names);
432+
}
433+
};
371434
if b.params.is_empty() {
372435
self.expr(&b.value);
373-
self.bind(key, &names);
436+
bind(self);
374437
} else {
375-
self.bind(key, &names);
438+
bind(self);
376439
self.closure_let(b);
377440
}
378441
}
@@ -396,7 +459,7 @@ impl Census {
396459
CeItem::LetBang { target, value, .. } | CeItem::Let { target, value, .. } => {
397460
self.expr(value);
398461
let names = target.bound_names();
399-
self.bind(None, &names);
462+
self.bind_root(binder_key(it), &names);
400463
}
401464
CeItem::DoBang(e)
402465
| CeItem::Return(e)
@@ -440,5 +503,6 @@ impl Census {
440503
} else {
441504
self.closure_let(b);
442505
}
506+
self.bind_root(binder_key(b), &b.bound_names());
443507
}
444508
}

src/lowering/mod.rs

Lines changed: 21 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1327,21 +1327,26 @@ impl Lowerer {
13271327
nested: bool,
13281328
) -> Result<(), LowerError> {
13291329
let mut targets = HashMap::new();
1330-
if nested {
1331-
for name in b.bound_names() {
1332-
if targets.contains_key(&name) {
1333-
continue;
1334-
}
1335-
let Some(frame) = self.frames.last() else {
1336-
break;
1337-
};
1338-
if !frame.must_rename(&name, captures::binder_key(b)) {
1339-
continue;
1340-
}
1341-
let fresh = self.fresh_capture_name(&name);
1342-
self.frames.last_mut().unwrap().fresh.insert(fresh.clone());
1343-
targets.insert(name, fresh);
1330+
for name in b.bound_names() {
1331+
if targets.contains_key(&name) {
1332+
continue;
13441333
}
1334+
let Some(frame) = self.frames.last() else {
1335+
break;
1336+
};
1337+
// A nested `let` renames under the liveness rule; a root-level one
1338+
// only when a read in the frame means a binding outside it (#99).
1339+
let rename = if nested {
1340+
frame.must_rename(&name, captures::binder_key(b))
1341+
} else {
1342+
frame.must_rename_root(&name)
1343+
};
1344+
if !rename {
1345+
continue;
1346+
}
1347+
let fresh = self.fresh_capture_name(&name);
1348+
self.frames.last_mut().unwrap().fresh.insert(fresh.clone());
1349+
targets.insert(name, fresh);
13451350
}
13461351
self.enter_block_let(b);
13471352
if !b.params.is_empty() {
@@ -1471,7 +1476,8 @@ impl Lowerer {
14711476
let shadowed = self.shadow_local_fns(params);
14721477
self.fn_local_stack.push(bound);
14731478
// The body is its own Python frame for the capture-rename census.
1474-
self.frames.push(captures::Frame::of_body(body));
1479+
self.frames
1480+
.push(captures::Frame::of_body(prelude_params, body));
14751481
let lowered = self.lower_return(body, inner);
14761482
self.frames.pop();
14771483
self.fn_local_stack.pop();

0 commit comments

Comments
 (0)