Bind the handler-dispatch handle to its dispatch window, add GC traversal, fail fast on observer errors, and reclaim arena slots (#492, #497, #500, #506)

[proboscis]

Closes #492, #497, #500, #506.

Part of the 2026-07-10 execution-substrate audit (#507 is the index). Independent of the other audit PRs (Rust VM crates + their tests; verified by integration merge). Rust changed → reviewers running locally need make sync.

What changed

• vm-core: stale pending_handler_k_handle delivers an unrelated exception into an abandoned continuation and substitutes its return value #492 — stale pending_handler_k_handle eliminated structurally (a565673f). The VM-global slot is removed entirely; the handler-dispatch PyK handle is a local in eval_perform/eval_perform_with_k, threaded through the new eval_expand into an EvalReturnContinuation::ExpandReturn frame variant and consumed on both exits — value path (Program frame) and raise path (step_raise pops ExpandReturn and discontinues the perform-site chain via recover_from_k_handle). A wrong-arity @do handler's late TypeError can no longer deliver an unrelated exception into an abandoned continuation or substitute another dispatch's result. Uses the VM's own frame system (per the repo Change Protocol — no bolted-on external mechanism); the invariant checker scans ExpandReturn frames in live and detached fibers.
• vm: Py-holding pyclasses lack __traverse__/__clear__ — reference cycles through doeff_vm objects are permanently uncollectable #500 — PyO3 GC protocol (3eebe836). __traverse__ on all Py-holding pyclasses (15 DoExpr classes, PythonCallable, PyIRStream, Ok/Err), __clear__ on the non-frozen ones; EffectBase's pyo3 dict slot removed so subclass __dict__ is CPython-managed (traversable). Cycles through VM objects are now collectable. PyK is deliberately excluded with a documented rationale (sole-owner chain + mid-dispatch recovery would break under tp_clear).
• vm-core: WithObserve observer exceptions are silently swallowed #506 — observer exceptions fail fast (eda61eb2). call_all_observers returns a Result; the first failing observer aborts the walk and the exception is raised at the perform site via continue_raise (catchable like a handler error); the effect is not delivered. The DoCtrl::WithObserve contract doc now states this.
• vm-core: ~0.5KB native memory leaked per abandoned continuation; arena slots stranded within a run (segment-leak family) #497 — arena slot stranding fixed; the "native leak" was this (c4fefe61). SlotReclaimQueue armed on detach_chain; DetachedFiberChain::Drop reports bare slot indices (fiber ownership rules preserved — pinned by a new arch test); alloc() drains the queue; clear() swaps it so late drops can't poison the next session. Measured: 20k-abort loops went from +9.5–10MB/run unbounded to a plateau at +0KB/run (head fiber index 2 vs 400 pre-fix).

Adversarial review (no blocking findings)

• All four commits survived refutation. Test-theater check: the branch's new regression tests were run against the merge-base build directly — 8 red pre-fix, green post-fix.
• vm-core: stale pending_handler_k_handle delivers an unrelated exception into an abandoned continuation and substitutes its return value #492 hardened with 7 adversarial variants (Pass-path wrong arity, consecutive bad dispatches, non-contamination of subsequent dispatches, pre-consumption handler raise, raise-after-Resume, victim catching the TypeError with a correct result, sibling-task error propagation under Spawn/Gather) — all pass.
• Two informational notes: (a) DetachedFiberChain::append keeps self's reclaim queue and drops other's — only reachable via already-unsupported cross-VM continuation sharing; a debug assertion (Arc::ptr_eq) is suggested as cheap hardening. (b) A pre-existing residual class (measured identical at the merge base): Race-loser cancellation leaves per-Race segments/continuations/IR-streams uncollectable through PyK interiors — the documented PyK exclusion; recommend a separate issue for long-lived processes with heavy Race/timeout use.

Semantic changes

• Observer exceptions now abort the dispatch and propagate (previously silently swallowed) — intended fail-fast, but a visible behavior change for any consumer with a raising observer.
• Direct EffectBase() instances (not subclasses) no longer accept attribute assignment / pickle (base dict slot removed); zero in-tree or downstream (proboscis-ema) usage.
• Residual GC gap documented: cycles routed exclusively through a DoExpr instance __dict__ or through PyK interiors remain uncollectable (pyo3 limitation / deliberate exclusion).

Verification

• cargo test --features python_bridge 36 passed / 0 failed (+ invariant-checks build: 42+2 passed). Note for CI: a bare cargo test -p doeff-vm-core runs only 2 tests — vm_tests is feature-gated; any CI gate must pass --features python_bridge.
• uv run pytest packages/doeff-vm/tests packages/doeff-vm-core/tests tests/ -q: 966 passed / 86 skipped / 0 failed (after make sync).
• Issue repros: vm-core: stale pending_handler_k_handle delivers an unrelated exception into an abandoned continuation and substitutes its return value #492 wrong-scope capture repro prints its OK line; vm: Py-holding pyclasses lack __traverse__/__clear__ — reference cycles through doeff_vm objects are permanently uncollectable #500 cycle shapes all collected; vm-core: WithObserve observer exceptions are silently swallowed #506 observer RuntimeError propagates out of run(); vm-core: ~0.5KB native memory leaked per abandoned continuation; arena slots stranded within a run (segment-leak family) #497 leak harness plateaus at 0KB/run.
• Perf guard: resume p50 ≈ 4.8–5.3µs; 100k-dispatch Get loop 1.82–1.88µs/dispatch (vs ~2.1µs Try is ~60-100x the cost of an ordinary effect dispatch, scaling with context depth (~0.1-3ms per Try) #488 baseline) — GC and observer changes did not regress the hot path.
• Integration merge with the other audit PRs + Wake resolved external waits eagerly instead of stalling behind unresolved peers (#490) #491: 950 root + 125 package tests passed, 0 failed.

🤖 Generated with Claude Code