scheduler: Spawn inherits WithObserve boundaries via GetBoundaries(k) (ADR-DOE-CORE-001)

[proboscis]

Fixes issue scheduler-spawn-drops-observer-boundary (/home/kento/repos/doeff-VAULT/issues/scheduler-spawn-drops-observer-boundary.md).

Problem

WithObserve placed inside scheduled() never saw effects performed by Spawn'd tasks, while WithHandler at the same position was inherited. Mechanism: the Spawn branch captured only prompt handlers (get_inner_handlers → GetHandlers(k)), and a spawned task's fiber chain roots at the scheduler's evaluation context, so spawn-site observer boundaries were silently dropped (call_all_observers walks the parent chain and never reaches them).

Design decision (issue 論点 1–3) — ADR-DOE-CORE-001

The issue asked for an explicit design judgment. Decision (recorded as executable ADR docs/adr/defadr_doeff_core_001_spawn_boundary_inheritance.hy):

1. Symmetric inheritance (論点 1): Spawn captures and reinstalls observer boundaries exactly like handler boundaries, preserving the interleaved nesting order. Order matters: for handler(WithObserve(obs, body)) nesting, effects performed by that handler's own body are dispatched above the observer and must stay unobserved — a flat "observers outermost" reinstall would over-observe (e.g. duplicate OTel spans). Covered by a dedicated test.
2. Opt-in primitive (論点 2): new non-consuming VM query GetBoundaries(k) (mirrors GetHandlers(k)), returning innermost-first ["handler"|"observer", callable] pairs with the catching handler as last entry, plus helper doeff.handler_utils.get_inner_boundaries(k) that drops the caller entry (RuntimeError if the chain doesn't terminate at a handler — fail-fast). GetHandlers and all its existing consumers (try_handler, doeff-traverse, agents) are untouched.
3. Ownership / cancel / multi-shot (論点 3): reinstalled boundaries are plain WithHandler/WithObserve wrappers owned by the child task's program, released by _release_task_refs on completion/failure/cancel. GetBoundaries borrows the continuation without consuming it (multi-shot safe). Scheduler machinery (wrap_task's TaskCompleted) performs outside the reinstalled boundaries, so subtree observers do not observe scheduler internals. Mask boundaries remain non-inherited (out of scope, ADR R5).

Changes

Layer	Change
packages/doeff-vm-core/src/continuation.rs	BoundaryKind enum + DetachedFiberChain::boundary_callables() / Continuation::boundary_callables() (interleaved walk, innermost-first)
packages/doeff-vm/src/{do_expr,python_generator_stream,pyvm}.rs	GetBoundaries(k) pyclass, eager classify → Pure([[kind, callable], ...]), registration
packages/doeff-vm/doeff_vm/__init__.py(.pyi)	export + stub
doeff/program.py, doeff/__init__.py	re-export; added to _DOEXPR_TYPES
doeff/handler_utils.py	get_inner_boundaries(k) helper (+ docstring note on get_inner_handlers)
packages/doeff-core-effects/doeff_core_effects/scheduler.py	Spawn captures inner_boundaries; pick_next reinstalls each kind via _reinstall_boundary preserving order; _release_task_refs updated
.semgrep.yaml	doeff-scheduler-spawn-must-capture-boundaries — bans get_inner_handlers inside scheduler.py (regression guard)
docs/adr/defadr_doeff_core_001_spawn_boundary_inheritance.hy	executable ADR (validates the installed semgrep rule with hit/clean fixtures)
tests/test_spawn_observer_inheritance.py	6 tests (committed failing first, TDD phase 1: commit 9339e6a — 5 failed at HEAD, case-B control passed)

Verification

The issue predates the ## Verification template section; mapping below covers every claim in its 症状/再現/論点 sections. No substitutions or weakenings.

Issue claim / criterion	Evidence
Case A scheduled(WithObserve(obs, mark_handler(main))) must observe child-effect	tests/test_spawn_observer_inheritance.py::test_observer_inside_scheduler_sees_spawned_child_effects + repro script output below
Case B WithObserve(obs, scheduled(...)) keeps working	::test_observer_outside_scheduler_still_sees_all
Handler/observer inheritance symmetric with nesting order preserved (論点 1)	::test_spawned_observer_does_not_see_handler_internal_effects (discriminates interleaved vs flat reinstall)
GetObservers-equivalent opt-in query primitive (論点 2)	::test_get_inner_boundaries_shape (GetBoundaries via get_inner_boundaries)
Multi-shot / cancel ownership (論点 3)	ADR-DOE-CORE-001 R4 (design doc); GetBoundaries is a non-consuming borrow like GetHandlers; boundaries released in _release_task_refs
Works across Gather and nested Spawn	::test_gather_multiple_children_all_observed, ::test_nested_spawn_inherits_observer_transitively
Handler inheritance unchanged	tests/test_spawn_with_handlers.py (9 tests), full suite
Semgrep guard fires on old pattern, clean on new	ADR executable enforcement docs/adr/defadr_doeff_core_001_spawn_boundary_inheritance.hy (hit fixture = old get_inner_handlers line in scheduler.py path; clean fixture = same call in handlers.py where it stays legitimate)

Issue repro script (verbatim from issue), after fix

A: scheduled(WithObserve(obs, mark_handler(main)))
  result=1 observed=['main-effect', 'child-effect']
B: WithObserve(obs, scheduled(mark_handler(main)))
  result=1 observed=['main-effect', 'child-effect']

(before fix @ cacc4b8, A observed only ['main-effect'] — confirmed by the TDD-phase-1 failing run)

Test / lint runs

• uv run pytest tests/test_spawn_observer_inheritance.py -q → 6 passed (5 of them failed at HEAD before the fix)
• uv run pytest docs/adr/defadr_doeff_core_001_spawn_boundary_inheritance.hy -q → 2 passed
• Focused neighbors (test_spawn_with_handlers, test_scheduler, test_with_observe_visibility, test_lazy_ask_inner_handler_propagation, test_get_handlers_defk + new file + ADR) → 68 passed
• Full suite uv run pytest -q → 959 passed, 87 skipped, 1 failed — the single failure is pre-existing at base (see below)
• uv run pyright doeff/ → 0 errors; semgrep --config .semgrep.yaml doeff/ packages/ --error → pass; doeff-linter → 0 errors on changed files; cargo test (doeff-vm-core) / cargo check (doeff-vm) → pass
• Rust VM rebuilt with maturin (uv sync --reinstall-package doeff-vm with cargo on PATH) before all test runs, per AGENTS.md stale-build warning

Pre-existing failures (not from this branch)

• tests/architecture/test_no_public_withhandler_shim.py::test_public_withhandler_shim_imports_are_gone fails at base cacc4b85: the fixture string (import doeff [WithHandler]) inside packages/doeff-adr/tests/test_defadr_macros.py (commit c6bf954) trips the shim-guard regex (commit b1fcca0). Neither file is touched by this branch.
• make lint-ruff has 20 pre-existing errors in files untouched by this branch (doeff-agents, doeff-conductor, memo tests, doeff/program.py:77 SIM108). The 1 ruff error introduced by this branch (import sort in the new test file) was fixed.

Follow-up

• doeff-traverse handlers use the same get_inner_handlers capture pattern for its parallel workers and still drop observers; migrating them to get_inner_boundaries is a natural follow-up (kept out of scope here to match the issue's Spawn scope).

Issue: scheduler-spawn-drops-observer-boundary

🤖 Generated with Claude Code