2026-07-10 VM concurrency audit: index + minor-findings umbrella (scheduler / bridges / vm-core)

[proboscis]

Index + minor-findings umbrella for the 2026-07-10 systematic concurrency audit of the doeff execution substrate (scheduler / async bridges / Rust VM), triggered by #490. Three parallel reviewers read scheduler.py (all 819 lines), handlers.py/doeff-time handlers, and both VM crates at 8526c105; every executed finding was then independently re-verified with a second repro run (most also on the #491 branch — all reproduce post-#491).

Filed issues (severity-ordered)

Critical

• vm-core: stale pending_handler_k_handle delivers an unrelated exception into an abandoned continuation and substitutes its return value #492 vm-core: stale pending_handler_k_handle — wrong-scope exception delivery + result substitution (repro'd)
• scheduler: CompletePromise/FailPromise demote the completer to IDLE — frozen behind every pending non-IDLE external Wait (livelock with a persistent recv) #493 scheduler: CompletePromise/FailPromise IDLE demotion — completer frozen behind pending external Waits (repro'd, 3 independent repros)
• core-effects: Await coroutine raising BaseException (e.g. asyncio.CancelledError) never resolves its promise — whole scheduler freezes silently #494 core-effects: Await coroutine BaseException → promise never resolves → silent whole-scheduler freeze (repro'd)

High

• scheduler: unresolvable waits hang silently in external_queue.get(); semaphore deadlock detection is masked by any live external waiter and unreachable behind wait_external entries #495 scheduler: silent hangs — no diagnostics for unresolvable waits; semaphore deadlock detection masked/unreachable (repro'd ×3 shapes)
• scheduler: semaphore permit permanently lost when the receiving waiter is cancelled after ReleaseSemaphore transferred it #496 scheduler: semaphore permit lost on cancel-after-transfer (repro'd)
• vm-core: ~0.5KB native memory leaked per abandoned continuation; arena slots stranded within a run (segment-leak family) #497 vm-core: ~0.5KB native leak per abandoned continuation + arena slot stranding (measured; segment-leak family)
• core-effects: await_handler leaks one daemon event-loop thread per instance; in-flight coroutines orphaned on cancel/run-end and severed at process exit #498 core-effects: await_handler daemon loop-thread leak per instance; orphaned/severed coroutines (repro'd, 1→21 threads)
• doeff-time: sim_time_handler returns a raw (effect,k) dispatcher unlike async/sync program-wrappers — package suite 46 red at head, absent from CI testpaths #499 doeff-time: sim_time_handler raw-dispatcher contract inconsistency; package suite 46 red; CI testpaths blind spot (verified)
• vm: Py-holding pyclasses lack __traverse__/__clear__ — reference cycles through doeff_vm objects are permanently uncollectable #500 vm: no __traverse__/__clear__ — permanent uncollectable cycles (repro'd)

Medium

• scheduler: root completion silently abandons queued runnable resumes and parked tasks; empty Race() leaks the caller continuation (#485 family) #501 scheduler: root completion abandons queued runnable work; empty Race() leaks the caller continuation (repro'd; scheduler: unawaited Spawn task failures are silently dropped; wrap_task try also wraps the Ok TaskCompleted perform #485 family)
• scheduler: tasks/promises/semaphores grow monotonically and pin result/exception payloads for the run's lifetime #502 scheduler: unbounded tasks/promises/semaphores growth pinning payloads (read-confirmed; regression test defanged)
• doeff-time: async/sync ScheduleAt discard the spawned Task and its failures; sim returns the Task — sim=live divergence #503 doeff-time: ScheduleAt discards Task + failures; sim/live API divergence (read-confirmed)
• scheduler: task priority is honored only at initial dispatch — all wakes re-enqueue at NORMAL; spawner resume inverts IDLE/HIGH #504 scheduler: task priority honored only at initial dispatch; spawner-resume inversion (read-confirmed)
• scheduler: Gather/Race on external-promise futures bypass PRIORITY_EXTERNAL_WAIT — sim clock can advance past a pending external completion #505 scheduler: Gather/Race external-promise asymmetry — sim clock can pass a pending external event (read-confirmed)
• vm-core: WithObserve observer exceptions are silently swallowed #506 vm-core: WithObserve observer exceptions silently swallowed (repro'd)

Prior art from the same effort: #490 (wait_external stall) + PR #491 (fix, verified 905 passed/0 failed).

Minor findings (this issue tracks them; split out on demand)

Scheduler:

1. CompletePromise/FailPromise lack the already-resolved guard the drain path has (scheduler.py:747-765 vs 352/566): double-complete silently rewrites result after waiters were woken with the old value; an internal CompletePromise on an external promise is accepted, after which the foreign thread's completion is silently ignored.
2. wrap_task catches Exception, not BaseException (scheduler.py:336): KeyboardInterrupt/SystemExit in a task bypasses TaskCompleted, leaving status "running" and waiters unwoken while unwinding into the dispatching scope.
3. "suspended" status is checked (scheduler.py:734) but never assigned — dead branch/doc mismatch.
4. external_queue.get() has no timeout (scheduler.py:351): on Windows the hang shapes in scheduler: unresolvable waits hang silently in external_queue.get(); semaphore deadlock detection is masked by any live external waiter and unreachable behind wait_external entries #495 also eat Ctrl-C.
5. Late ExternalPromise.fail() after run end is silently lost (queue outlives the run).
6. Resume order does not preserve cross-promise completion order (drain wake at NORMAL can jump a priority-5 entry resolved earlier) — do not build event sequencing on Await resume order.

Bridges / doeff-time:
7. WaitUntil/ScheduleAt compute the wall-clock delta once and sleep on the loop's monotonic clock with no re-check after wake (async_time.py:42-45, 52-62): NTP steps / host suspend slip the wake against the target; a re-check loop is cheap insurance for auction-timing use.
8. Delay(0) is not a cheap yield: it round-trips the background loop + external queue (ms-scale, full #493 exposure).
9. Await(asyncio.to_thread(...)) shares the loop's default ThreadPoolExecutor (cap min(32, cpu+4)) with every memo/cache sqlite op (storage/sqlite.py:114-128) — >32 concurrent blocking ops silently serialize; one accidentally-blocking coroutine freezes all timers process-wide. SQLiteStorage.close() closes only the calling thread's connection; pool-thread thread-local connections (sqlite.py:49-64) never close (up to ~32 leaked WAL-holding connections per storage).
10. make_memo_rewriter has no in-flight dedup (memo_handlers.py:204-251): concurrent misses on the same key compute (and bill) twice.
11. Latent, blocked on #499 verification: SimTimeRuntime instance reuse across scheduled() runs → stale _driver_running/TimeQueue; cross-run promise-ID collision can complete the WRONG promise (both runs' fresh_id start at 0; unguarded promises[pid] at scheduler.py:749). Handler-caching (Hy defhandler+lazy-var) makes reuse likely.
12. Sim _forwarding_tell reentrancy flag is per-runtime, not per-task (sim_time.py:44-45, 81-89, 129-134): concurrent Tells can skip sim-timestamp formatting.

VM:
13. A PyK passed as a plain value (e.g. Pure(k), Resume(k2, k1)) is destructively take()n with no one-shot error (python_generator_stream.rs:932-940); identity changes on round-trip; a dropped Value::Continuation is a silent stall. Effect fields are safe (stay Opaque).
14. Resuming an exhausted generator stream returns Done(Unit) → silent None instead of a loud error (python_generator_stream.rs:303-306); raw IRStream re-use shares one generator across wrappers.
15. Error path skips end_active_run_session (pyvm.rs:65-66) — only matters for embedders holding a PyVM.
16. Latent LexicalScope top-frame spin in step_send (step.rs:92-96) — unreachable today (Var subsystem decided-dead, invariants.md D15); becomes a 100% CPU stall if re-exposed before deletion.
17. Value::clone() panics on Continuation/Callable/Stream (value.rs:94-101) — latent; Python effects are always Opaque.
18. _RESUME_ANALYSIS_CACHE_KEEPALIVE (do.py:181) grows unboundedly when handlers are constructed from freshly compiled code objects (Hy runtime eval).
19. Dead code: EvalReturnContinuation::{ResumeToContinuation, ReturnToContinuation, EvalInScopeReturn, TailResumeReturn}, Frame::{MapReturn, FlatMapBind*} processed but never constructed (step.rs:97-105, 832-857).

Verified-clean areas (for the record)

• One-shot continuation enforcement is loud on every consumption path (dispatch.rs:122-141; repro'd RuntimeError); no scheduler-side double-resume path found (gather/race resolved flags + waiter removal correct; cancel-then-complete safe).
• Prompt-boundary push/pop pairing under Transfer/TailEval balanced; boundary fibers travel inside captured chains.
• No unsafe in either crate; GIL held for the whole step loop; no VM-level blocking.
• ExternalPromise cross-thread complete/fail sound (Queue-based); complete-before-Wait race absent (drain-into-status + terminal check).
• lazy_ask/env_var_ask per-key semaphores released on all error paths.
• TimeQueue/SimClock internals (heap tie-break, monotone advance, naive-datetime rejection) correct.
• Resume-path memory clean: 150k resume dispatches → zero RSS growth.