Process never exits after hangup: asyncio.run() deadlocks joining blocked run_in_executor threads

[pikedave]

backtalk prints [backtalk] hung up and then never exits. The process stays alive indefinitely and has to be killed. This affects every exit path: the spoken quit phrase, a typed quit phrase, and Ctrl-C.

Observed on: 84b3a6c, Python 3.12.14, macOS 26.6.2 arm64. Also present before that commit; this is long-standing rather than a regression.

main.py:53 currently states "Ctrl-C works." It does not.

Root cause

log("[backtalk] hung up") at main.py:1062 is the last statement in amain()'s finally, so the coroutine itself completes normally. The hang is in what asyncio.run() does next: loop.run_until_complete(loop.shutdown_default_executor()), which joins the default ThreadPoolExecutor with wait=True.

Two jobs parked in that executor can never return:

967:  typed_fut = loop.run_in_executor(None, typed_q.get)     # blocking Queue.get(), no timeout
969:  press_fut = loop.run_in_executor(None, ptt.wait_press)  # blocks in threading.Event.wait()

Cancelling typed_fut and press_fut cancels the asyncio futures, but it does not touch the underlying OS threads, which keep blocking. And ThreadPoolExecutor workers are non-daemon, so even bypassing shutdown_default_executor() would only move the hang to concurrent.futures' own atexit joiner.

ptt.wait_press() at ptt.py:117 does now wake periodically on RELEASE_GRACE after b8a697a, but the surrounding while True has no exit condition other than an actual keypress, so the thread still never returns.

Evidence

sample on the wedged process, taken after [backtalk] hung up was printed, puts the main thread here:

Thread_4982233  DispatchQueue_1: com.apple.main-thread
  Py_RunMain -> pymain_run_module -> builtin_exec -> PyEval_EvalCode
    -> _PyEval_EvalFrameDefault
      -> select_kqueue_control_impl
        -> kevent

That is the asyncio selector still pumping the loop, with the executor's joiner thread blocked on a lock. Worth noting for anyone reading a native stack here: CPython 3.11+ inlines Python-to-Python calls, so many Python frames collapse into a single _PyEval_EvalFrameDefault. The shallow-looking stack is a display artefact, not a shallow call chain. We misread it that way ourselves on the first pass and lost time chasing MLX threads that turned out to be irrelevant.

Minimal reproduction

Twenty lines, no backtalk imports, same shape as amain():

import asyncio, queue, threading

async def amain():
    loop = asyncio.get_running_loop()
    q = queue.Queue()
    evt = threading.Event()
    typed_fut = loop.run_in_executor(None, q.get)      # main.py:967
    press_fut = loop.run_in_executor(None, evt.wait)   # main.py:969
    try:
        await asyncio.sleep(0.2)
        return                                          # the quit phrase fires
    finally:
        typed_fut.cancel()                              # cancels the FUTURE, not the thread
        press_fut.cancel()
        print("[repro] hung up", flush=True)            # main.py:1062

asyncio.run(amain())
print("[repro] process exited cleanly", flush=True)

Prints [repro] hung up and then hangs forever. The second print is never reached.

What we are running, and why it is probably not what you want

We hard-exit at the end of that same finally, before asyncio.run() gets the chance to join anything:

exc = sys.exc_info()[0]
if exc is not None:
    traceback.print_exc()
sys.stdout.flush()
sys.stderr.flush()
os._exit(1 if exc is not None else 0)

Everything that needs cleaning up (mouth.shutdown(), the music restore, signals.static_stop(), await brain.stop()) already runs explicitly above it, so there is nothing left for the interpreter to do. The sys.exc_info() guard is there because that finally runs on crash paths too, and a bare os._exit(0) would swallow tracebacks and report success on a crash. It also fixes Ctrl-C for free, since except KeyboardInterrupt: pass sits directly above.

We know this is the blunt instrument, not the right answer, and we would expect you to prefer the proper fix. We are reporting it because the bug is real and reproducible, not to push our patch. The cleaner route is presumably to make both blockers interruptible, something like a shutdown Event with typed_q.get(timeout=...) and _press_evt.wait(timeout=...) in poll loops so the threads actually finish, though that does mean any future run_in_executor job that forgets a timeout reintroduces the hang. Your call entirely; we are running the blunt version only because it is verified working here.

Confirmed fixed on our side across all three exit paths: spoken quit, typed quit, and Ctrl-C.

Related: this was masked for a long time by a separate quit-phrase matching bug, filed as #23. With that one unfixed, the quit was never requested in the first place, so this deadlock was unreachable by the normal exit route and only showed up on Ctrl-C.