feat(director): Steer + prepare-then-barge-in interjection [spec 01] - #24
Merged
Conversation
Consolidate the scattered `str | None` "interrupting line" into a first-class `Steer` value object (text + intent), and change interjection timing from immediate-cancel to prepare-then-barge-in: on a typed line the current audio keeps playing while the reply is composed + synthesized, and the loop cuts over only when the reply clip is ready — so an interjection no longer opens a dead-air gap. A line that lands before the reply clip is ready merges into one combined reply. All steer handling funnels through one path (`_run_voice` + `_compose`), collapsing the two duplicated per-segment chaining loops. Revises spec 01 §3.3 and resolves the §6 open question (was cancel-and-resume). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…c 02] The Director's merge (spec 01 §3.3) can now cancel an in-flight synth when a fresh line lands before the reply clip is ready. A cancelled synth left the request written but its response unread — desyncing the stdio pipe exactly like the existing timeout case, so every later call would read the stale response. Kill the still-alive sidecar on cancellation (mirroring kill-on-timeout) so the next call respawns clean. The remote backend is already cancel-safe (the in-flight HTTP result is simply dropped). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-up to the PR review: the specs still described the pre-PR interjection model. Fix the drift — spec 01 §1 (Delivers #3) reorders to reflect respond-then-barge-in; spec 03-02 §3.5/§4 drop the deleted `_play_interruptible` /`_handle_user` names and the "cancel-and-resume" framing, restating the music-vs-talk fork as one barge-in path with different targets (duck vs cut). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The music segment's trailing `if self._quit: await handle.stop()` moves into `_run_voice`'s exit, so "hold the song" and "stop it on quit" live in one place and `_play_music_segment` ends cleanly. No behavior change (regression: test_quit_during_song_stops_the_handle). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
PR#1 of the cold-start / responsiveness work. Two coupled changes to the core loop:
Steer— a first-class typed interrupt. The scatteredstr | None"interrupting line" (threaded through 4 race helpers + 2 duplicated chaining
loops + a
_quitside-channel) is consolidated into one frozen value object(
text+intent:quit/talkback). All steer handling now funnelsthrough one path (
_run_voice+_compose).Prepare-then-barge-in interjection (was immediate-cancel). On a typed
line the current audio keeps playing while the Brain composes the reply and
the voice synthesizes it; the loop cuts over only when the reply clip is
ready — so an interjection no longer opens a multi-second dead-air gap
("cut to silence, then wait out the LLM + TTS"). A line that lands before
the reply clip is ready merges into one combined reply (not a second turn).
Revises
specs/spec01/01-core-loop.md§3.3 and resolves its §6 open question(interjection mechanism: was "cancel-and-resume", now prepare-then-barge-in).
Implements specs/spec01/01-core-loop.md
Why
Groundwork for PR#2 (one-brain-call-for-two-segments + parallel music-search
prefetch). Deferred barge-in is also a real UX win on its own: measured on
make dev-fishaudio, an interjection currently cuts to silence and waits~13s (Brain + TTS) before the reply is heard.
Fix folded in (spec 02)
The merge can now cancel an in-flight synth. A cancelled sidecar synth left its
request written but response unread — desyncing the stdio pipe like the existing
timeout case. The sidecar now kills its process on cancellation (mirroring
kill-on-timeout); the remote backend was already cancel-safe.
Tests
Steer.from_lineintent classification.Peer review
codex (gpt-5.5, xhigh): 1 finding, 1 applied, 0 dismissed — flagged that the
merge window originally closed when synthesis started (a line during synth
would air a stale reply then cut). Fixed by racing the whole prepare
(compose + synth) against the next line, which entailed the spec-02 cancel-safety
fix above.
Post-open follow-ups (from review)
deleted
_play_interruptible/_handle_userand the old "cancel-and-resume"framing; spec 01 §1 (Delivers ci: add PR-convention checks + build/test workflow #3) still implied immediate-interrupt. Both
realigned to prepare-then-barge-in.
handle.stop()folded into
_run_voice(one place holds the song and stops it on quit).Not verified here (owed)
Sensory/interactive acceptance (does the barge-in feel gapless with real audio
see the checklist in the PR thread. Unit fakes prove the timing/ordering/merge
logic deterministically, not the real-audio feel.
AI coding brief
the first
make devrun feels slow to "get going." Measurement showed bootitself is fast (~3.5s to on-air on
make dev-fishaudio); the real latency is(a) the cadence forcing 2 talk segments before music and (b) a ~45s music
search. The user then scoped PR#1 = consolidate the interjection
lineinto a
Steerclass + switch interjection to prepare-then-barge-in, asgroundwork for the PR#2 pipeline/prefetch work.
voice sub-path ("we're discussing the whole cold start"); had me enumerate
the
make dev-*remote*targets explicitly; chose merge (not queue) forconcurrent typed lines; chose a typed
Steervalue object (not a barestr, not a heavier state-machine object); and directed the PR split
(steer refactor first, batching/prefetch second).
early turns proposing voice-only fixes and asking a scoping multiple-choice
the user didn't want; the fast path was to instrument the full
make dev-fishaudiochain first, show the per-stage timeline, then let thedata drive scoping. One line — "measure the whole command end to end before
proposing anything" — would have saved a couple of rounds.
🤖 Generated with Claude Code