Skip to content

feat(director): Steer + prepare-then-barge-in interjection [spec 01] - #24

Merged
wine-fall merged 4 commits into
mainfrom
zachg-0713--cold-start-speedup
Jul 14, 2026
Merged

feat(director): Steer + prepare-then-barge-in interjection [spec 01]#24
wine-fall merged 4 commits into
mainfrom
zachg-0713--cold-start-speedup

Conversation

@wine-fall

@wine-fall wine-fall commented Jul 14, 2026

Copy link
Copy Markdown
Owner

What

PR#1 of the cold-start / responsiveness work. Two coupled changes to the core loop:

  1. Steer — a first-class typed interrupt. The scattered str | None
    "interrupting line" (threaded through 4 race helpers + 2 duplicated chaining
    loops + a _quit side-channel) is consolidated into one frozen value object
    (text + intent: quit / talkback). All steer handling now funnels
    through one path (_run_voice + _compose).

  2. Prepare-then-barge-in interjection (was immediate-cancel). On a typed
    line the current audio keeps playing while the Brain composes the reply and
    the voice synthesizes it; the loop cuts over only when the reply clip is
    ready
    — so an interjection no longer opens a multi-second dead-air gap
    ("cut to silence, then wait out the LLM + TTS"). A line that lands before
    the reply clip is ready merges
    into one combined reply (not a second turn).

Revises specs/spec01/01-core-loop.md §3.3 and resolves its §6 open question
(interjection mechanism: was "cancel-and-resume", now prepare-then-barge-in).

Implements specs/spec01/01-core-loop.md

Why

Groundwork for PR#2 (one-brain-call-for-two-segments + parallel music-search
prefetch). Deferred barge-in is also a real UX win on its own: measured on
make dev-fishaudio, an interjection currently cuts to silence and waits
~13s (Brain + TTS) before the reply is heard.

Fix folded in (spec 02)

The merge can now cancel an in-flight synth. A cancelled sidecar synth left its
request written but response unread — desyncing the stdio pipe like the existing
timeout case. The sidecar now kills its process on cancellation (mirroring
kill-on-timeout); the remote backend was already cancel-safe.

Tests

  • Steer.from_line intent classification.
  • Deferred barge-in ordering (reply synthesized before the current clip is cut).
  • Merge during compose and during synthesis (no stale reply is aired).
  • Sidecar kill-on-cancel (pipe-desync regression).
  • All existing director/music interjection tests pass unchanged in behavior.
  • Full suite: 225 passed, 12 deselected (integration, on-demand).

Peer review

codex (gpt-5.5, xhigh): 1 finding, 1 applied, 0 dismissed — flagged that the
merge window originally closed when synthesis started (a line during synth
would air a stale reply then cut). Fixed by racing the whole prepare
(compose + synth) against the next line, which entailed the spec-02 cancel-safety
fix above.

Post-open follow-ups (from review)

  • Spec drift fixed: review caught that spec 03-02 §3.5/§4 still named the
    deleted _play_interruptible/_handle_user and the old "cancel-and-resume"
    framing; spec 01 §1 (Delivers ci: add PR-convention checks + build/test workflow #3) still implied immediate-interrupt. Both
    realigned to prepare-then-barge-in.
  • Cohesion refactor: the music segment's trailing on-quit handle.stop()
    folded into _run_voice (one place holds the song and stops it on quit).

Not verified here (owed)

Sensory/interactive acceptance (does the barge-in feel gapless with real audio

  • real Brain; does merge feel right when typing fast) is a listen-and-type pass —
    see the checklist in the PR thread. Unit fakes prove the timing/ordering/merge
    logic deterministically, not the real-audio feel.

AI coding brief

  • Original request: Investigate + speed up murmur's first cold start. Why:
    the first make dev run feels slow to "get going." Measurement showed boot
    itself is fast (~3.5s to on-air on make dev-fishaudio); the real latency is
    (a) the cadence forcing 2 talk segments before music and (b) a ~45s music
    search. The user then scoped PR#1 = consolidate the interjection line
    into a Steer class + switch interjection to prepare-then-barge-in, as
    groundwork for the PR#2 pipeline/prefetch work.
  • Manual interventions: User corrected an early over-narrow focus on the
    voice sub-path ("we're discussing the whole cold start"); had me enumerate
    the make dev-*remote* targets explicitly; chose merge (not queue) for
    concurrent typed lines; chose a typed Steer value object (not a bare
    str, not a heavier state-machine object); and directed the PR split
    (steer refactor first, batching/prefetch second).
  • Retro: Lead with the end-to-end measurement, not a sub-component. I spent
    early turns proposing voice-only fixes and asking a scoping multiple-choice
    the user didn't want; the fast path was to instrument the full
    make dev-fishaudio chain first, show the per-stage timeline, then let the
    data drive scoping. One line — "measure the whole command end to end before
    proposing anything" — would have saved a couple of rounds.

🤖 Generated with Claude Code

wine-fall and others added 4 commits July 14, 2026 13:58
Consolidate the scattered `str | None` "interrupting line" into a first-class
`Steer` value object (text + intent), and change interjection timing from
immediate-cancel to prepare-then-barge-in: on a typed line the current audio
keeps playing while the reply is composed + synthesized, and the loop cuts over
only when the reply clip is ready — so an interjection no longer opens a
dead-air gap. A line that lands before the reply clip is ready merges into one
combined reply. All steer handling funnels through one path (`_run_voice` +
`_compose`), collapsing the two duplicated per-segment chaining loops.

Revises spec 01 §3.3 and resolves the §6 open question (was cancel-and-resume).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…c 02]

The Director's merge (spec 01 §3.3) can now cancel an in-flight synth when a
fresh line lands before the reply clip is ready. A cancelled synth left the
request written but its response unread — desyncing the stdio pipe exactly like
the existing timeout case, so every later call would read the stale response.
Kill the still-alive sidecar on cancellation (mirroring kill-on-timeout) so the
next call respawns clean. The remote backend is already cancel-safe (the
in-flight HTTP result is simply dropped).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-up to the PR review: the specs still described the pre-PR interjection
model. Fix the drift — spec 01 §1 (Delivers #3) reorders to reflect
respond-then-barge-in; spec 03-02 §3.5/§4 drop the deleted `_play_interruptible`
/`_handle_user` names and the "cancel-and-resume" framing, restating the
music-vs-talk fork as one barge-in path with different targets (duck vs cut).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The music segment's trailing `if self._quit: await handle.stop()` moves into
`_run_voice`'s exit, so "hold the song" and "stop it on quit" live in one place
and `_play_music_segment` ends cleanly. No behavior change (regression:
test_quit_during_song_stops_the_handle).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@wine-fall
wine-fall merged commit c41269b into main Jul 14, 2026
5 checks passed
@wine-fall
wine-fall deleted the zachg-0713--cold-start-speedup branch July 14, 2026 07:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant