Skip to content

fix: gate recording on mic liveness so AirPods stop losing the first words - #134

Closed
claude[bot] wants to merge 2 commits into
mainfrom
fix/airpods-mic-liveness-gate
Closed

fix: gate recording on mic liveness so AirPods stop losing the first words#134
claude[bot] wants to merge 2 commits into
mainfrom
fix/airpods-mic-liveness-gate

Conversation

@claude

@claude claude Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Requested by Dylan Fox · Slack thread

What & why

Before: on AirPods (any Bluetooth mic), the pill and the start chime appear the instant the hotkey is pressed — but the device takes ~1–2 s to switch its audio profile (A2DP→HFP) before the mic actually delivers anything, and macOS drops back to A2DP after every idle gap. So on the first dictation and every dictation after a pause, the user is cued to speak while nothing is being captured, and the first words of the utterance are missing from the transcript.

After: the pill still appears immediately on key-down, but in a distinct "Connecting…" warming-up state (no REC tag, no waveform). It flips to the normal recording state — and the start chime plays — only once the mic is actually delivering audio. On wired/built-in mics the warming state is a blink (audio flows near-instantly); on AirPods the chime now arrives when it's true. The trade-off is deliberate: the cue is honest, not early. Speech during the switch is not recovered — the OS never receives any audio while the profile switch is in flight, so nothing can capture it; the fix is to stop inviting the user to speak into that dead window.

How: MicCapture.start() no longer trusts AVAudioRecorder.record() returning true (that only means the AudioQueue started). It now polls recorder.currentTime every 50 ms until it advances past 0 — the recorder's clock only moves once the input device delivers frames, which distinguishes "route still switching" from "user is silent" (a level meter can't). The wait is capped per transport via CoreAudio's kAudioDevicePropertyTransportType: ~2.5 s for Bluetooth/BluetoothLE, ~300 ms for everything else, and on timeout it fails open (proceeds exactly as today) so a silent or broken mic can't brick the press. The cap policy and poll loop live in a new pure MicLiveness enum (unit-tested against the injected TestClock); the gap is logged so the fix is measurable in the field. DictationSession surfaces the wait as a new non-terminal .connecting phase claimed before mic.start(), projected to a new OverlayUIState.connecting (breathing "Connecting…" status line) while the menu bar stays at rest; RecordingCueGate needed no change — it keys on .recording, so the start chime now rides the connecting→recording edge by construction. A release or cancel landing mid-bring-up queues behind the press on the session's serial command queue and finalizes/tears down cleanly (pinned by a new GatedStartMic test); warmUp() is untouched and still starts no capture.

How it was tested

  • scripts/check.sh passes (or CI will, if I'm not on a Mac) — scripts/check.sh --portable passes locally; Swift build/tests need CI's macOS runner
  • I read AGENTS.md and this doesn't reintroduce anything deliberately removed (stays on the per-session AVAudioRecorder; no AVAudioEngine, no capture outside a press)
  • Docs updated if behavior changed (AGENTS.md, BLURTENGINE.md phase diagram + MicCapture contract)

New/updated engine tests: MicLivenessTests (transport caps, immediate/delayed/timeout waits on TestClock), DictationSessionTests (idle → connecting → recording sequencing; release landed during the bring-up still finalizes cleanly), RecordingCueGateTests (no chime during .connecting or on a failed bring-up), plus the .connecting rows in the PipelinePhase/OverlayUIState/MenuBarStatus projection tables.

Worth a real-device pass on AirPods before merging: dictate, wait ~10 s (so macOS falls back to A2DP), dictate again — the chime should land ~1–2 s after key-down and the first words should survive.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CNFeP9D8k1HJ1piwyidUv3


Generated by Claude Code

Blurt flipped to .recording — revealing the pill and playing the "speak
now" chime — the instant AVAudioRecorder.record() returned true. On a
Bluetooth input that only means the AudioQueue started: AirPods spend
~1-2 s switching A2DP→HFP first (again after every idle gap, since macOS
drops back to A2DP), and the OS captures nothing in that window, so the
user was cued to speak into a mic that wasn't delivering yet.

MicCapture.start() now holds until the recorder's clock advances past 0
(the AudioQueue timeline only moves once the device delivers frames,
which distinguishes a still-switching route from a silent user), polled
every 50 ms with a transport-aware cap (MicLiveness, new): ~2.5 s for
Bluetooth/BluetoothLE per kAudioDevicePropertyTransportType, ~300 ms for
everything else. On timeout it FAILS OPEN — proceeding exactly as
before — so a silent or broken mic degrades to today's behavior instead
of bricking the press. The wait/gap is logged for field measurement.

The session shows that wait as a new non-terminal .connecting phase,
claimed before mic.start(): the pill appears immediately in a distinct
"Connecting…" warming-up state (no REC tag, no waveform), and the start
chime rides the connecting→recording edge (RecordingCueGate needed no
change — it keys on .recording), so .recording keeps meaning "audio is
actually being captured". The menu bar stays at rest through the
bring-up for the same honesty rule. A release or cancel landing during
the bring-up queues behind the press on the session's serial command
queue and finalizes/tears down cleanly once start() returns.

Note the fix cues the user honestly rather than recovering early
speech — audio spoken during the switch never reaches the OS and cannot
be captured.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CNFeP9D8k1HJ1piwyidUv3
@github-actions

github-actions Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Dev build

Download Blurt.app — built from 6f91766, Debug-Local,
ad-hoc signed. Expires in 14 days.

Installing it
cd ~/Downloads
unzip -o blurt-dev-build-pr-134.zip   # GitHub wraps every artifact in a zip
unzip -o Blurt-dev-6f91766.zip
find Blurt.app -exec xattr -c {} +   # clear quarantine: xattr lost -r in macOS 12.3
rm -rf /Applications/Blurt.app && cp -R Blurt.app /Applications/
open -a Blurt

It is ad-hoc signed and not notarized: Gatekeeper refuses to open it until
the quarantine flag is cleared, and macOS treats it as a different app from a
released Blurt, so you have to re-grant Microphone, Accessibility, and Input
Monitoring. Reinstall the release DMG
when you are done reviewing.

Expect that re-grant once per dev build, including a second build of this
same PR. TCC pins an Accessibility grant to the signature that took it, and an
ad-hoc signature is just a hash of the binary, so every build is a new app as far
as tccd is concerned. Blurt clears the orphaned grant at launch, which is what
keeps the Accessibility step from getting stuck on a Blurt row that is switched on
and still denied. If you are coming from a build old enough to predate that,
clear the grant yourself once:

tccutil reset Accessibility dev.alex.blurt

… reveal + announcement, doc/test cleanups

Concurrency review:
- MicCapture: a stop() that interleaves during start()'s liveness wait now
  wins — start() re-checks a stop generation counter after the wait and
  tears the recorder down instead of installing it (previously the mic
  stayed hot and the temp WAV leaked until start() resumed).
- Document why the off-actor recorder.currentTime polling is safe
  (sole reference by confinement while start() is suspended).
- AGENTS.md: add CoreAudio to the engine's system-framework allowlist
  (and the project-guardrails skill's copy, which must stay in agreement).

HIG review:
- ConnectingLabel: raise the breath trough to the documented ~55%
  legibility floor — the faster 0.9 s period alone carries distinctness.
- ConnectingLabel: hold the label for 200 ms before revealing, so a fast
  (wired/built-in) bring-up shows only the dark capsule instead of
  flashing "Connecting…" mid fade-in.
- OverlayWindowController: announce .connecting to VoiceOver after the
  same 200 ms hold — Bluetooth-length waits get non-visual feedback,
  fast routes stay silent and go straight to the start chime.

Cleanup review:
- Hoist the Duration→milliseconds helper to Duration+Milliseconds.swift
  (internal) and reuse it in MicCapture instead of two hand-derived copies.
- Fix stale "idle→recording edge" wording on RecordingCueGate and its suite
  (the production edge is connecting→recording).
- Document on MicCaptureProtocol that start() may hold until frames flow
  (~2.5 s on Bluetooth) and hosts render the window as connecting.
- MicLiveness: drop pollInterval 50 ms → 10 ms (KeyInjector precedent);
  tests drive an injected clock, so they stay fast.
- Fold failedConnectingIsSilent into silentBetweenNonRecordingPhases.
- Assert GatedStartMic.startCalls in releaseDuringConnectingFinalizes.
- AGENTS.md repo map: mention the liveness gate in the Audio/ entry.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CNFeP9D8k1HJ1piwyidUv3
@claude
claude Bot marked this pull request as ready for review August 13, 2026 17:59
alexkroman pushed a commit that referenced this pull request Aug 13, 2026
Adopts the central fix from #134 and drops the one change here that
contradicted it.

`record()` returning true only means the AudioQueue started, not that the
input route is delivering frames. A Bluetooth mic spends ~1-2s switching
into its mic-capable profile first and the OS captures nothing in that
window, so returning immediately cues the user to speak into a dead mic
and the first words never reach the transcript. start() now polls
recorder.currentTime until it advances past 0 — the recorder's clock only
moves once frames arrive, which distinguishes "route still switching"
from "user is silent" (a level meter can't). Capped per transport by the
new pure MicLiveness (2.5s Bluetooth, 300ms otherwise) and failing open
on timeout, so a broken mic degrades to the old behavior rather than
bricking the press. stopGeneration covers the suspension this
introduces: a teardown landing mid-wait wins and the recorder is torn
down rather than installed.

The chime change is reverted. RecordingCueGate keys on .recording again,
so it rides the connecting->recording edge by construction and fires only
once audio actually flows. Firing it at the press — what this branch did
before — moved the "speak now" cue *earlier* into the dead window, making
the lost-first-words symptom worse. PipelinePhase.isCapturing existed
only to serve that, and is gone.

.starting is renamed .connecting throughout to match #134, and the pill
now breathes "Connecting…" on the same curve as "Transcribing…" since
the wait can last a second or two.

Kept from this branch, none of which #134 covers: the per-session
recorder re-warm (which composes with the gate — it keeps the route open
so the honest wait usually returns immediately), the Bluetooth tail
linger for the last word, cancelCapture(), and the cue re-prime on output
route change.

Transport classification moves to a pure, tested AudioTransport, leaving
AudioRoute as raw CoreAudio reads only — policy shouldn't hide in a file
the coverage gate can't reach. The warm-recorder lifecycle moves to
MicCapture+Warm.swift to stay inside the lint file-length budget.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JU1Ln2MKMsF9PJf1FLMQRX

Copy link
Copy Markdown
Collaborator

Closing in favour of #133, which now carries this PR's liveness gate plus four fixes for the same report that this one doesn't cover.

This PR had the key insight and #133 didn't. record() returning true only means the AudioQueue started, not that frames are arriving — so polling recorder.currentTime until it advances is the right signal, and it's the only thing here that actually saves the first words of an utterance. #133 was independently addressing the same Slack report from a different angle and had it wrong on one point: it moved the start chime to the press, which pushes the "speak now" cue even earlier into the dead window. That's reverted, and this PR's ordering (chime rides the connecting→recording edge) is what shipped.

Carried over from here essentially as written:

  • MicLiveness — the poll loop and the per-transport wait caps (2.5 s Bluetooth / 300 ms otherwise), failing open on timeout, with its TestClock-driven tests
  • stopGeneration, covering the suspension the wait introduces
  • The .connecting phase and its projections, and the Duration.milliseconds hoist out of AssemblyAITranscriber

What #133 adds on top, none of which this PR touches:

  • The tail. stop() keeps capturing 220 ms past key-up on a Bluetooth input, so the last word isn't truncated by the link's buffering. This PR fixes the head of the utterance; the tail was the other half of the same complaint.
  • A per-session recorder re-warm. The profile switch is paid at prepareToRecord(), i.e. every dictation, and warmUp() only ever covered the first one. Re-arming after each capture keeps the route open, so the honest wait this PR introduces usually returns immediately instead of costing 1–2 s. The two compose: the re-warm makes it fast, the gate makes it correct. (Bounded by a 60 s expiry, since holding the input open is what pins AirPods in the degraded-output profile.)
  • cancelCapture(), so a cancel skips both the tail linger and the file read-back rather than paying them to preserve audio it's about to discard.
  • Cue re-priming on output-route changes. Opening the mic drops the format the chime players were pre-rolled against, so the first chime after the flip is the one that stalls — and that's the chime at the start of a dictation.

The Slack request this came from is being served, not dropped — #133 is where it landed, and it keeps the real-device checklist from this PR's description (dictate, wait ~10 s for the link to fall back, dictate again).

Thanks — the gate was the right call and it's the core of the merged version.


Generated by Claude Code

@alexkroman alexkroman closed this Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants