fix: gate recording on mic liveness so AirPods stop losing the first words - #134
fix: gate recording on mic liveness so AirPods stop losing the first words#134claude[bot] wants to merge 2 commits into
Conversation
Blurt flipped to .recording — revealing the pill and playing the "speak now" chime — the instant AVAudioRecorder.record() returned true. On a Bluetooth input that only means the AudioQueue started: AirPods spend ~1-2 s switching A2DP→HFP first (again after every idle gap, since macOS drops back to A2DP), and the OS captures nothing in that window, so the user was cued to speak into a mic that wasn't delivering yet. MicCapture.start() now holds until the recorder's clock advances past 0 (the AudioQueue timeline only moves once the device delivers frames, which distinguishes a still-switching route from a silent user), polled every 50 ms with a transport-aware cap (MicLiveness, new): ~2.5 s for Bluetooth/BluetoothLE per kAudioDevicePropertyTransportType, ~300 ms for everything else. On timeout it FAILS OPEN — proceeding exactly as before — so a silent or broken mic degrades to today's behavior instead of bricking the press. The wait/gap is logged for field measurement. The session shows that wait as a new non-terminal .connecting phase, claimed before mic.start(): the pill appears immediately in a distinct "Connecting…" warming-up state (no REC tag, no waveform), and the start chime rides the connecting→recording edge (RecordingCueGate needed no change — it keys on .recording), so .recording keeps meaning "audio is actually being captured". The menu bar stays at rest through the bring-up for the same honesty rule. A release or cancel landing during the bring-up queues behind the press on the session's serial command queue and finalizes/tears down cleanly once start() returns. Note the fix cues the user honestly rather than recovering early speech — audio spoken during the switch never reaches the OS and cannot be captured. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CNFeP9D8k1HJ1piwyidUv3
Dev buildDownload Blurt.app — built from Installing itcd ~/Downloads
unzip -o blurt-dev-build-pr-134.zip # GitHub wraps every artifact in a zip
unzip -o Blurt-dev-6f91766.zip
find Blurt.app -exec xattr -c {} + # clear quarantine: xattr lost -r in macOS 12.3
rm -rf /Applications/Blurt.app && cp -R Blurt.app /Applications/
open -a BlurtIt is ad-hoc signed and not notarized: Gatekeeper refuses to open it until Expect that re-grant once per dev build, including a second build of this tccutil reset Accessibility dev.alex.blurt |
… reveal + announcement, doc/test cleanups Concurrency review: - MicCapture: a stop() that interleaves during start()'s liveness wait now wins — start() re-checks a stop generation counter after the wait and tears the recorder down instead of installing it (previously the mic stayed hot and the temp WAV leaked until start() resumed). - Document why the off-actor recorder.currentTime polling is safe (sole reference by confinement while start() is suspended). - AGENTS.md: add CoreAudio to the engine's system-framework allowlist (and the project-guardrails skill's copy, which must stay in agreement). HIG review: - ConnectingLabel: raise the breath trough to the documented ~55% legibility floor — the faster 0.9 s period alone carries distinctness. - ConnectingLabel: hold the label for 200 ms before revealing, so a fast (wired/built-in) bring-up shows only the dark capsule instead of flashing "Connecting…" mid fade-in. - OverlayWindowController: announce .connecting to VoiceOver after the same 200 ms hold — Bluetooth-length waits get non-visual feedback, fast routes stay silent and go straight to the start chime. Cleanup review: - Hoist the Duration→milliseconds helper to Duration+Milliseconds.swift (internal) and reuse it in MicCapture instead of two hand-derived copies. - Fix stale "idle→recording edge" wording on RecordingCueGate and its suite (the production edge is connecting→recording). - Document on MicCaptureProtocol that start() may hold until frames flow (~2.5 s on Bluetooth) and hosts render the window as connecting. - MicLiveness: drop pollInterval 50 ms → 10 ms (KeyInjector precedent); tests drive an injected clock, so they stay fast. - Fold failedConnectingIsSilent into silentBetweenNonRecordingPhases. - Assert GatedStartMic.startCalls in releaseDuringConnectingFinalizes. - AGENTS.md repo map: mention the liveness gate in the Audio/ entry. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CNFeP9D8k1HJ1piwyidUv3
Adopts the central fix from #134 and drops the one change here that contradicted it. `record()` returning true only means the AudioQueue started, not that the input route is delivering frames. A Bluetooth mic spends ~1-2s switching into its mic-capable profile first and the OS captures nothing in that window, so returning immediately cues the user to speak into a dead mic and the first words never reach the transcript. start() now polls recorder.currentTime until it advances past 0 — the recorder's clock only moves once frames arrive, which distinguishes "route still switching" from "user is silent" (a level meter can't). Capped per transport by the new pure MicLiveness (2.5s Bluetooth, 300ms otherwise) and failing open on timeout, so a broken mic degrades to the old behavior rather than bricking the press. stopGeneration covers the suspension this introduces: a teardown landing mid-wait wins and the recorder is torn down rather than installed. The chime change is reverted. RecordingCueGate keys on .recording again, so it rides the connecting->recording edge by construction and fires only once audio actually flows. Firing it at the press — what this branch did before — moved the "speak now" cue *earlier* into the dead window, making the lost-first-words symptom worse. PipelinePhase.isCapturing existed only to serve that, and is gone. .starting is renamed .connecting throughout to match #134, and the pill now breathes "Connecting…" on the same curve as "Transcribing…" since the wait can last a second or two. Kept from this branch, none of which #134 covers: the per-session recorder re-warm (which composes with the gate — it keeps the route open so the honest wait usually returns immediately), the Bluetooth tail linger for the last word, cancelCapture(), and the cue re-prime on output route change. Transport classification moves to a pure, tested AudioTransport, leaving AudioRoute as raw CoreAudio reads only — policy shouldn't hide in a file the coverage gate can't reach. The warm-recorder lifecycle moves to MicCapture+Warm.swift to stay inside the lint file-length budget. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JU1Ln2MKMsF9PJf1FLMQRX
|
Closing in favour of #133, which now carries this PR's liveness gate plus four fixes for the same report that this one doesn't cover. This PR had the key insight and #133 didn't. Carried over from here essentially as written:
What #133 adds on top, none of which this PR touches:
The Slack request this came from is being served, not dropped — #133 is where it landed, and it keeps the real-device checklist from this PR's description (dictate, wait ~10 s for the link to fall back, dictate again). Thanks — the gate was the right call and it's the core of the merged version. Generated by Claude Code |
Requested by Dylan Fox · Slack thread
What & why
Before: on AirPods (any Bluetooth mic), the pill and the start chime appear the instant the hotkey is pressed — but the device takes ~1–2 s to switch its audio profile (A2DP→HFP) before the mic actually delivers anything, and macOS drops back to A2DP after every idle gap. So on the first dictation and every dictation after a pause, the user is cued to speak while nothing is being captured, and the first words of the utterance are missing from the transcript.
After: the pill still appears immediately on key-down, but in a distinct "Connecting…" warming-up state (no REC tag, no waveform). It flips to the normal recording state — and the start chime plays — only once the mic is actually delivering audio. On wired/built-in mics the warming state is a blink (audio flows near-instantly); on AirPods the chime now arrives when it's true. The trade-off is deliberate: the cue is honest, not early. Speech during the switch is not recovered — the OS never receives any audio while the profile switch is in flight, so nothing can capture it; the fix is to stop inviting the user to speak into that dead window.
How:
MicCapture.start()no longer trustsAVAudioRecorder.record()returningtrue(that only means the AudioQueue started). It now pollsrecorder.currentTimeevery 50 ms until it advances past 0 — the recorder's clock only moves once the input device delivers frames, which distinguishes "route still switching" from "user is silent" (a level meter can't). The wait is capped per transport via CoreAudio'skAudioDevicePropertyTransportType: ~2.5 s for Bluetooth/BluetoothLE, ~300 ms for everything else, and on timeout it fails open (proceeds exactly as today) so a silent or broken mic can't brick the press. The cap policy and poll loop live in a new pureMicLivenessenum (unit-tested against the injectedTestClock); the gap is logged so the fix is measurable in the field.DictationSessionsurfaces the wait as a new non-terminal.connectingphase claimed beforemic.start(), projected to a newOverlayUIState.connecting(breathing "Connecting…" status line) while the menu bar stays at rest;RecordingCueGateneeded no change — it keys on.recording, so the start chime now rides the connecting→recording edge by construction. A release or cancel landing mid-bring-up queues behind the press on the session's serial command queue and finalizes/tears down cleanly (pinned by a newGatedStartMictest);warmUp()is untouched and still starts no capture.How it was tested
scripts/check.shpasses (or CI will, if I'm not on a Mac) —scripts/check.sh --portablepasses locally; Swift build/tests need CI's macOS runnerAVAudioRecorder; noAVAudioEngine, no capture outside a press)MicCapturecontract)New/updated engine tests:
MicLivenessTests(transport caps, immediate/delayed/timeout waits onTestClock),DictationSessionTests(idle → connecting → recordingsequencing; release landed during the bring-up still finalizes cleanly),RecordingCueGateTests(no chime during.connectingor on a failed bring-up), plus the.connectingrows in thePipelinePhase/OverlayUIState/MenuBarStatusprojection tables.Worth a real-device pass on AirPods before merging: dictate, wait ~10 s (so macOS falls back to A2DP), dictate again — the chime should land ~1–2 s after key-down and the first words should survive.
🤖 Generated with Claude Code
https://claude.ai/code/session_01CNFeP9D8k1HJ1piwyidUv3
Generated by Claude Code