Skip to content

record_held()'s blocking stream.read() loop captures ~3-5x quieter audio than callback-based capture (Windows/MME) #20

Description

@TeddyNONSTOP

Summary

On Windows, using the MME host API, ears.record_held()'s blocking stream.read() polling loop captures audio at roughly 1/3 to 1/5 the amplitude of PortAudio's own callback-based capture path (the one sounddevice.rec() uses internally), on identical hardware, identical settings, back to back. The attenuation is severe enough to push real speech into Whisper's near-silence hallucination range — every utterance transcribes as a stray filler word ("You", "you you you") even though the person is speaking normally and at full mic volume.

This is a different bug from #12 (phantom key-release fragmenting the recording): here a single clean multi-second hold produces one continuous low-amplitude buffer, not many empty fragments. The RELEASE_GRACE fix for #12 is present and working correctly in this case — is_held() stayed True for the full hold, [you] logged a real (if garbled) transcription each time, not silence.

Evidence

Tested on a Logitech C922 Pro Stream Webcam mic, Windows 11, MME host API, stt_device: "cpu", mic_device resolved correctly to the intended device (confirmed via [ears] mic_device ... not found never logging). OS-level input volume confirmed at 100 and boosted; Windows' own mic-level meter confirmed picking up the voice clearly.

Four capture methods, same device index, same samplerate=16000, channels=1, dtype="int16", same person speaking at consistent volume for ~5s each, run back to back in the same process:

Method Peak (% of int16 full scale)
sd.rec() (PortAudio callback under the hood) 43.4%
Manual callback-based InputStream(..., callback=...) 25.0%
InputStream + blocking stream.read(FRAME_LEN) loop (record_held()'s actual method) 9.1%
InputStream + blocking .read(), default blocksize instead of FRAME_LEN 8.3%

Blocksize was ruled out as the variable (8.3% vs 9.1%, no meaningful difference). The DirectSound variant of the same device was tested too and is unrelated/separately broken — it returns a runaway loop of empty zero-filled buffers under the same test (stream.read() returning near-instantly, over a million reads in 5 seconds), so that's not a viable fallback either. The WASAPI variant rejects 16000 Hz outright (Invalid sample rate [PaErrorCode -9997]) since WASAPI wants the device's native rate.

Reproduce

  1. Windows, MME host API, a USB device like the C922 webcam mic.
  2. mic_device set to match that device (or system default, if MME is what it resolves to).
  3. Hold the PTT key, speak a full, clearly audible sentence for several seconds, release.
  4. record_held() produces a real but low-amplitude buffer; Whisper returns a stray hallucinated word instead of the actual sentence, even though the same hardware captures cleanly at ~2-4x the amplitude via a callback-based stream.

Suggested fix

Switch record_held()'s frame collection from a blocking stream.read() polling loop to a callback-based InputStream(..., callback=...), appending frames from the callback instead of polling .read() in the while is_held() loop. This matches the internal approach sounddevice.rec() already uses and measured 2.75x louder on the same hardware in the table above. is_held() would gate when to stop appending (or when to close the stream) rather than gating the read loop itself.

Happy to test a patch against this hardware if useful — this is blocking Windows + MME + USB webcam mic users from getting usable transcription at all, with no error or log line indicating why (matches the framing at the top of #12 — a healthy-looking voice line that simply mishears everything).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions