AudioWorklet voice pipeline: speech-to-LLM-to-TTS in browser

[sauravpanda]

Context

AudioWorklet runs in a separate high-priority thread. Combined with streaming TTS, sub-2.5s voice-to-voice latency is achievable entirely in-browser.

Proposal

• AudioWorklet captures audio → WASM processes speech features
• Streaming speech-to-text (Whisper-tiny in WASM or WebNN)
• LLM generates response token-by-token
• Stream tokens to browser TTS (SpeechSynthesis API or neural TTS model)
• All in-browser, zero server roundtrips

Impact

• Voice assistant with complete privacy — nothing leaves the device
• Works offline with cached models