Skip to content

Spec 0002: whisper-flow as the transcription engine - #7

Merged
dimastatz merged 2 commits into
mainfrom
docs/spec-whisper-flow
Oct 6, 2026
Merged

dimastatz merged 2 commits into
mainfrom
docs/spec-whisper-flow

Conversation

@dimastatz

Copy link
Copy Markdown
Owner

Summary

Adds docs/specs/0002-whisper-flow-transcription.md. It requires whisper-flow as the streaming transcription framework. The extension captures audio and streams it to a whisper-flow server's /ws endpoint, and does not run Whisper itself.

Requirements have IDs and are marked must or should:

  • Engine (TR): whisper-flow only, no bundled model and no cloud STT. Minimum version 1.1.0, checked with GET /ready.
  • Configuration (CF): whisperCode.serverUrl (default http://localhost:8181), API key in SecretStorage sent as x-api-key, and explicit consent before streaming to a non-local host.
  • Audio (AU): 16 kHz mono int16 PCM in binary frames of 1024 samples, captured by a sidecar process rather than a webview.
  • Results (RS): partials are not monotonic, so a pending range is replaced on each partial and committed when is_partial is false. One undo step per segment.
  • Errors (ER): close codes 1008 and 1013, retry on disconnect, no orphaned sidecar processes.
  • Testing (TS): client logic lives in modules that don't import vscode and counts toward the 95% gate. CI uses a fake WebSocket server, so it doesn't need whisper-flow or a microphone.

The spec also lists gaps in whisper-flow 1.1.0 that affect whisper-code: no finalize-on-stop, no protocol_version, no initial_prompt for vocabulary, a fixed model per server, a window cap of about 64 s, and endpointing based on unchanged text.

Changes elsewhere

  • Spec 0001: goal 1 changes from "local-only" to "self-hosted whisper-flow server, localhost by default, no cloud STT". The architecture diagram is updated, and the runtime and mic-capture open questions are marked resolved.
  • CLAUDE.md and README: updated to name whisper-flow as the only transcription engine.

Review notes

  • This changes the privacy promise. Audio can now go to a remote host the user configures, with consent (CF-4).
  • Workspace-symbol biasing (0001 goal 2) is blocked until whisper-flow accepts a vocabulary prompt.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TC6WCRgEsuzNtiSyAN59vi

dimastatz and others added 2 commits October 6, 2026 05:52
Requires all streaming transcription to go through a whisper-flow
server over its /ws WebSocket. Covers configuration, audio format,
partial/final result handling, errors, testing, and the gaps in
whisper-flow 1.1.0 that affect the integration. Updates spec 0001's
local-only goal and open questions, CLAUDE.md, and README to match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TC6WCRgEsuzNtiSyAN59vi
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TC6WCRgEsuzNtiSyAN59vi
@dimastatz
dimastatz merged commit ae0489a into main Oct 6, 2026
5 checks passed
@dimastatz
dimastatz deleted the docs/spec-whisper-flow branch October 6, 2026 03:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant