Implement dictation end to end (spec 0001)

CHANGELOG.md

@@ -4,6 +4,29 @@ All notable changes to this project are documented here. The format is based on
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project follows
 [Semantic Versioning](https://semver.org/).
 
+## [Unreleased]
+
+Dictation works end to end ([spec 0001](docs/specs/0001-overview.md), on
+[spec 0002](docs/specs/0002-whisper-flow-transcription.md)).
+
+### Added
+
+- Streaming transcription through a whisper-flow server (1.1.0+): `/ready` check, version
+  check, WebSocket streaming of 16 kHz PCM, and finalizing the last phrase on stop (1.2.0+).
+- Microphone capture in a sidecar process: SoX, `arecord` or `ffmpeg`, or
+  `whisperCode.recorderCommand`.
+- Live editor insertion: partial text is dimmed and replaced as you speak, each phrase is one
+  undo step, and the selection is replaced.
+- Workspace vocabulary: identifiers from open editors are sent as Whisper's prompt (1.2.0+)
+  and respelled in the output.
+- Context-aware formatting for code comments and commit messages.
+- Settings `whisperCode.serverUrl`, `whisperCode.model`, `whisperCode.vocabulary.enabled` and
+  `whisperCode.recorderCommand`; commands **Set API Key** and **Clear API Key** (secret
+  storage).
+- Confirmation before sending audio to a non-local server, and a warning without TLS.
+- Status bar states: idle, connecting, recording, error. Errors for a bad API key, a full
+  server, and lost connections (with **Retry**).
+
 ## [0.1.0] - 2026-10-05
 
 First preview release. Sets up the extension and project tooling; audio capture and

CLAUDE.md

@@ -43,9 +43,18 @@ Press **F5** in VS Code ("Run Extension") to launch the Extension Development Ho
 
 ## Layout
 
-- `src/extension.ts` — entry point (`activate`/`deactivate`); registers
-  `whisperCode.toggleDictation` and the status bar item. Dictation itself is still a stub.
-- `src/test/*.test.ts` — integration tests (Mocha `suite`/`test`).
+- `src/extension.ts` — entry point (`activate`/`deactivate`): commands, settings, consent
+  prompts for non-local servers, the four-state status bar item.
+- `src/dictation/session.ts` — one dictation session (`/ready` → socket → recorder → sink).
+  No `vscode` import; `editorSink.ts` (pending range, decorations) and
+  `workspaceVocabulary.ts` (symbol index) are the `vscode` side.
+- `src/whisperflow/` — whisper-flow protocol: URLs and `/ready` (`server.ts`), server
+  messages and close codes (`messages.ts`), the WebSocket client (`client.ts`).
+- `src/audio/recorder.ts` — microphone sidecar process (sox / arecord / ffmpeg) and framing.
+- `src/text/` — context detector, post-processor, vocabulary. Plain functions.
+- `src/test/*.test.ts` — tests (Mocha `suite`/`test`). `fakeServer.ts` is a fake whisper-flow
+  (real HTTP + WebSocket) and `fakeRecorder.ts` a fake recorder process; no real server or
+  microphone is needed.
 - `docs/specs/` — design specs; start with `0001-overview.md`.
 - `docs/images/` — images referenced from docs.
 
@@ -57,4 +66,8 @@ Press **F5** in VS Code ("Run Extension") to launch the Extension Development Ho
 - Keep coverage at or above 95%: add tests with new code rather than excluding files.
 - Logic that doesn't need the `vscode` API should live in plain modules so it can be
   tested directly.
+- whisper-flow (uvicorn) rejects a handshake with HTTP 403 for both a bad API key and a full
+  server, never close codes 1008/1013; `classifyRejectedHandshake` tells them apart.
+- The `undo` command is a no-op in the test window (it never has OS focus), so undo grouping
+  can't be asserted in tests.
 - Commit on a feature branch and open a PR against `main`; don't push directly to `main`.

README.md

@@ -2,9 +2,8 @@
 
 Real-time voice dictation for VS Code, powered by self-hosted Whisper through [whisper-flow](https://github.com/dimastatz/whisper-flow). Biases transcription with your workspace symbols so it spells useMemo right, and adapts to whether you're dictating a comment, a commit message, or a prompt.
 
-> **Status: early development.** The extension installs and the dictation command and
-> status bar toggle work, but audio capture and transcription are not implemented yet.
-> See [docs/specs/0001-overview.md](docs/specs/0001-overview.md) for the plan.
+> **Status: preview.** Dictation works end to end against a whisper-flow server. Expect rough
+> edges; see [docs/specs](docs/specs) for the design.
 
 ## Usage
 
@@ -14,10 +13,39 @@ Real-time voice dictation for VS Code, powered by self-hosted Whisper through [w
 |                        | Or click the microphone **Dictate** item in the status bar                                                              |
 |                        | Or run **Whisper Code: Toggle Dictation** from the Command Palette                                                      |
 
-While dictating, the status bar item changes to **Dictating**. Toggle again to stop.
+The status bar shows **Connecting**, then **Dictating** while it records. Text appears dimmed as
+you speak and settles when each phrase is final; toggle again to stop. Each phrase is one undo
+step.
 
-Transcription runs on a [whisper-flow](https://github.com/dimastatz/whisper-flow) server that you
-run yourself, on `localhost` by default. No third-party cloud speech service is used.
+Output adapts to where the cursor is:
+
+- **Code comment:** sentences start with a capital letter after the comment marker.
+- **Commit message** (`COMMIT_EDITMSG`, git commit editor): the subject line has no trailing
+  period.
+- **Anything else** (prose, prompts): inserted as spoken.
+
+Identifiers from your open editors (such as `useMemo` or `getUserById`) are sent to the server
+as a vocabulary hint and respelled in the text, so "use memo" comes out as `useMemo`.
+
+## Requirements
+
+- A [whisper-flow](https://github.com/dimastatz/whisper-flow) server, 1.1.0 or newer (1.2.0+
+  for the vocabulary hint and model choice). Run it yourself, for example with
+  `./run.sh -run-server` or Docker. No third-party cloud speech service is used.
+- A command-line audio recorder: [SoX](https://sourceforge.net/projects/sox/) (`brew install sox`,
+  `apt install sox`), or `arecord` on Linux, or `ffmpeg` on macOS and Linux.
+
+## Settings
+
+| Setting                          | Default                 | Description                                                                                             |
+| -------------------------------- | ----------------------- | ------------------------------------------------------------------------------------------------------- |
+| `whisperCode.serverUrl`          | `http://localhost:8181` | whisper-flow server. A non-local server asks for your confirmation once, and warns without `https`.     |
+| `whisperCode.model`              | (server default)        | Model from the server's `/ready` list, such as `base.en.pt`.                                            |
+| `whisperCode.vocabulary.enabled` | `true`                  | Bias transcription toward identifiers from open editors.                                                |
+| `whisperCode.recorderCommand`    | (auto)                  | Command that writes raw 16 kHz mono 16-bit little-endian PCM to stdout, if the defaults don't suit you. |
+
+If the server sets `WF_API_KEY`, run **Whisper Code: Set API Key**. The key is kept in VS Code's
+secret storage, not in `settings.json`.
 
 ## Running from source
 

package.json

@@ -1,7 +1,7 @@
 {
   "name": "whisper-code",
   "displayName": "Whisper Code",
-  "description": "Real-time voice dictation for VS Code, powered by local Whisper.",
+  "description": "Real-time voice dictation for VS Code, powered by a self-hosted whisper-flow server.",
   "version": "0.1.0",
   "publisher": "dimastatz",
   "license": "MIT",
@@ -25,6 +25,16 @@
         "command": "whisperCode.toggleDictation",
         "title": "Toggle Dictation",
         "category": "Whisper Code"
+      },
+      {
+        "command": "whisperCode.setApiKey",
+        "title": "Set API Key",
+        "category": "Whisper Code"
+      },
+      {
+        "command": "whisperCode.clearApiKey",
+        "title": "Clear API Key",
+        "category": "Whisper Code"
       }
     ],
     "keybindings": [
@@ -33,7 +43,35 @@
         "key": "ctrl+alt+d",
         "mac": "cmd+alt+d"
       }
-    ]
+    ],
+    "configuration": {
+      "title": "Whisper Code",
+      "properties": {
+        "whisperCode.serverUrl": {
+          "type": "string",
+          "default": "http://localhost:8181",
+          "markdownDescription": "URL of the [whisper-flow](https://github.com/dimastatz/whisper-flow) server. The WebSocket URL is derived from it (`http` \u2192 `ws`, `https` \u2192 `wss`, plus `/ws`). Audio is sent only to this server; a non-local server needs your confirmation once."
+        },
+        "whisperCode.model": {
+          "type": "string",
+          "default": "",
+          "markdownDescription": "Model for each session, from the server's `GET /ready` `models` list (for example `base.en.pt`). Empty uses the server's default. Needs whisper-flow 1.2 or newer."
+        },
+        "whisperCode.vocabulary.enabled": {
+          "type": "boolean",
+          "default": true,
+          "markdownDescription": "Bias transcription toward identifiers from open editors (such as `useMemo` or `getUserById`), and respell them in the inserted text. Sending the vocabulary to the server needs whisper-flow 1.2 or newer."
+        },
+        "whisperCode.recorderCommand": {
+          "type": "array",
+          "items": {
+            "type": "string"
+          },
+          "default": [],
+          "markdownDescription": "Command that records the microphone and writes raw 16 kHz mono signed 16-bit little-endian PCM to stdout, for example `[\"sox\", \"-q\", \"-d\", \"-t\", \"raw\", \"-r\", \"16000\", \"-e\", \"signed-integer\", \"-b\", \"16\", \"-c\", \"1\", \"-L\", \"-\"]`. Empty tries `sox`, then `arecord` (Linux) or `ffmpeg`."
+        }
+      }
+    }
   },
   "scripts": {
     "vscode:prepublish": "npm run compile",
@@ -53,11 +91,15 @@
     "@types/mocha": "^10.0.10",
     "@types/node": "20.x",
     "@types/vscode": "^1.95.0",
+    "@types/ws": "^8.18.2",
     "@vscode/test-cli": "^0.0.15",
     "@vscode/test-electron": "^3.1.0",
     "eslint": "^9.25.1",
     "prettier": "^3.9.9",
     "typescript": "^5.8.3",
     "typescript-eslint": "^8.31.1"
+  },
+  "dependencies": {
+    "ws": "^8.22.0"
   }
 }