diff --git a/AGENTS.md b/AGENTS.md index 8345b99..e8c70fa 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -164,7 +164,7 @@ Each was tried the other way and reverted. If a task seems to require one, stop | Add an external SPM dependency to the engine | Dependency-free by rule (biggest supply-chain risk); a `check.sh` guard fails on `.package(` in `Package.swift` or a `url:`/`github:` package in `project.yml`. Extend `BlurtEngine` instead. | | Use `AVAudioEngine` / `installTap` for capture | A long-lived engine bound its input graph to one device and went stale on a mic↔built-in switch — `-10868` (`kAudioUnitErr_FormatNotSupported`) or all-zero buffers. `MicCapture` uses a fresh `AVAudioRecorder` per session. | | Add streaming STT | The dictation API returns the full (already rewritten) text in one response; the overlay shows "Transcribing…" then the full text. | -| Add a client-side LLM cleanup pass | Cleanup is the dictation API's server-side rewrite, requested by the `llm` block on the same `/transcribe` call. No LLM Gateway client, no `StylerProtocol`, no styling stage, no second request — transcription steering belongs in `TranscriptionPrompt`. | +| Add a client-side LLM cleanup pass | Cleanup is the dictation API's server-side rewrite, requested by the `llm` block on the same `/v1/transcribe` call. No LLM Gateway client, no `StylerProtocol`, no styling stage, no second request — transcription steering belongs in `TranscriptionPrompt`. | | Add local models or model downloads | Transcription is a remote AssemblyAI call: no on-device ASR/LLM, no model cache, no download UI. | | Pin the prompt to English | Hurt non-English transcription; language is left to the model's own detection. | | Add a "remove filler words (um, uh, like)" clause | Not in the STT model's trained instruction set — a no-op, deliberately dropped; disfluency removal is the server-side LLM rewrite's job. | @@ -213,7 +213,7 @@ framework, or notarization rejects the build; roll-forward-only for a bad releas ```text DictationKeyTap (CGEventTap + DictationKeyGate) → AppCoordinator → DictationSession (actor) → MicCapture ↓ - AssemblyAITranscriber (STT + LLM cleanup, AssemblyAI dictation API: one POST /transcribe) + AssemblyAITranscriber (STT + LLM cleanup, AssemblyAI dictation API: one POST /v1/transcribe) ↓ KeyInjector → focused app (clipboard paste via a synthesized ⌘V CGEvent) ``` @@ -241,7 +241,7 @@ the seam they inject. ### `AssemblyAITranscriber` — `Sources/BlurtEngine/STT/AssemblyAITranscriber.swift` Implements `TranscriberProtocol` against AssemblyAI's **dictation** API: a single -`POST https://dictation.assemblyai.com/transcribe` with the captured audio as a raw S16LE PCM blob +`POST https://dictation.assemblyai.com/v1/transcribe` with the captured audio as a raw S16LE PCM blob in the `audio` multipart part plus a JSON `config` part (`sample_rate`, `channels`, `prompt`, and an empty `llm` block). No model header — the service pins the STT model server-side. The `prompt` (built per utterance by `TranscriptionPrompt`) steers _transcription_; the `llm` block asks the diff --git a/BLURTENGINE.md b/BLURTENGINE.md index 19d2b62..74e8f60 100644 --- a/BLURTENGINE.md +++ b/BLURTENGINE.md @@ -51,7 +51,7 @@ Before the first dictation can succeed the host must have: ```text press() ──▶ MicCapture.start() release() ──▶ MicCapture.stop() → Data (raw S16LE PCM) (16 kHz mono 16-bit PCM) AssemblyAITranscriber.transcribe(pcm:sampleRate:context:) - + focus/context capture (one POST dictation.assemblyai.com/transcribe: STT + LLM rewrite) + + focus/context capture (one POST dictation.assemblyai.com/v1/transcribe: STT + LLM rewrite) + connection warm-up KeyInjector.insert(text, after: priorText) (clipboard paste via synthesized ⌘V) ``` @@ -133,7 +133,7 @@ func transcribe(pcm: Data, sampleRate: Int, context: TranscriptionContext?) asyn func warmUp() async // optional; no-op default ``` -`AssemblyAITranscriber` is a stateless `Sendable` struct. One `POST https://dictation.assemblyai.com/transcribe` per utterance: the audio as raw S16LE PCM (the `pcm` blob, byte-for-byte) in the `audio` multipart part, plus a JSON `config` part (`sample_rate`, `channels`, the rendered `prompt`, and — while enhanced transcripts are enabled, the default — an empty `llm` block requesting the service's default cleanup rewrite), with the API key in `Authorization` (no model header — the service pins the STT model server-side). The response carries the verbatim `text` and the rewritten `llm_response`; the transcriber returns the rewrite and falls back to `text` when it is null (the rewrite is best-effort — `llm_error` is logged, never surfaced as a failure). Its initializer takes an `apiKeyProvider` closure (defaults to `APIKeyStore.current`), a `baseURL`, an `HTTPTransport` — inject a fake transport (see `Tests/BlurtEngineTests/Stubs/FakeHTTPTransport.swift`) to test against canned responses — and an `enhancedTranscripts` closure deciding, per request, whether the `llm` block is sent (nil, the default, reads `EnhancedTranscriptsStore`). `warmUp()` fires a throwaway GET at the host root to pre-pool the connection; it never throws and any failure just means the real request pays connection setup as before. +`AssemblyAITranscriber` is a stateless `Sendable` struct. One `POST https://dictation.assemblyai.com/v1/transcribe` per utterance: the audio as raw S16LE PCM (the `pcm` blob, byte-for-byte) in the `audio` multipart part, plus a JSON `config` part (`sample_rate`, `channels`, the rendered `prompt`, and — while enhanced transcripts are enabled, the default — an empty `llm` block requesting the service's default cleanup rewrite), with the API key in `Authorization` (no model header — the service pins the STT model server-side). The response carries the verbatim `text` and the rewritten `llm_response`; the transcriber returns the rewrite and falls back to `text` when it is null (the rewrite is best-effort — `llm_error` is logged, never surfaced as a failure). Its initializer takes an `apiKeyProvider` closure (defaults to `APIKeyStore.current`), a `baseURL`, an `HTTPTransport` — inject a fake transport (see `Tests/BlurtEngineTests/Stubs/FakeHTTPTransport.swift`) to test against canned responses — and an `enhancedTranscripts` closure deciding, per request, whether the `llm` block is sent (nil, the default, reads `EnhancedTranscriptsStore`). `warmUp()` fires a throwaway GET at the host root to pre-pool the connection; it never throws and any failure just means the real request pays connection setup as before. The model's limits live in `SyncSTTLimits` (16 kHz sample rate, ~0.1 s–120 s audio, and the auto-release math — the sync STT model behind the dictation service) — the single source shared by the mic, the session, and the request so recorded and declared geometry can't drift. diff --git a/README.md b/README.md index 9075c6c..a8ada21 100644 --- a/README.md +++ b/README.md @@ -165,7 +165,7 @@ directories, so the app needs a stable install path to be usable at all. Sources/BlurtEngine/ Swift 6 package owning the pipeline — no external dependencies Audio/ MicCapture: fresh AVAudioRecorder per session, 16 kHz mono PCM, live level meter; DX7/Juno-106 sound packs - STT/ AssemblyAITranscriber: one POST to dictation.assemblyai.com/transcribe + STT/ AssemblyAITranscriber: one POST to dictation.assemblyai.com/v1/transcribe (STT + LLM rewrite) + TranscriptionPrompt contextual priming Pipeline/ DictationSession actor: press/release/cancel commands, phase stream, auto-release before the API's recording cap diff --git a/Sources/BlurtEngine/STT/AssemblyAITranscriber.swift b/Sources/BlurtEngine/STT/AssemblyAITranscriber.swift index 55da320..b0a6e9f 100644 --- a/Sources/BlurtEngine/STT/AssemblyAITranscriber.swift +++ b/Sources/BlurtEngine/STT/AssemblyAITranscriber.swift @@ -9,7 +9,7 @@ private let transcriberLog = Logger(subsystem: BlurtIdentity.subsystem, category /// `TranscriberProtocol` backed by AssemblyAI's **dictation** API. /// -/// A single `POST dictation.assemblyai.com/transcribe` carries the captured +/// A single `POST dictation.assemblyai.com/v1/transcribe` carries the captured /// audio (raw S16LE PCM, exactly the bytes the mic recorded — there is no /// re-encoding pass) plus a JSON `config` part, and the response body carries /// both the verbatim transcript and — when the config requests one via its @@ -65,7 +65,7 @@ public struct AssemblyAITranscriber: TranscriberProtocol { let config = try makeConfigData(sampleRate: sampleRate, prompt: prompt) let boundary = "blurt-\(UUID().uuidString)" - var request = URLRequest(url: baseURL.appendingPathComponent("transcribe")) + var request = URLRequest(url: baseURL.appendingPathComponent("v1/transcribe")) request.httpMethod = "POST" // Bounds a stalled connection; see `requestTimeoutSeconds` for why an idle // timeout is the right shape here. @@ -98,7 +98,7 @@ public struct AssemblyAITranscriber: TranscriberProtocol { /// `transcribe` reuses it instead of paying DNS+TCP+TLS on the hot path /// (~170 ms cold, more on mobile — measured). A throwaway GET to the host /// root is enough to establish the HTTP/2 connection `URLSession` then reuses - /// for the POST to `/transcribe`; the response (an auth-less 4xx) is + /// for the POST to `/v1/transcribe`; the response (an auth-less 4xx) is /// discarded. No key, so it never counts as a transcription. A short timeout /// keeps a dead network from leaving the task hanging. Fire-and-forget: any /// error is swallowed — a failed warm-up just means the next request pays diff --git a/Tests/BlurtEngineTests/AssemblyAITranscriberTests.swift b/Tests/BlurtEngineTests/AssemblyAITranscriberTests.swift index 9744a61..de05d4b 100644 --- a/Tests/BlurtEngineTests/AssemblyAITranscriberTests.swift +++ b/Tests/BlurtEngineTests/AssemblyAITranscriberTests.swift @@ -16,7 +16,7 @@ struct HTTPClientTests { let hits = Counter() let transport = FakeHTTPTransport { request in _ = hits.next() - guard request.url?.path.hasSuffix("/transcribe") == true, + guard request.url?.path == "/v1/transcribe", request.httpMethod == "POST" else { return (404, Data()) } return (200, json(["text": "um hello world", "llm_response": "Hello world."])) @@ -55,7 +55,7 @@ struct HTTPClientTests { @Test("transcriber succeeds with a real context (builds and sends a prompt)") func transcribeWithContext() async throws { let transport = FakeHTTPTransport { request in - guard request.url?.path.hasSuffix("/transcribe") == true else { return (404, Data()) } + guard request.url?.path == "/v1/transcribe" else { return (404, Data()) } return (200, json(["text": "hello world"])) }