diff --git a/.claude/skills/project-guardrails/SKILL.md b/.claude/skills/project-guardrails/SKILL.md index 42671d3..8cecfd9 100644 --- a/.claude/skills/project-guardrails/SKILL.md +++ b/.claude/skills/project-guardrails/SKILL.md @@ -30,11 +30,15 @@ one, stop and ask the user first. This is the fast "don't" list; AGENTS.md's call. No on-device ASR/LLM, no model cache, no download UI. - **Never send `config.prompt`.** It takes a _description of the audio_, not instructions, and a custom value replaces the service's managed default - including its language steering. Formatting goes in `llm.instruction` - ("Format the result as markdown."), vocabulary in `keyterms_prompt`, the text - before the cursor in `conversation_context`. An imperative like "Transcribe - speech into markdown." in `prompt` is a measured no-op — that is exactly why - these three fields exist. + including its language steering. Vocabulary goes in `keyterms_prompt`, the + text before the cursor in `conversation_context`. An imperative like + "Transcribe speech into markdown." in `prompt` is a measured no-op — that is + exactly why those fields exist. +- **Never send anything about the frontmost app.** `AppKindPriming` recognized + the app's bundle ID as a kind (terminal, code editor, Slack, Obsidian) and + sent a formatting clause as `llm.instruction`; the whole path was removed. + The bundle ID isn't captured, and `llm` always goes out empty so the + service's default cleanup rewrite applies everywhere. - Don't reintroduce a "remove filler words (um, uh, like)" directive — the STT prompt doesn't act on it; it was deliberately dropped, and disfluency removal is the LLM rewrite's job. Same for a language directive: pinning to English diff --git a/AGENTS.md b/AGENTS.md index 83fd15c..93672bf 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -8,10 +8,9 @@ the Claude-Code-specific tooling under `.claude/` (hooks, skills, subagents). Blurt is a macOS dictation app powered by [AssemblyAI](https://www.assemblyai.com). Tap or hold a trigger key, speak, and polished text is pasted into the focused app. Transcription is **one remote AssemblyAI dictation API call**: per-utterance steering rides along with the request — the text before -the cursor as `conversation_context`, the user's key terms as `keyterms_prompt`, and an app-kind -formatting clause recognized from the frontmost app — and the same request asks the service for its -server-side LLM cleanup rewrite (`config.llm`), so the text that comes back is already polished. The -user supplies their own API key. +the cursor as `conversation_context` and the user's key terms as `keyterms_prompt` — and the same +request asks the service for its server-side LLM cleanup rewrite (`config.llm`), so the text that +comes back is already polished. The user supplies their own API key. Four reflexes before you touch anything: @@ -53,7 +52,7 @@ Sources/BlurtEngine/ the engine (dependency-free Swift package) Injection/ KeyInjector (clipboard paste), SystemClipboard Permissions/ PermissionsChecker (mic + Accessibility) Pipeline/ DictationSession (actor) + phases, UI projections, geometry, log - STT/ AssemblyAITranscriber, TranscriptionSteering/Context, AppKindPriming, SyncSTTLimits + STT/ AssemblyAITranscriber, TranscriptionSteering/Context, SyncSTTLimits Update/ UpdateChecker (download-only) + the launch-check policy App/Blurt/ project.yml XcodeGen source of truth — Blurt.xcodeproj is GENERATED @@ -159,26 +158,27 @@ In Claude Code on the web, a `SessionStart` hook installs the portable linters a Each was tried the other way and reverted. If a task seems to require one, stop and ask first. (`.claude/skills/project-guardrails` is the compressed version of this list.) -| Don't | Because | -| --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Add an external SPM dependency to the engine | Dependency-free by rule (biggest supply-chain risk); a `check.sh` guard fails on `.package(` in `Package.swift` or a `url:`/`github:` package in `project.yml`. Extend `BlurtEngine` instead. | -| Use `AVAudioEngine` / `installTap` for capture | A long-lived engine bound its input graph to one device and went stale on a mic↔built-in switch — `-10868` (`kAudioUnitErr_FormatNotSupported`) or all-zero buffers. `MicCapture` uses a fresh `AVAudioRecorder` per session. | -| Add streaming STT | The dictation API returns the full (already rewritten) text in one response; the overlay shows "Transcribing…" then the full text. | -| Add a client-side LLM cleanup pass | Cleanup is the dictation API's server-side rewrite, requested by the `llm` block on the same `/transcribe` call. No LLM Gateway client, no `StylerProtocol`, no styling stage, no second request — request steering belongs in `TranscriptionSteering`. | -| Add local models or model downloads | Transcription is a remote AssemblyAI call: no on-device ASR/LLM, no model cache, no download UI. | -| Send `config.prompt` at all | The field takes a _description of the audio_, not instructions, and a custom value replaces the service's managed default **including its language steering**. Formatting goes in `llm.instruction`, vocabulary in `keyterms_prompt`, preceding text in `conversation_context`. | -| Pin the prompt to English | Hurt non-English transcription; language is left to the model's own detection — and a custom prompt is what drops the managed default's language steering in the first place. | -| Put formatting instructions in `config.prompt` | Reshaping output is not something the STT prompt acts on, so _"Transcribe speech into markdown."_ was a measured no-op. `llm.instruction` (_"Format the result as markdown."_) is the lever that works. | -| Pack key terms into the prompt as `Keywords: a, b, c.` | The API documents `keyterms_prompt` for exactly this, and warns against packing keyword lists into the prompt. | -| Add a "remove filler words (um, uh, like)" clause | Not something the STT prompt acts on — a no-op, deliberately dropped; disfluency removal is the server-side LLM rewrite's job. | -| Add a keystroke-typing paste path or a length threshold | Injection is **always** clipboard paste (save → write → ⌘V → settle → restore), with the copied-to-clipboard degradation when the target is lost. | -| Add `LSUIElement` or a menu-bar-**only** mode | Blurt is a Dock app first. The `MenuBarExtra` status item is convenience layered on the Dock icon; the notch can hide a status item, so nothing may depend on it. A menu-bar-only variant was reverted twice. | -| Add a `KeyboardShortcuts` package or a key+modifier chord | The trigger is a single lone modifier, home-grown (`CGEventTap` + `DictationKeyGate`), and swallows nothing. | -| Add a self-replacing install or background auto-updater | Updates are download-only; `mxcl/AppUpdater` and its in-place updater were removed. The once-a-day launch _check_ (`AutomaticUpdateCheck`) is fine; installing for the user, or polling, is not. Extend `UpdateCheckModel`. | -| Hand-edit `Blurt.xcodeproj/project.pbxproj` | Generated from `project.yml`; `check.sh`'s drift check fails on any manual edit (a Claude PreToolUse hook also blocks it). | -| Redirect the post-build install away from `/Applications` | TCC won't register apps in DerivedData/`/tmp`, so permission toggles never appear. | -| Touch the real Keychain in tests | `APIKeyStore` is the production item — a test that writes it triggers Keychain prompts and corrupts the real item's ACL. Use an isolated service (see `KeychainStoreTests`) or `InMemoryAPIKeyStore`. | -| Add backwards-compat shims for removed types | Deleted types stay deleted — no deprecated re-exports. | +| Don't | Because | +| --------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Add an external SPM dependency to the engine | Dependency-free by rule (biggest supply-chain risk); a `check.sh` guard fails on `.package(` in `Package.swift` or a `url:`/`github:` package in `project.yml`. Extend `BlurtEngine` instead. | +| Use `AVAudioEngine` / `installTap` for capture | A long-lived engine bound its input graph to one device and went stale on a mic↔built-in switch — `-10868` (`kAudioUnitErr_FormatNotSupported`) or all-zero buffers. `MicCapture` uses a fresh `AVAudioRecorder` per session. | +| Add streaming STT | The dictation API returns the full (already rewritten) text in one response; the overlay shows "Transcribing…" then the full text. | +| Add a client-side LLM cleanup pass | Cleanup is the dictation API's server-side rewrite, requested by the `llm` block on the same `/transcribe` call. No LLM Gateway client, no `StylerProtocol`, no styling stage, no second request — request steering belongs in `TranscriptionSteering`. | +| Add local models or model downloads | Transcription is a remote AssemblyAI call: no on-device ASR/LLM, no model cache, no download UI. | +| Send `config.prompt` at all | The field takes a _description of the audio_, not instructions, and a custom value replaces the service's managed default **including its language steering**. Vocabulary goes in `keyterms_prompt`, preceding text in `conversation_context`. | +| Pin the prompt to English | Hurt non-English transcription; language is left to the model's own detection — and a custom prompt is what drops the managed default's language steering in the first place. | +| Put formatting instructions in `config.prompt` | Reshaping output is not something the STT prompt acts on, so _"Transcribe speech into markdown."_ was a measured no-op. | +| Send anything about the frontmost app to the service | Removed on purpose. `AppKindPriming` recognized the app's bundle ID as a kind and sent a formatting clause as `llm.instruction`; the bundle ID is no longer captured, and `llm` always goes out empty so the service's default cleanup applies everywhere. | +| Pack key terms into the prompt as `Keywords: a, b, c.` | The API documents `keyterms_prompt` for exactly this, and warns against packing keyword lists into the prompt. | +| Add a "remove filler words (um, uh, like)" clause | Not something the STT prompt acts on — a no-op, deliberately dropped; disfluency removal is the server-side LLM rewrite's job. | +| Add a keystroke-typing paste path or a length threshold | Injection is **always** clipboard paste (save → write → ⌘V → settle → restore), with the copied-to-clipboard degradation when the target is lost. | +| Add `LSUIElement` or a menu-bar-**only** mode | Blurt is a Dock app first. The `MenuBarExtra` status item is convenience layered on the Dock icon; the notch can hide a status item, so nothing may depend on it. A menu-bar-only variant was reverted twice. | +| Add a `KeyboardShortcuts` package or a key+modifier chord | The trigger is a single lone modifier, home-grown (`CGEventTap` + `DictationKeyGate`), and swallows nothing. | +| Add a self-replacing install or background auto-updater | Updates are download-only; `mxcl/AppUpdater` and its in-place updater were removed. The once-a-day launch _check_ (`AutomaticUpdateCheck`) is fine; installing for the user, or polling, is not. Extend `UpdateCheckModel`. | +| Hand-edit `Blurt.xcodeproj/project.pbxproj` | Generated from `project.yml`; `check.sh`'s drift check fails on any manual edit (a Claude PreToolUse hook also blocks it). | +| Redirect the post-build install away from `/Applications` | TCC won't register apps in DerivedData/`/tmp`, so permission toggles never appear. | +| Touch the real Keychain in tests | `APIKeyStore` is the production item — a test that writes it triggers Keychain prompts and corrupts the real item's ACL. Use an isolated service (see `KeychainStoreTests`) or `InMemoryAPIKeyStore`. | +| Add backwards-compat shims for removed types | Deleted types stay deleted — no deprecated re-exports. | Release-side invariants (hardened runtime and a secure timestamp on every nested mach-o and embedded framework, or notarization rejects the build; roll-forward-only for a bad release) live in @@ -250,8 +250,8 @@ fields built per utterance by `TranscriptionSteering`: `conversation_context`, ` the `llm` block). No model header — the service pins the STT model server-side. **`prompt` is never sent** — see [Transcription steering](#transcription-steering). The `llm` block asks the service to run a cleanup rewrite (remove disfluencies, fix punctuation) over the verbatim transcript, all inside -the same request; its optional `instruction` is the app-kind formatting clause, and an empty block -selects the service's own default cleanup. The response carries both `text` (verbatim) and +the same request; it always goes out empty, which selects the service's own default cleanup +instruction. The response carries both `text` (verbatim) and `llm_response` (the rewrite); the transcriber returns the rewrite and falls back to `text` when `llm_response` is null — the rewrite is best-effort (5 s server-side budget), so a rewrite failure (`llm_error`) is a logged degradation, never a user-facing error. @@ -380,37 +380,40 @@ to right ⌘), so views must not re-declare `TriggerKey.rightCommand.rawValue` t ## Transcription steering -`Sources/BlurtEngine/STT/TranscriptionSteering.swift` renders the captured context into the three +`Sources/BlurtEngine/STT/TranscriptionSteering.swift` renders the captured context into the two request-customization fields, each with one job. It's unit-tested in `Tests/BlurtEngineTests/TranscriptionSteeringTests.swift`. -| Field | Carries | Cap | -| ---------------------- | ----------------------------------------------------------------- | ----------------------- | -| `conversation_context` | prior-cursor text, as a single turn | 4096 chars, clip head | -| `keyterms_prompt` | the user's key terms, verbatim | 2048 chars, whole terms | -| `llm.instruction` | the `AppKindPriming` formatting clause, or absent for the default | — | +| Field | Carries | Cap | +| ---------------------- | ----------------------------------- | ----------------------- | +| `conversation_context` | prior-cursor text, as a single turn | 4096 chars, clip head | +| `keyterms_prompt` | the user's key terms, verbatim | 2048 chars, whole terms | **`config.prompt` is never sent, and that is the whole point of this design.** The field takes a _description of the audio_ ("Cardiology consultation about chest pain symptoms."), not instructions — -transcription behavior is optimized out of the box — so the app-kind priming's old imperative form -(_"Transcribe speech into markdown."_) was aimed at a field that doesn't act on instructions and was a -no-op. Formatting now rides in `llm.instruction`, where an LLM actually rewrites the text -(_"Format the result as markdown."_ — for a code editor, the language is inferred from the window -title's filename: _"Format the result as Swift code."_), and vocabulary rides in `keyterms_prompt` -rather than being packed into the prompt as a `Keywords: a, b, c.` clause. Sending no prompt also -keeps the service's managed default, which a custom prompt replaces wholesale — **including its -language steering**, which is the mechanism behind the older finding that pinning the prompt to -English hurt non-English speech. - -The remaining focus context is **not** sent: app and field names render nowhere (real-world logs -showed them crowding the request — VS Code, for one, parks a screen-reader help announcement in the -focused field's description), the window title is read only to name a code editor's language, and -selected text is never priming because the paste replaces it. All of it is still captured — the -injector's separator logic consumes the prior text and window title — and reading stays -privacy-guarded: prior and selected text are skipped in secure fields, detected by AX role **or** -subrole and failing closed when the role can't be read, so a password is never read out of the field -at all. That guard, not the steering builder, is what keeps a password out of `conversation_context` -and off the dictation log. +transcription behavior is optimized out of the box — so an imperative like +_"Transcribe speech into markdown."_ was aimed at a field that doesn't act on instructions and was a +no-op. Vocabulary rides in `keyterms_prompt` rather than being packed into the prompt as a +`Keywords: a, b, c.` clause. Sending no prompt also keeps the service's managed default, which a +custom prompt replaces wholesale — **including its language steering**, which is the mechanism behind +the older finding that pinning the prompt to English hurt non-English speech. + +**Nothing describing the destination app is sent.** An earlier design recognized the frontmost app's +bundle ID as a _kind_ (terminal, code editor, Slack, Obsidian) and sent a matching formatting clause +as `llm.instruction`; that whole path — `AppKindPriming`, `TranscriptionContext.bundleID`, the +`CapturedFocus` bundle-ID capture, and the log's `llm_instruction` column — was removed. The `llm` +block now always goes out empty, so the service's own default cleanup rewrite runs for every +utterance regardless of where the text is going. + +The remaining focus context is **not** sent either: app and field names render nowhere (real-world +logs showed them crowding the request — VS Code, for one, parks a screen-reader help announcement in +the focused field's description), the window title is read only to anchor the injector's paste +separator, and selected text is never priming because the paste replaces it. All of it is still +captured — the injector's separator logic consumes the prior text and window title — and reading +stays privacy-guarded: prior and selected text are skipped in secure fields, detected by AX role +**or** subrole and failing closed when the role can't be read, so a password is never read out of the +field at all. That guard, not the steering builder, is what keeps a password out of +`conversation_context` and off the dictation log. `build(context:)` returns `.empty` when there's no usable context (or when nothing renders — an unrecognized app with no prior text and no key terms), and every empty field is **omitted** from the @@ -448,8 +451,7 @@ resolves. History: **`RecentDictations`** is an in-memory, newest-first ring shown in the ready window (never written to disk). **`DictationLog`** appends each completed dictation — the transcript plus the exact -steering fields sent (`conversation_context`, `keyterms_prompt`, `llm_instruction`, under the wire's -own names) — to +steering fields sent (`conversation_context` and `keyterms_prompt`, under the wire's own names) — to `~/Library/Logs/Blurt/dictations.jsonl` (`DictationLog.defaultURL`, or `defaultDisplayPath` for the home-abbreviated form to show in UI — derived next to the URL so the label can't drift from the write target) — but **only** while developer mode is on; with it off, nothing is written. The Settings diff --git a/BLURTENGINE.md b/BLURTENGINE.md index dbb2ea5..26d39e1 100644 --- a/BLURTENGINE.md +++ b/BLURTENGINE.md @@ -59,7 +59,7 @@ press() ──▶ MicCapture.start() release() ──▶ MicCapture.s Key properties of the design, which your integration can rely on: - **One request per utterance, no streaming.** The dictation API returns the complete transcript — and its LLM-rewritten form — in the response body: no upload step, no job polling, no incremental deltas, no second request for the cleanup. `TranscriberProtocol.transcribe` is a single `async throws -> String`. UIs should show a "transcribing…" state and then the whole result; there is nothing to stream. -- **Cleanup happens server-side.** The request's `llm` block asks the service for a cleanup rewrite (remove disfluencies, fix punctuation), applied to the verbatim transcript inside the same call; its optional `instruction` carries the app-kind formatting clause, and an empty block selects the service's own default cleanup. Recognition is primed separately by `conversation_context` and `keyterms_prompt` (built by `TranscriptionSteering` from the captured context); `config.prompt` is never sent. The engine pastes `llm_response`, falling back to the verbatim `text` when the best-effort rewrite failed (`llm_error`) — a degradation, never a user-facing error. There is no client-side LLM pass, no styling stage, and deliberately no hook for one. +- **Cleanup happens server-side.** The request's `llm` block asks the service for a cleanup rewrite (remove disfluencies, fix punctuation), applied to the verbatim transcript inside the same call; it always goes out empty, which selects the service's own default cleanup instruction. Recognition is primed separately by `conversation_context` and `keyterms_prompt` (built by `TranscriptionSteering` from the captured context); `config.prompt` is never sent, and neither is anything describing the destination app. The engine pastes `llm_response`, falling back to the verbatim `text` when the best-effort rewrite failed (`llm_error`) — a degradation, never a user-facing error. There is no client-side LLM pass, no styling stage, and deliberately no hook for one. - **Latency is pre-paid where possible.** `press()` fires a detached `warmUp()` at the transcriber (pre-opening the HTTPS connection while the user speaks, ~170 ms saved cold) and kicks off the cross-process accessibility read of the focused field without awaiting it — the read is then consumed at transcribe time with a bounded wait (`DictationSession.contextWaitBudget`, 500 ms), so an unresponsive frontmost app costs the transcript its priming, never a multi-second stall — and never delays the recording indicator. On the way out, `release()` flips the phase to `.transcribing` _before_ reading the recorded audio back, so a host's stop cue fires at key-up rather than after the disk read. - **A held trigger auto-releases.** `DictationSession` stops recording after `maxRecordingSeconds` (default `SyncSTTLimits.autoReleaseSeconds`, 115 s) so audio never exceeds what the endpoint accepts, and transcribes what it has. Clips shorter than `SyncSTTLimits.minPCMBytes` (~100 ms of audio — an accidental tap) are dropped as a silent no-op rather than sent to earn a 400. @@ -152,15 +152,15 @@ The session calls `setTargetApp` at press time with the app that was frontmost w Recognition quality comes from per-utterance priming, assembled automatically inside `press()` — hosts don't call these APIs directly, but should know what's collected: -- **`TranscriptionContext`** carries the frontmost app name and bundle identifier, window title, focused-field label, the text before the caret, the selected text (which a paste will replace), and the user's key terms. It's captured via Accessibility at press time (skipped in secure fields), off the hot path — and consumed at transcribe time with a bounded wait (`DictationSession.contextWaitBudget`), so a hung read is abandoned rather than stalling the transcript. Only the bundle ID, window title, and key terms reach the prompt; the prior text and window title also steer the injector's paste separator, and nothing else is consumed. -- **`TranscriptionSteering.build(context:)`** renders that into the request's three customization fields: the prior-cursor text as the single `conversation_context` turn (clipped to 4096 chars keeping the tail, since the words nearest the cursor carry the continuity), the user's key terms as `keyterms_prompt` (whole terms fitted to 2048 chars total), and — when the bundle ID identifies a recognized destination kind (`AppKindPriming` — terminals, code editors, Slack, Obsidian) — the formatting clause as `llm.instruction`, e.g. "Format the result as a shell command with no trailing period." or, for a code editor, the language inferred from the window title's filename ("Format the result as Swift code."). **`config.prompt` is never sent**: it takes a description of the audio rather than instructions, and a custom value replaces the service's managed default including its language steering, which is why the app-kind clause moved to the `llm` block and the key terms to their own field. Nothing else is rendered — no app/field names, no selected text (the paste replaces it), and no standing annotation-suppression clause ("Transcribe without speaker labels, …" is part of the service's own default). Empty fields are omitted rather than sent as `[]`. Two further deliberate omissions: no language directive and no "remove filler words" clause (not something the STT prompt acts on — a no-op). Don't reintroduce either. +- **`TranscriptionContext`** carries the frontmost app name, window title, focused-field label, the text before the caret, the selected text (which a paste will replace), and the user's key terms. It's captured via Accessibility at press time (skipped in secure fields), off the hot path — and consumed at transcribe time with a bounded wait (`DictationSession.contextWaitBudget`), so a hung read is abandoned rather than stalling the transcript. Only the prior text and key terms reach the request; the prior text and window title also steer the injector's paste separator, and nothing else is consumed. +- **`TranscriptionSteering.build(context:)`** renders that into the request's two customization fields: the prior-cursor text as the single `conversation_context` turn (clipped to 4096 chars keeping the tail, since the words nearest the cursor carry the continuity) and the user's key terms as `keyterms_prompt` (whole terms fitted to 2048 chars total). **`config.prompt` is never sent**: it takes a description of the audio rather than instructions, and a custom value replaces the service's managed default including its language steering, which is why the key terms ride in their own field. Nothing else is rendered — no app/field names, no selected text (the paste replaces it), and no standing annotation-suppression clause ("Transcribe without speaker labels, …" is part of the service's own default). Empty fields are omitted rather than sent as `[]`. Three further deliberate omissions: no language directive, no "remove filler words" clause (not something the STT prompt acts on — a no-op), and nothing identifying the destination app — an earlier `AppKindPriming` sent a bundle-ID-derived formatting clause as `llm.instruction`, and that whole path was removed. Don't reintroduce any of them. - **`KeyTermsStore`** persists the user's domain vocabulary (names, jargon) in `UserDefaults`; `DictationSession` re-reads it at every press via its `keyTermsProvider` closure, so Settings edits apply to the next utterance without rebuilding the session. Pass your own provider to source terms from elsewhere. For key storage, compose against **`APIKeyGateway`** — the injectable `current` / `save(_:)` / `hasKey` seam over the key store. `ProductionAPIKeyStore` forwards to the Keychain-backed `APIKeyStore`; `InMemoryAPIKeyStore` is a ready-made in-memory conformance for tests and harnesses (Blurt's XCUITest runs use it so the real Keychain item is never touched, and its `hasKey` backs the session's `readinessCheck`). For a settings UI, **`APIKeySubmission`** wraps the gateway with the validate-then-save flow (`submit(_:)` → valid / invalid / unreachable / saveFailed, via `APIKeyValidator`): it saves only a key AssemblyAI actively accepts, so an unverified key never persists. Two projections keep the surrounding UI out of your views: `Outcome.failureReport` classifies a failure as `.inline(message:)` (recoverable — show it beside the field) or `.alert(title:message:)` (a Keychain fault retyping can't fix), and **`APIKeyDisplay.resolve(key:)`** renders the stored key for an account row — masked tail, status and VoiceOver wording, and the connect-vs-rotate control titles. The mask reveals only the last `revealedTailLength` characters and, below `minimumLengthToMask`, none at all, so a short key can't be shown whole. Setup gating has a projection too: **`SetupReadiness.isReady(permissions:hasAPIKey:)`** is the "fully configured" rule (deliberately excluding the trigger key, which has a default), `SetupReadiness.pollInterval(isReady:)` is the permission-poll cadence (brisk during setup, coasting once ready), and `PermissionStatus.lostGrant(since:)` detects a permission revoked out from under a configured app. -Each completed dictation is appended to **`DictationLog`** (a local JSONL history at `~/Library/Logs/Blurt/dictations.jsonl` — `DictationLog.defaultURL`, or `defaultDisplayPath` for the home-abbreviated form to show in UI) — each entry is the transcript plus the exact steering fields sent (`conversation_context`, `keyterms_prompt`, `llm_instruction`), never the context that wasn't sent — but only while developer mode is switched on. **`DeveloperModeStore`** persists that opt-in in `UserDefaults` (`BlurtDeveloperMode`, off by default); with it off, nothing is written to disk. Blurt surfaces the switch (and the log path) in the Settings window's Developer section. +Each completed dictation is appended to **`DictationLog`** (a local JSONL history at `~/Library/Logs/Blurt/dictations.jsonl` — `DictationLog.defaultURL`, or `defaultDisplayPath` for the home-abbreviated form to show in UI) — each entry is the transcript plus the exact steering fields sent (`conversation_context`, `keyterms_prompt`), never the context that wasn't sent — but only while developer mode is switched on. **`DeveloperModeStore`** persists that opt-in in `UserDefaults` (`BlurtDeveloperMode`, off by default); with it off, nothing is written to disk. Blurt surfaces the switch (and the log path) in the Settings window's Developer section. ## Hotkey building blocks diff --git a/README.md b/README.md index 9dd68c0..f229f0b 100644 --- a/README.md +++ b/README.md @@ -166,7 +166,7 @@ Sources/BlurtEngine/ Swift 6 package owning the pipeline — no external dep Audio/ MicCapture: fresh AVAudioRecorder per session, 16 kHz mono PCM, live level meter; DX7/Juno-106 sound packs STT/ AssemblyAITranscriber: one POST to dictation.assemblyai.com/transcribe - (STT + LLM rewrite) + TranscriptionSteering context/keyterms/format + (STT + LLM rewrite) + TranscriptionSteering context/keyterms Pipeline/ DictationSession actor: press/release/cancel commands, phase stream, auto-release before the API's recording cap Hotkey/ DictationKeyGate/Router: pure, unit-tested state machine for the diff --git a/Sources/BlurtEngine/FocusCapture/FocusCapture.swift b/Sources/BlurtEngine/FocusCapture/FocusCapture.swift index 21005ac..4724e42 100644 --- a/Sources/BlurtEngine/FocusCapture/FocusCapture.swift +++ b/Sources/BlurtEngine/FocusCapture/FocusCapture.swift @@ -4,20 +4,13 @@ import ApplicationServices struct CapturedFocus: Sendable { let pid: pid_t let processName: String? - /// The frontmost app's stable identity, feeding the prompt's app-kind - /// recognition (`AppKindPriming`) via `TranscriptionContext.bundleID`. - let bundleID: String? } enum FocusCapture { @MainActor static func captureFrontmost() -> CapturedFocus? { guard let app = NSWorkspace.shared.frontmostApplication else { return nil } - return CapturedFocus( - pid: app.processIdentifier, - processName: app.localizedName, - bundleID: app.bundleIdentifier - ) + return CapturedFocus(pid: app.processIdentifier, processName: app.localizedName) } static func runningApp(for captured: CapturedFocus) -> NSRunningApplication? { diff --git a/Sources/BlurtEngine/Pipeline/DictationLog.swift b/Sources/BlurtEngine/Pipeline/DictationLog.swift index 1974874..8441771 100644 --- a/Sources/BlurtEngine/Pipeline/DictationLog.swift +++ b/Sources/BlurtEngine/Pipeline/DictationLog.swift @@ -28,14 +28,12 @@ public enum DictationLog { /// an entry directly from a context. let conversationContext: [String] let keytermsPrompt: [String] - let llmInstruction: String? enum CodingKeys: String, CodingKey { case transcript case ts case conversationContext = "conversation_context" case keytermsPrompt = "keyterms_prompt" - case llmInstruction = "llm_instruction" } /// Mirrors `DictationConfig.encode(to:)`: an empty array omits its field, so @@ -50,7 +48,6 @@ public enum DictationLog { if !keytermsPrompt.isEmpty { try container.encode(keytermsPrompt, forKey: .keytermsPrompt) } - try container.encodeIfPresent(llmInstruction, forKey: .llmInstruction) } } @@ -118,8 +115,7 @@ public enum DictationLog { let entry = Entry( transcript: transcript, ts: now.formatted(timestampFormat), conversationContext: steering.conversationContext, - keytermsPrompt: steering.keyterms, - llmInstruction: steering.rewriteInstruction) + keytermsPrompt: steering.keyterms) guard var line = try? makeEncoder().encode(entry) else { return } line.append(0x0A) // '\n' diff --git a/Sources/BlurtEngine/Pipeline/DictationSession.swift b/Sources/BlurtEngine/Pipeline/DictationSession.swift index c5ed388..b3f278a 100644 --- a/Sources/BlurtEngine/Pipeline/DictationSession.swift +++ b/Sources/BlurtEngine/Pipeline/DictationSession.swift @@ -215,7 +215,6 @@ public actor DictationSession { let field = FocusCapture.captureFieldContext() let context = TranscriptionContext( appName: captured?.processName, - bundleID: captured?.bundleID, windowTitle: field.windowTitle, fieldLabel: field.fieldLabel, priorText: field.priorText, diff --git a/Sources/BlurtEngine/STT/AppKindPriming.swift b/Sources/BlurtEngine/STT/AppKindPriming.swift deleted file mode 100644 index 3620173..0000000 --- a/Sources/BlurtEngine/STT/AppKindPriming.swift +++ /dev/null @@ -1,151 +0,0 @@ -import Foundation - -/// The app-kind formatting instruction: recognizes what *kind* of app the -/// dictation targets (a terminal, a code editor, Slack, Obsidian) from the -/// frontmost app's bundle identifier and renders the one sentence -/// `TranscriptionSteering` sends as `config.llm.instruction` — "Format the -/// result as a shell command with no trailing period." / "… as Swift code." / -/// "… as a casual Slack message, using Slack emoji where they fit." / "… as -/// markdown." — telling the rewrite what shape of text the destination expects, -/// which the app's display name alone doesn't convey. -/// -/// For code editors the window title usually names the open file, so the -/// clause names the language inferred from that filename's extension -/// ("Format the result as Python code.") when one is recognizable, and stays -/// generic ("… as code.") otherwise. -/// -/// These are instructions to the **LLM that rewrites the finished transcript**, -/// not to the STT model. Reshaping output is not something `config.prompt` acts -/// on — that field takes a description of the audio — so a clause phrased as -/// "Transcribe speech into markdown." was a no-op wherever it landed. Keep the -/// imperative "Format the result as …" shape. -/// -/// Detection keys on bundle IDs, not display names: names are localized and -/// user-editable, while the bundle ID is the app's stable identity. An -/// unrecognized app contributes no clause — the request's `llm` block is then -/// empty and the service's own default cleanup instruction applies. Exercised by -/// `Tests/BlurtEngineTests/AppKindPrimingTests.swift`. -enum AppKindPriming { - /// The recognized destination families. Each renders one guidance sentence; - /// anything else contributes no clause. - enum Kind: Sendable, Equatable { - case terminal - case codeEditor - case slack - case obsidian - } - - /// Exact bundle-ID → kind matches for the recognized apps. - private static let kindsByBundleID: [String: Kind] = [ - // Terminals. - "com.apple.Terminal": .terminal, - "com.googlecode.iterm2": .terminal, - "dev.warp.Warp-Stable": .terminal, - "dev.warp.Warp-Preview": .terminal, - "com.mitchellh.ghostty": .terminal, - "net.kovidgoyal.kitty": .terminal, - "org.alacritty": .terminal, - "com.github.wez.wezterm": .terminal, - "co.zeit.hyper": .terminal, - // Code editors. Cursor ships under an opaque ToDesktop build id. - "com.microsoft.VSCode": .codeEditor, - "com.microsoft.VSCodeInsiders": .codeEditor, - "com.vscodium": .codeEditor, - "com.todesktop.230313mzl4w4u92": .codeEditor, - "com.exafunction.windsurf": .codeEditor, - "com.apple.dt.Xcode": .codeEditor, - "dev.zed.Zed": .codeEditor, - "dev.zed.Zed-Preview": .codeEditor, - "com.panic.Nova": .codeEditor, - "com.macromates.TextMate": .codeEditor, - // Chat and notes. - "com.tinyspeck.slackmacgap": .slack, - "md.obsidian": .obsidian, - ] - - /// Prefix matches for app families that ship many bundle IDs under one - /// vendor prefix (every JetBrains IDE, Sublime Text's versioned IDs). - private static let kindsByBundleIDPrefix: [(prefix: String, kind: Kind)] = [ - ("com.jetbrains.", .codeEditor), - ("com.sublimetext.", .codeEditor), - ] - - /// The kind `bundleID` identifies, or `nil` for an unrecognized (or absent) - /// bundle ID. - static func kind(ofBundleID bundleID: String?) -> Kind? { - guard let bundleID = bundleID.trimmedNonEmpty() else { return nil } - if let kind = kindsByBundleID[bundleID] { return kind } - return kindsByBundleIDPrefix.first { bundleID.hasPrefix($0.prefix) }?.kind - } - - /// The formatting instruction for the app `bundleID` identifies, or `nil` when - /// the app isn't recognized. `windowTitle` refines the code-editor clause - /// with the open file's language; the other kinds ignore it. - static func clause(bundleID: String?, windowTitle: String?) -> String? { - guard let kind = kind(ofBundleID: bundleID) else { return nil } - switch kind { - case .terminal: - // "Trailing period", not "terminal punctuation": in a clause about - // terminals the latter reads as the app, not the end of a sentence. - return "Format the result as a shell command with no trailing period." - case .codeEditor: - let subject = windowTitle.flatMap(language(inWindowTitle:)) ?? "code" - return "Format the result as \(subject)." - case .slack: - return "Format the result as a casual Slack message, using Slack emoji where they fit." - case .obsidian: - return "Format the result as markdown." - } - } - - /// What the open file says speech becomes, from the first token in `title` - /// that reads as a filename with a recognized extension - /// ("● main.py — blurt — Visual Studio Code" → "Python code"), or `nil` when - /// no token does. Editors lead their window titles with the open file, so - /// first match wins. - static func language(inWindowTitle title: String) -> String? { - for token in title.split(whereSeparator: \.isWhitespace) { - let name = token.trimmingCharacters(in: Self.filenameTrim) - // A leading-dot name (".zshrc") is a dotfile, not a base name + extension. - guard let dot = name.lastIndex(of: "."), dot != name.startIndex else { continue } - if let language = languagesByExtension[name[name.index(after: dot)...].lowercased()] { - return language - } - } - return nil - } - - /// Decoration editors wrap around the filename in a window title — dirty - /// markers, quotes, brackets, dash separators. - private static let filenameTrim = CharacterSet(charactersIn: "\"'`•●◆*()[]{}<>,;:—–-") - - /// Filename extension → what the clause says the result should be. Values - /// complete "Format the result as …", so languages carry a trailing "code" - /// while markup/data formats stand alone. Lowercased keys; lookups lowercase - /// the extension first. - private static let languagesByExtension: [String: String] = [ - "c": "C code", "h": "C code", - "cc": "C++ code", "cpp": "C++ code", "cxx": "C++ code", "hpp": "C++ code", - "cs": "C# code", - "css": "CSS", "scss": "CSS", - "go": "Go code", - "htm": "HTML", "html": "HTML", - "java": "Java code", - "cjs": "JavaScript code", "js": "JavaScript code", "jsx": "JavaScript code", "mjs": "JavaScript code", - "json": "JSON", - "kt": "Kotlin code", "kts": "Kotlin code", - "lua": "Lua code", - "m": "Objective-C code", "mm": "Objective-C code", - "markdown": "markdown", "md": "markdown", - "php": "PHP code", - "py": "Python code", "pyi": "Python code", - "rb": "Ruby code", - "rs": "Rust code", - "bash": "a shell script", "sh": "a shell script", "zsh": "a shell script", - "sql": "SQL", - "swift": "Swift code", - "toml": "TOML", - "ts": "TypeScript code", "tsx": "TypeScript code", - "yaml": "YAML", "yml": "YAML", - ] -} diff --git a/Sources/BlurtEngine/STT/AssemblyAITranscriber.swift b/Sources/BlurtEngine/STT/AssemblyAITranscriber.swift index 6071e43..5fb1faf 100644 --- a/Sources/BlurtEngine/STT/AssemblyAITranscriber.swift +++ b/Sources/BlurtEngine/STT/AssemblyAITranscriber.swift @@ -106,12 +106,11 @@ public struct AssemblyAITranscriber: TranscriberProtocol { } /// Builds the JSON `config` part sent alongside the audio. Each steering field - /// is included only when it carries something — an empty array or a nil - /// instruction omits the field rather than stating an empty value, so the - /// service applies its own default. The `llm` block always rides along — see - /// `DictationConfig.llm`. Internal so tests can assert the steering wiring - /// without inspecting the multipart upload body (which `URLProtocol` mocks - /// can't observe reliably for `upload(from:)`). + /// is included only when it carries something — an empty array omits the field + /// rather than stating an empty value, so the service applies its own default. + /// The `llm` block always rides along — see `DictationConfig.llm`. Internal so + /// tests can assert the steering wiring without inspecting the multipart upload + /// body (which `URLProtocol` mocks can't observe reliably for `upload(from:)`). func makeConfigData(sampleRate: Int, steering: TranscriptionSteering.Fields) throws -> Data { try JSONEncoder().encode( DictationConfig( @@ -119,7 +118,7 @@ public struct AssemblyAITranscriber: TranscriberProtocol { channels: 1, conversationContext: steering.conversationContext, keytermsPrompt: steering.keyterms, - llm: LLMRewrite(instruction: steering.rewriteInstruction) + llm: LLMRewrite() ) ) } @@ -212,10 +211,8 @@ public struct AssemblyAITranscriber: TranscriberProtocol { /// The user's key terms as the explicit vocabulary list, omitted when empty. let keytermsPrompt: [String] /// The rewrite request. Always present so the service runs the rewrite at - /// all; its `instruction` is the app-kind formatting clause, or absent to - /// select the service's default cleanup instruction. Per the API's - /// `instruction`-mode rules, output format and don't-answer-the-text - /// safeguards are enforced server-side either way. + /// all, and always empty so the instruction it runs is the service's own + /// default cleanup — Blurt sends nothing describing the destination app. let llm: LLMRewrite enum CodingKeys: String, CodingKey { case sampleRate = "sample_rate" @@ -244,9 +241,11 @@ public struct AssemblyAITranscriber: TranscriberProtocol { } } - private struct LLMRewrite: Encodable { - let instruction: String? - } + /// Encodes as `{}`: asking for the rewrite without steering it. An + /// `instruction` here once carried a formatting clause derived from the + /// frontmost app's bundle ID; that was removed, and the service's default + /// cleanup instruction is what runs now. + private struct LLMRewrite: Encodable {} private struct DictationResponse: Decodable { /// The verbatim transcript — always present, never altered by the LLM. diff --git a/Sources/BlurtEngine/STT/TranscriptionContext.swift b/Sources/BlurtEngine/STT/TranscriptionContext.swift index 7e637b5..331ec29 100644 --- a/Sources/BlurtEngine/STT/TranscriptionContext.swift +++ b/Sources/BlurtEngine/STT/TranscriptionContext.swift @@ -1,11 +1,10 @@ /// Per-utterance snapshot of where the dictation is going, gathered at /// dictation start from the focused app and field. `TranscriptionSteering.build` /// renders only the parts the request uses — the prior text (as -/// `conversation_context`), the key terms (as `keyterms_prompt`), the bundle ID -/// (via the `AppKindPriming` formatting clause in `llm.instruction`), and the -/// window title (the language refinement of that clause). The prior text and -/// window title also steer the injector's paste separator; nothing else is -/// consumed, and the rest of the context is neither sent nor logged. +/// `conversation_context`) and the key terms (as `keyterms_prompt`). Nothing +/// identifying the destination app is sent. The prior text and window title +/// also steer the injector's paste separator; nothing else is consumed, and the +/// rest of the context is neither sent nor logged. /// /// Every focus field is optional and best-effort: whatever couldn't be read is /// `nil`, and an entirely empty context customizes nothing (the server applies @@ -15,19 +14,9 @@ public struct TranscriptionContext: Sendable, Equatable { /// sent. public let appName: String? - /// The frontmost application's bundle identifier (e.g. - /// "com.tinyspeck.slackmacgap"). Never sent verbatim: it keys the - /// app-*kind* recognition (`AppKindPriming`) that selects the rewrite's - /// formatting instruction — terminal, code editor, Slack, Obsidian. - /// Preferred over `appName` for recognition because display names are - /// localized and user-editable while the bundle ID is stable. - public let bundleID: String? - /// The focused window's title (e.g. "main.py — blurt", a document name, a - /// Slack channel). Never sent verbatim. In a code editor it usually names the - /// open file, which is how the formatting clause learns the language ("… as - /// Swift code."); it also anchors the injector's same-window separator - /// fallback. + /// Slack channel). Never sent. Read only to anchor the injector's same-window + /// separator fallback. public let windowTitle: String? /// A short label for the focused field (placeholder/title/role, e.g. "To", @@ -54,7 +43,6 @@ public struct TranscriptionContext: Sendable, Equatable { public init( appName: String?, - bundleID: String? = nil, windowTitle: String? = nil, fieldLabel: String? = nil, priorText: String?, @@ -62,7 +50,6 @@ public struct TranscriptionContext: Sendable, Equatable { keyTerms: [String] = [] ) { self.appName = appName - self.bundleID = bundleID self.windowTitle = windowTitle self.fieldLabel = fieldLabel self.priorText = priorText @@ -74,7 +61,7 @@ public struct TranscriptionContext: Sendable, Equatable { /// are no key terms. public var isEmpty: Bool { keyTerms.isEmpty - && [appName, bundleID, windowTitle, fieldLabel, priorText, selectedText].allSatisfy { + && [appName, windowTitle, fieldLabel, priorText, selectedText].allSatisfy { $0.trimmedNonEmpty() == nil } } diff --git a/Sources/BlurtEngine/STT/TranscriptionSteering.swift b/Sources/BlurtEngine/STT/TranscriptionSteering.swift index 92fed52..f809f14 100644 --- a/Sources/BlurtEngine/STT/TranscriptionSteering.swift +++ b/Sources/BlurtEngine/STT/TranscriptionSteering.swift @@ -1,4 +1,4 @@ -/// Renders a `TranscriptionContext` into the three request-customization fields +/// Renders a `TranscriptionContext` into the two request-customization fields /// the dictation API accepts, each of which has one job (see /// `AssemblyAITranscriber` for the wire encoding): /// @@ -8,22 +8,24 @@ /// proper-noun consistency. /// - `keyterms_prompt` — the user's key terms, verbatim, as the explicit /// vocabulary list the field is for. -/// - `llm.instruction` — the app-kind formatting clause (`AppKindPriming`), -/// recognized from the frontmost app's bundle ID: markdown in Obsidian, Swift -/// code in a code editor (the language inferred from the window title's -/// filename), a shell command in a terminal, a casual message in Slack. /// /// **`config.prompt` is deliberately never sent.** The field takes a /// *description of the audio* ("Cardiology consultation about chest pain /// symptoms."), not instructions — transcription behavior is optimized out of /// the box, so an imperative like "Transcribe speech into markdown." was aimed -/// at a field that does not act on instructions. That is why the app-kind -/// priming now rides in `llm.instruction`, where an LLM actually rewrites the -/// text, and why the key terms moved to `keyterms_prompt` rather than being -/// packed into the prompt as a `Keywords:` clause. Sending no prompt also keeps -/// the service's managed default — a custom prompt replaces it wholesale, -/// including its language steering, which is the mechanism behind the earlier -/// finding that pinning the prompt to English hurt non-English speech. +/// at a field that does not act on instructions. That is also why the key terms +/// ride in `keyterms_prompt` rather than being packed into the prompt as a +/// `Keywords:` clause. Sending no prompt keeps the service's managed default — +/// a custom prompt replaces it wholesale, including its language steering, +/// which is the mechanism behind the earlier finding that pinning the prompt to +/// English hurt non-English speech. +/// +/// **Nothing describing the destination app is sent either.** An earlier +/// version recognized the frontmost app's bundle ID as a *kind* (terminal, code +/// editor, Slack, Obsidian) and sent a matching formatting clause as +/// `llm.instruction`. That is gone: the `llm` block now carries no instruction, +/// so the service's own default cleanup rewrite applies to every utterance +/// regardless of where the text is going. /// /// Blurt has no earlier turns to send: `conversation_context` carries exactly /// one entry, the prior-cursor text, or none. @@ -31,8 +33,8 @@ /// The rest of the captured context is not sent at all. The app name and field /// label render nowhere (real-world logs showed them crowding the request — VS /// Code parks a screen-reader help announcement in the focused field's -/// description), and the window title is read only to name a code editor's -/// language. Selected text is never priming: the paste replaces it, so +/// description), and the window title is read only to anchor the injector's +/// paste separator. Selected text is never priming: the paste replaces it, so /// conditioning the model on it would prime for text on its way out. /// /// Exercised by `Tests/BlurtEngineTests/TranscriptionSteeringTests.swift`. @@ -47,16 +49,13 @@ enum TranscriptionSteering { /// Explicit vocabulary to bias recognition toward, in the user's own /// spelling and capitalization. let keyterms: [String] - /// The rewrite instruction, or `nil` to let the service's default cleanup - /// instruction stand. - let rewriteInstruction: String? /// Nothing to customize — every field omitted, so the service applies its /// managed default prompt and its default cleanup rewrite. Also the value /// to compare against for "does this utterance customize anything?" — /// `Fields` is `Equatable`, so no separate emptiness predicate exists to /// drift from the fields themselves. - static let empty = Fields(conversationContext: [], keyterms: [], rewriteInstruction: nil) + static let empty = Fields(conversationContext: [], keyterms: []) } /// Cap the dictation API documents for `conversation_context`: 4096 characters @@ -84,9 +83,7 @@ enum TranscriptionSteering { guard let context, !context.isEmpty else { return .empty } return Fields( conversationContext: priorTurn(of: context).map { [$0] } ?? [], - keyterms: fittedKeyterms(context.keyTerms), - rewriteInstruction: AppKindPriming.clause( - bundleID: context.bundleID, windowTitle: context.windowTitle.trimmedNonEmpty())) + keyterms: fittedKeyterms(context.keyTerms)) } /// The prior-cursor text as one conversation turn, clipped to the field's cap diff --git a/Tests/BlurtEngineTests/AppKindPrimingTests.swift b/Tests/BlurtEngineTests/AppKindPrimingTests.swift deleted file mode 100644 index e491758..0000000 --- a/Tests/BlurtEngineTests/AppKindPrimingTests.swift +++ /dev/null @@ -1,129 +0,0 @@ -import Testing - -@testable import BlurtEngine - -@Suite("AppKindPriming") -struct AppKindPrimingTests { - // MARK: - Kind recognition - - /// One bundle-ID → kind expectation, tabled for per-case failure output like - /// `TranscriptionSteeringTests.Case`. - struct KindCase: Sendable, CustomTestStringConvertible { - let bundleID: String? - let expected: AppKindPriming.Kind? - var testDescription: String { bundleID ?? "nil" } - } - - static let kindCases: [KindCase] = [ - KindCase(bundleID: "com.apple.Terminal", expected: .terminal), - KindCase(bundleID: "com.googlecode.iterm2", expected: .terminal), - KindCase(bundleID: "com.mitchellh.ghostty", expected: .terminal), - KindCase(bundleID: "com.microsoft.VSCode", expected: .codeEditor), - KindCase(bundleID: "com.apple.dt.Xcode", expected: .codeEditor), - KindCase(bundleID: "com.todesktop.230313mzl4w4u92", expected: .codeEditor), - // Prefix families: any JetBrains IDE, any Sublime Text major version. - KindCase(bundleID: "com.jetbrains.pycharm", expected: .codeEditor), - KindCase(bundleID: "com.sublimetext.4", expected: .codeEditor), - KindCase(bundleID: "com.tinyspeck.slackmacgap", expected: .slack), - KindCase(bundleID: "md.obsidian", expected: .obsidian), - KindCase(bundleID: "com.apple.mail", expected: nil), - KindCase(bundleID: "", expected: nil), - KindCase(bundleID: " ", expected: nil), - KindCase(bundleID: nil, expected: nil), - ] - - @Test("bundle IDs map to their app kind", arguments: kindCases) - func kind(_ c: KindCase) { - #expect(AppKindPriming.kind(ofBundleID: c.bundleID) == c.expected) - } - - // MARK: - Language inference from window titles - - /// One window-title → language expectation. - struct LanguageCase: Sendable, CustomTestStringConvertible { - let title: String - let expected: String? - var testDescription: String { title } - } - - static let languageCases: [LanguageCase] = [ - LanguageCase(title: "main.py — blurt — Visual Studio Code", expected: "Python code"), - // VS Code prepends ● to the filename of a dirty buffer. - LanguageCase(title: "● api.ts — server", expected: "TypeScript code"), - LanguageCase(title: "blurt — TranscriptionPrompt.swift", expected: "Swift code"), - LanguageCase(title: "deploy.sh — infra", expected: "a shell script"), - // Extension matching is case-insensitive. - LanguageCase(title: "README.MD — notes", expected: "markdown"), - // First filename wins when several tokens carry extensions. - LanguageCase(title: "index.js next.config.ts", expected: "JavaScript code"), - // A dotfile has no base name + extension split. - LanguageCase(title: ".zshrc — dotfiles", expected: nil), - // Version numbers and hostnames are not filenames. - LanguageCase(title: "release v0.1.34 — dictation.assemblyai.com", expected: nil), - LanguageCase(title: "Untitled-1", expected: nil), - LanguageCase(title: "", expected: nil), - ] - - @Test("window titles yield the open file's language", arguments: languageCases) - func language(_ c: LanguageCase) { - #expect(AppKindPriming.language(inWindowTitle: c.title) == c.expected) - } - - // MARK: - Clause rendering - - @Test("a terminal renders the shell-command formatting instruction") - func terminalClause() { - #expect( - AppKindPriming.clause(bundleID: "com.apple.Terminal", windowTitle: nil) - == "Format the result as a shell command with no trailing period.") - } - - @Test("a code editor names the open file's language when the title carries one") - func codeEditorClauseWithLanguage() { - #expect( - AppKindPriming.clause(bundleID: "com.microsoft.VSCode", windowTitle: "main.py — blurt") - == "Format the result as Python code.") - } - - @Test("a code editor stays generic when the title names no recognizable file") - func codeEditorClauseGeneric() { - #expect( - AppKindPriming.clause(bundleID: "com.apple.dt.Xcode", windowTitle: "Welcome to Xcode") - == "Format the result as code.") - } - - @Test("Slack renders the casual-message formatting instruction") - func slackClause() { - #expect( - AppKindPriming.clause(bundleID: "com.tinyspeck.slackmacgap", windowTitle: "#eng-backend") - == "Format the result as a casual Slack message, using Slack emoji where they fit.") - } - - @Test("Obsidian renders the markdown formatting instruction") - func obsidianClause() { - #expect( - AppKindPriming.clause(bundleID: "md.obsidian", windowTitle: "Meeting notes") - == "Format the result as markdown.") - } - - @Test("every clause reads as a rewrite instruction, not a transcription instruction") - func clausesAreRewriteInstructions() { - // These render into `config.llm.instruction`, which is an instruction to an - // LLM rewriting finished text. The Sync STT docs are explicit that - // `config.prompt` takes a *description of the audio* instead — so a clause - // that slipped back into "Transcribe speech into …" phrasing would be aimed - // at the wrong field, which is how the app-kind priming was silently a no-op - // before. - for bundleID in ["com.apple.Terminal", "com.microsoft.VSCode", "com.tinyspeck.slackmacgap", "md.obsidian"] { - let clause = AppKindPriming.clause(bundleID: bundleID, windowTitle: nil) - #expect(clause?.hasPrefix("Format the result as ") == true, "\(bundleID)") - #expect(clause?.contains("Transcribe") == false, "\(bundleID)") - } - } - - @Test("an unrecognized app contributes no clause") - func unrecognizedApp() { - #expect(AppKindPriming.clause(bundleID: "com.apple.mail", windowTitle: "Re: Q3") == nil) - #expect(AppKindPriming.clause(bundleID: nil, windowTitle: "main.py") == nil) - } -} diff --git a/Tests/BlurtEngineTests/AssemblyAITranscriberTests.swift b/Tests/BlurtEngineTests/AssemblyAITranscriberTests.swift index b038127..5f641c2 100644 --- a/Tests/BlurtEngineTests/AssemblyAITranscriberTests.swift +++ b/Tests/BlurtEngineTests/AssemblyAITranscriberTests.swift @@ -158,11 +158,9 @@ struct HTTPClientTests { func configIncludesSteeringFields() throws { let object = try configObject( steering: TranscriptionSteering.Fields( - conversationContext: ["$ git status"], keyterms: ["Blurt", "AssemblyAI"], - rewriteInstruction: "Format the result as markdown.")) + conversationContext: ["$ git status"], keyterms: ["Blurt", "AssemblyAI"])) #expect(object["conversation_context"] as? [String] == ["$ git status"]) #expect(object["keyterms_prompt"] as? [String] == ["Blurt", "AssemblyAI"]) - #expect((object["llm"] as? [String: Any])?["instruction"] as? String == "Format the result as markdown.") #expect(object["sample_rate"] as? Int == 16_000) // The capture path is mono by construction; the declared geometry must agree. #expect(object["channels"] as? Int == 1) @@ -172,27 +170,33 @@ struct HTTPClientTests { func configNeverSendsPrompt() throws { // A custom `config.prompt` replaces the service's managed default *and* its // language steering, and the field wants a description of the audio rather - // than instructions — which is why formatting moved to `llm.instruction` and - // vocabulary to `keyterms_prompt`. Sending no prompt keeps the managed - // default, so this pins the field's absence for every steering shape. + // than instructions — which is why vocabulary rides in `keyterms_prompt`. + // Sending no prompt keeps the managed default, so this pins the field's + // absence for every steering shape. let shapes: [TranscriptionSteering.Fields] = [ .empty, - TranscriptionSteering.Fields( - conversationContext: ["Hi Sam,"], keyterms: ["Blurt"], - rewriteInstruction: "Format the result as markdown."), + TranscriptionSteering.Fields(conversationContext: ["Hi Sam,"], keyterms: ["Blurt"]), ] for steering in shapes { #expect(try configObject(steering: steering).keys.contains("prompt") == false) } } - @Test("config part requests the default cleanup rewrite when no formatting is needed") + @Test("config part always requests the unsteered default cleanup rewrite") func configRequestsDefaultRewrite() throws { - // `llm` must be present and empty when there is no app-kind clause: present - // so the service runs the rewrite at all, empty so the server-owned default - // cleanup instruction (and its guardrails) applies rather than a client-side - // copy. `isEmpty == true` also covers presence — false for a missing `llm`. - #expect((try configObject(steering: .empty)["llm"] as? [String: Any])?.isEmpty == true) + // `llm` must be present and empty for every utterance: present so the service + // runs the rewrite at all, empty so the server-owned default cleanup + // instruction (and its guardrails) applies rather than a client-side copy. + // Empty also means nothing about the destination app rides here, which is + // what an earlier `instruction` carried. `isEmpty == true` covers presence + // too — it is false for a missing `llm`. + let shapes: [TranscriptionSteering.Fields] = [ + .empty, + TranscriptionSteering.Fields(conversationContext: ["Hi Sam,"], keyterms: ["Blurt"]), + ] + for steering in shapes { + #expect((try configObject(steering: steering)["llm"] as? [String: Any])?.isEmpty == true) + } } @Test("config part omits the context and keyterms fields when they are empty") diff --git a/Tests/BlurtEngineTests/DictationLogTests.swift b/Tests/BlurtEngineTests/DictationLogTests.swift index 0b2b696..84187ad 100644 --- a/Tests/BlurtEngineTests/DictationLogTests.swift +++ b/Tests/BlurtEngineTests/DictationLogTests.swift @@ -14,12 +14,10 @@ private struct DecodedEntry: Decodable { private struct DecodedSteering: Decodable { let conversationContext: [String] let keytermsPrompt: [String] - let llmInstruction: String? enum CodingKeys: String, CodingKey { case conversationContext = "conversation_context" case keytermsPrompt = "keyterms_prompt" - case llmInstruction = "llm_instruction" } // A missing array decodes to empty rather than nil: whether a field was @@ -29,7 +27,6 @@ private struct DecodedSteering: Decodable { let container = try decoder.container(keyedBy: CodingKeys.self) conversationContext = try container.decodeIfPresent([String].self, forKey: .conversationContext) ?? [] keytermsPrompt = try container.decodeIfPresent([String].self, forKey: .keytermsPrompt) ?? [] - llmInstruction = try container.decodeIfPresent(String.self, forKey: .llmInstruction) } } @@ -110,18 +107,18 @@ struct DictationLogTests { func logsOnlyWhatWasSent() { let url = makeTempLogURL() let context = TranscriptionContext( - appName: "Obsidian", bundleID: "md.obsidian", windowTitle: "Grocery list", + appName: "Obsidian", windowTitle: "Grocery list", fieldLabel: "text entry area", priorText: "- milk", selectedText: "- bread") DictationLog.write(transcript: "p", context: context, to: url, now: Date()) let line = readLog(url).split(separator: "\n").first.map(String.init) ?? "" // Everything the request carried is recorded, under the wire's own names… let decoded = try? JSONDecoder().decode(DecodedSteering.self, from: Data(line.utf8)) - #expect(decoded?.llmInstruction == "Format the result as markdown.") #expect(decoded?.conversationContext == ["- milk"]) // …and none of the captured-but-unsent context is. Values, not just keys: - // the entry must carry no trace of what stayed on the machine. Selected text - // is on this list because the paste replaces it, so it is never sent. - for unsent in ["Obsidian", "md.obsidian", "Grocery list", "text entry area", "- bread"] { + // the entry must carry no trace of what stayed on the machine. Two are + // load-bearing: selected text, which the paste replaces so it is never sent, + // and anything naming the destination app, which is never sent either. + for unsent in ["Obsidian", "Grocery list", "text entry area", "- bread"] { #expect(!line.contains(unsent)) } } @@ -130,7 +127,7 @@ struct DictationLogTests { func logsAssembledSteering() { let url = makeTempLogURL() let context = TranscriptionContext( - appName: "Obsidian", bundleID: "md.obsidian", windowTitle: "Grocery list", + appName: "Obsidian", windowTitle: "Grocery list", fieldLabel: "text entry area", priorText: "- milk", keyTerms: ["Blurt"]) DictationLog.write(transcript: "p", context: context, to: url, now: Date()) let line = readLog(url).split(separator: "\n").first.map(String.init) ?? "" @@ -138,10 +135,9 @@ struct DictationLogTests { let sent = TranscriptionSteering.build(context: context) // The log is the corpus prompt iteration reads, so it has to agree with the // builder field-for-field rather than approximately. - #expect(decoded?.llmInstruction == sent.rewriteInstruction) #expect(decoded?.conversationContext == sent.conversationContext) #expect(decoded?.keytermsPrompt == sent.keyterms) - #expect(decoded?.llmInstruction == "Format the result as markdown.") + #expect(decoded?.conversationContext == ["- milk"]) #expect(decoded?.keytermsPrompt == ["Blurt"]) } @@ -151,7 +147,9 @@ struct DictationLogTests { DictationLog.write(transcript: "p", context: nil, to: url, now: Date()) let line = readLog(url).split(separator: "\n").first.map(String.init) ?? "" // Absent, not `null` and not `[]` — the entry should read as "nothing was - // customized", matching the request, which omits these fields too. + // customized", matching the request, which omits these fields too. `prompt` + // and `llm_instruction` are never written for any context, so their absence + // here doubles as a pin against reintroducing either. #expect(!line.contains("\"prompt\"")) #expect(!line.contains("\"llm_instruction\"")) #expect(!line.contains("\"conversation_context\"")) @@ -210,8 +208,7 @@ struct DictationLogGateTests { // covered in `FocusCaptureTests`; this suite can only see contexts that // already cleared it. let context = TranscriptionContext( - appName: "Terminal", bundleID: "com.apple.Terminal", - windowTitle: "Vault", fieldLabel: "Command", + appName: "Terminal", windowTitle: "Vault", fieldLabel: "Command", priorText: "$ git", selectedText: nil) let offURL = makeTempLogURL() DictationLog.append( @@ -226,8 +223,8 @@ struct DictationLogGateTests { DictationLog.queue.sync {} #expect(!FileManager.default.fileExists(atPath: offURL.path)) let logged = readLog(onURL) - #expect(logged.contains("Format the result as a shell command with no trailing period.")) #expect(logged.contains("$ git")) + #expect(!logged.contains("Terminal")) #expect(!logged.contains("Vault")) #expect(!logged.contains("Command")) } diff --git a/Tests/BlurtEngineTests/TranscriptionContextTests.swift b/Tests/BlurtEngineTests/TranscriptionContextTests.swift index 0db647f..3550178 100644 --- a/Tests/BlurtEngineTests/TranscriptionContextTests.swift +++ b/Tests/BlurtEngineTests/TranscriptionContextTests.swift @@ -7,7 +7,7 @@ import Testing /// The agreement with `TranscriptionSteering.build` is one-directional: /// `isEmpty == true` must always correspond to empty steering fields, while a /// non-empty context may still steer nothing — its signals can be carry-only -/// (app name, field label, or selected text from an unrecognized app). +/// (app name, window title, field label, or selected text). @Suite("TranscriptionContext") struct TranscriptionContextTests { @Test("both fields nil is empty") @@ -30,13 +30,6 @@ struct TranscriptionContextTests { #expect(!TranscriptionContext(appName: nil, priorText: "hello there").isEmpty) } - @Test("a bundle ID alone makes it non-empty (and produces a rewrite instruction)") - func bundleIDPresent() { - let context = TranscriptionContext(appName: nil, bundleID: "com.apple.Terminal", priorText: nil) - #expect(!context.isEmpty) - #expect(TranscriptionSteering.build(context: context).rewriteInstruction != nil) - } - @Test("real selected text makes it non-empty") func selectedTextPresent() { #expect(!TranscriptionContext(appName: nil, priorText: nil, selectedText: "highlighted").isEmpty) @@ -65,10 +58,8 @@ struct TranscriptionContextTests { #expect(TranscriptionSteering.build(context: context) == .empty) } - // Renderable signals (a recognized bundle ID, key terms, prior text) each - // steer at least one field… + // The two renderable signals (key terms, prior text) each steer a field… let renderable = [ - TranscriptionContext(appName: nil, bundleID: "com.apple.Terminal", priorText: nil), TranscriptionContext(appName: nil, priorText: nil, keyTerms: ["Blurt"]), TranscriptionContext(appName: nil, priorText: "Hi Sam,"), ] @@ -78,11 +69,18 @@ struct TranscriptionContextTests { } // …while carry-only signals make the context non-empty (worth carrying for - // the injector) yet steer nothing. Selected text is the interesting one: the - // paste replaces it, so it is never priming. - let carryOnly = TranscriptionContext(appName: "Mail", priorText: nil, selectedText: "sel") - #expect(!carryOnly.isEmpty) - #expect(TranscriptionSteering.build(context: carryOnly) == .empty) + // the injector) yet steer nothing. Two are load-bearing: selected text, + // because the paste replaces it so it is never priming; and everything + // naming the destination app, which is deliberately not sent at all. + let carryOnly = [ + TranscriptionContext(appName: "Mail", priorText: nil, selectedText: "sel"), + TranscriptionContext(appName: "Code", windowTitle: "main.py — blurt", priorText: nil), + TranscriptionContext(appName: "Slack", fieldLabel: "Message", priorText: nil), + ] + for context in carryOnly { + #expect(!context.isEmpty) + #expect(TranscriptionSteering.build(context: context) == .empty) + } } @Test("Equatable compares every field") @@ -108,8 +106,5 @@ struct TranscriptionContextTests { #expect( TranscriptionContext(appName: "Notes", priorText: "x", keyTerms: ["a"]) != TranscriptionContext(appName: "Notes", priorText: "x", keyTerms: ["b"])) - #expect( - TranscriptionContext(appName: "Notes", bundleID: "com.a.b", priorText: "x") - != TranscriptionContext(appName: "Notes", bundleID: "com.c.d", priorText: "x")) } } diff --git a/Tests/BlurtEngineTests/TranscriptionSteeringTests.swift b/Tests/BlurtEngineTests/TranscriptionSteeringTests.swift index 5e57261..850b4ce 100644 --- a/Tests/BlurtEngineTests/TranscriptionSteeringTests.swift +++ b/Tests/BlurtEngineTests/TranscriptionSteeringTests.swift @@ -10,10 +10,11 @@ struct TranscriptionSteeringTests { /// /// Each recognized signal has exactly one home, and the table pins both /// directions — what renders and what must never leak into the wrong field: - /// prior-cursor text → `conversation_context`, key terms → `keyterms_prompt`, - /// the app-kind formatting clause → `llm.instruction`. The remaining focus - /// signals (app name, field label, selected text) render nowhere; the window - /// title is read only to name a code editor's language. + /// prior-cursor text → `conversation_context`, key terms → `keyterms_prompt`. + /// Every other focus signal renders nowhere. That includes anything naming the + /// destination app: the table keeps a case per app family that once earned a + /// formatting clause, each pinned to `.empty`, so reintroducing app-kind + /// steering fails here. struct Case: Sendable, CustomTestStringConvertible { let name: String let context: TranscriptionContext? @@ -35,67 +36,45 @@ struct TranscriptionSteeringTests { appName: "Mail", windowTitle: "Re: Q3 pricing", fieldLabel: "Body", priorText: nil, selectedText: "the old plan"), expected: .empty), - Case( - name: "unrecognized bundle ID renders no instruction", - context: TranscriptionContext(appName: "Mail", bundleID: "com.apple.mail", priorText: nil), - expected: .empty), Case( name: "prior-cursor text becomes the single conversation-context turn", context: TranscriptionContext(appName: "Mail", priorText: "Hi Sam, thanks for"), expected: TranscriptionSteering.Fields( - conversationContext: ["Hi Sam, thanks for"], keyterms: [], rewriteInstruction: nil)), + conversationContext: ["Hi Sam, thanks for"], keyterms: [])), Case( name: "prior text is trimmed of surrounding whitespace", context: TranscriptionContext(appName: nil, priorText: " Hi Sam,\n "), - expected: TranscriptionSteering.Fields( - conversationContext: ["Hi Sam,"], keyterms: [], rewriteInstruction: nil)), + expected: TranscriptionSteering.Fields(conversationContext: ["Hi Sam,"], keyterms: [])), Case( name: "key terms become keyterms_prompt, not a prompt clause", context: TranscriptionContext(appName: nil, priorText: nil, keyTerms: ["AssemblyAI", "Kubernetes"]), expected: TranscriptionSteering.Fields( - conversationContext: [], keyterms: ["AssemblyAI", "Kubernetes"], rewriteInstruction: nil)), - Case( - name: "terminal → shell-command formatting instruction", - context: TranscriptionContext(appName: "Terminal", bundleID: "com.apple.Terminal", priorText: nil), - expected: TranscriptionSteering.Fields( - conversationContext: [], keyterms: [], - rewriteInstruction: "Format the result as a shell command with no trailing period.")), + conversationContext: [], keyterms: ["AssemblyAI", "Kubernetes"])), Case( - name: "code editor names the window title's language", - context: TranscriptionContext( - appName: "Code", bundleID: "com.microsoft.VSCode", windowTitle: "main.py — blurt", priorText: nil), - expected: TranscriptionSteering.Fields( - conversationContext: [], keyterms: [], - rewriteInstruction: "Format the result as Python code.")), + name: "a terminal steers nothing", + context: TranscriptionContext(appName: "Terminal", windowTitle: "zsh — 80×24", priorText: nil), + expected: .empty), Case( - name: "code editor with no recognizable filename stays generic", - context: TranscriptionContext( - appName: "Xcode", bundleID: "com.apple.dt.Xcode", windowTitle: "Welcome to Xcode", priorText: nil), - expected: TranscriptionSteering.Fields( - conversationContext: [], keyterms: [], rewriteInstruction: "Format the result as code.")), + name: "a code editor steers nothing, not even the open file's language", + context: TranscriptionContext(appName: "Code", windowTitle: "main.py — blurt", priorText: nil), + expected: .empty), Case( - name: "Slack → casual-message formatting instruction", - context: TranscriptionContext( - appName: "Slack", bundleID: "com.tinyspeck.slackmacgap", fieldLabel: "Message", priorText: nil), - expected: TranscriptionSteering.Fields( - conversationContext: [], keyterms: [], - rewriteInstruction: "Format the result as a casual Slack message, using Slack emoji where they fit.")), + name: "Slack steers nothing", + context: TranscriptionContext(appName: "Slack", fieldLabel: "Message", priorText: nil), + expected: .empty), Case( - name: "Obsidian → markdown formatting instruction", + name: "Obsidian steers nothing", context: TranscriptionContext( - appName: "Obsidian", bundleID: "md.obsidian", - windowTitle: "Grocery list - Cowork - Obsidian 1.12.7", fieldLabel: "text entry area", - priorText: nil), - expected: TranscriptionSteering.Fields( - conversationContext: [], keyterms: [], rewriteInstruction: "Format the result as markdown.")), + appName: "Obsidian", windowTitle: "Grocery list - Cowork - Obsidian 1.12.7", + fieldLabel: "text entry area", priorText: nil), + expected: .empty), Case( - name: "all three fields populate independently", + name: "both fields populate independently, and the app still renders nowhere", context: TranscriptionContext( - appName: "Terminal", bundleID: "com.apple.Terminal", windowTitle: "zsh — 80×24", + appName: "Terminal", windowTitle: "zsh — 80×24", priorText: "$ git status", selectedText: "modified: README.md", keyTerms: ["Blurt"]), expected: TranscriptionSteering.Fields( - conversationContext: ["$ git status"], keyterms: ["Blurt"], - rewriteInstruction: "Format the result as a shell command with no trailing period.")), + conversationContext: ["$ git status"], keyterms: ["Blurt"])), ] @Test("build maps focus context to the dictation steering fields", arguments: cases)