You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Custom style instructions for the cleanup rewrite (#130)
* feat: custom style instructions appended to the cleanup rewrite
An Advanced-settings free-text field (e.g. "add some emojis where
appropriate sparingly", "always write in lowercase") whose contents are
appended to the cleanup instruction the dictation request's llm block
carries, so the server-side rewrite also applies the user's formatting
preferences. Empty or blank means the request is exactly what ships today.
The dictation API rejects the whole request over its 2048-character
instruction cap, so the appended text is capped at the real headroom the
base instruction and a bridging preamble leave
(CleanupInstruction.customStyleBudget, currently 352 characters) —
enforced in the Settings field (with a visible counter) and again at send
time, and derived from the actual lengths rather than restated.
Storage follows KeyTermsStore (read-only store, @AppStorage is the sole
writer, trimming on the read side); the transcriber reads it per request
via an injected closure like enhancedTranscripts, so an edit applies to
the very next dictation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gwjo9LJw6z5PDyNeZjHbhc
* fix: review findings on custom style instructions
- Measure the instruction budget in UTF-8 bytes everywhere (cap
arithmetic, engine trim, Settings counter and truncation), derived
from the one budget definition. The API's 2048 cap was measured
against the live endpoint but its unit was not; bytes are the largest
plausible unit, so conservative against all of them. Truncation drops
whole Characters so a multi-scalar emoji is never split — one shared
String.prefix(maxUTF8Bytes:) used by both enforcement points, plus an
emoji regression test.
- Skip building the combined instruction when enhanced transcripts are
off (the result was discarded); the over-cap log guard is unchanged.
- Hide the Settings counter while the section is disabled, and give it
an accessibility label ("N of M characters used") instead of the raw
"N/M" digits.
- Drop the transcriber-test assertions duplicating the equality check
and CleanupInstructionTests' structure tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gwjo9LJw6z5PDyNeZjHbhc
---------
Co-authored-by: Claude <noreply@anthropic.com>
Key properties of the design, which your integration can rely on:
60
60
61
61
-**One request per utterance, no streaming.** The dictation API returns the complete transcript — and its LLM-rewritten form — in the response body: no upload step, no job polling, no incremental deltas, no second request for the cleanup. `TranscriberProtocol.transcribe` is a single `async throws -> String`. UIs should show a "transcribing…" state and then the whole result; there is nothing to stream.
62
-
-**Cleanup happens server-side, and it's optional.** The request's `llm` block asks the service to apply our own cleanup instruction (`CleanupInstruction.text` — delete disfluencies, change nothing else) to the verbatim transcript inside the same call. It is the only instruction on the request: the separate `config.prompt` field, which primes the _transcription_, is switched off at `TranscriptionPrompt.isEnabled` and omitted from every request. The block is gated by the **enhanced transcripts** setting (`EnhancedTranscriptsStore`, on by default): turned off, the config omits `llm` and the verbatim transcript is pasted as spoken. The engine pastes `llm_response`, falling back to the verbatim `text` when the best-effort rewrite failed (`llm_error`) — a degradation, never a user-facing error. There is no client-side LLM pass, no styling stage, and deliberately no hook for one.
62
+
- **Cleanup happens server-side, and it's optional.** The request's `llm` block asks the service to apply our own cleanup instruction (`CleanupInstruction.text` — delete disfluencies, change nothing else) to the verbatim transcript inside the same call. It is the only instruction on the request: the separate `config.prompt` field, which primes the _transcription_, is switched off at `TranscriptionPrompt.isEnabled` and omitted from every request. The block is gated by the **enhanced transcripts** setting (`EnhancedTranscriptsStore`, on by default): turned off, the config omits `llm` and the verbatim transcript is pasted as spoken. The user's **custom style instructions** (`CustomStyleStore`, empty by default) are appended to that instruction via `CleanupInstruction.sendable(appending:)`, trimmed to the headroom the API's 2048 instruction cap leaves (measured in UTF-8 bytes, the conservative bound — the cap's own unit is unmeasured); blank means the base instruction goes out unchanged. The engine pastes `llm_response`, falling back to the verbatim `text` when the best-effort rewrite failed (`llm_error`) — a degradation, never a user-facing error. There is no client-side LLM pass, no styling stage, and deliberately no hook for one.
63
63
-**Latency is pre-paid where possible.**`press()` fires a detached `warmUp()` at the transcriber (pre-opening the HTTPS connection while the user speaks, ~170 ms saved cold) and kicks off the cross-process accessibility read of the focused field without awaiting it — the read is then consumed at transcribe time with a bounded wait (`DictationSession.contextWaitBudget`, 500 ms), so an unresponsive frontmost app costs the transcript its priming, never a multi-second stall — and never delays the recording indicator. On the way out, `release()` flips the phase to `.transcribing`_before_ reading the recorded audio back, so a host's stop cue fires at key-up rather than after the disk read.
64
64
-**A held trigger auto-releases.**`DictationSession` stops recording after `maxRecordingSeconds` (default `SyncSTTLimits.autoReleaseSeconds`, 115 s) so audio never exceeds what the endpoint accepts, and transcribes what it has. Clips shorter than `SyncSTTLimits.minPCMBytes` (~100 ms of audio — an accidental tap) are dropped as a silent no-op rather than sent to earn a 400.
`AssemblyAITranscriber` is a stateless `Sendable` struct. One `POST https://dictation.assemblyai.com/transcribe` per utterance: the audio as raw S16LE PCM (the `pcm` blob, byte-for-byte) in the `audio` multipart part, plus a JSON `config` part (`sample_rate`, `channels`, the rendered `prompt` (nil today, so the field is omitted), and — while enhanced transcripts are enabled, the default — an `llm` block whose one `instruction` field carries `CleanupInstruction.text`), with the API key in `Authorization` (no model header — the service pins the STT model server-side). The response carries the verbatim `text` and the rewritten `llm_response`; the transcriber returns the rewrite and falls back to `text` when it is null (the rewrite is best-effort — `llm_error` is logged, never surfaced as a failure). Its initializer takes an `apiKeyProvider` closure (defaults to `APIKeyStore.current`), a `baseURL`, an `HTTPTransport` — inject a fake transport (see `Tests/BlurtEngineTests/Stubs/FakeHTTPTransport.swift`) to test against canned responses — and an `enhancedTranscripts` closure deciding, per request, whether the `llm` block is sent (nil, the default, reads `EnhancedTranscriptsStore`). `warmUp()` fires a throwaway GET at the host root to pre-pool the connection; it never throws and any failure just means the real request pays connection setup as before.
136
+
`AssemblyAITranscriber` is a stateless `Sendable` struct. One `POST https://dictation.assemblyai.com/transcribe` per utterance: the audio as raw S16LE PCM (the `pcm` blob, byte-for-byte) in the `audio` multipart part, plus a JSON `config` part (`sample_rate`, `channels`, the rendered `prompt` (nil today, so the field is omitted), and — while enhanced transcripts are enabled, the default — an `llm` block whose one `instruction` field carries `CleanupInstruction.text`, with any custom style instructions appended — `CleanupInstruction.sendable(appending:)`), with the API key in `Authorization` (no model header — the service pins the STT model server-side). The response carries the verbatim `text` and the rewritten `llm_response`; the transcriber returns the rewrite and falls back to `text` when it is null (the rewrite is best-effort — `llm_error` is logged, never surfaced as a failure). Its initializer takes an `apiKeyProvider` closure (defaults to `APIKeyStore.current`), a `baseURL`, an `HTTPTransport` — inject a fake transport (see `Tests/BlurtEngineTests/Stubs/FakeHTTPTransport.swift`) to test against canned responses — an `enhancedTranscripts` closure deciding, per request, whether the `llm` block is sent (nil, the default, reads `EnhancedTranscriptsStore`), and a `customStyle` closure supplying the style instructions appended to the cleanup instruction (nil, the default, reads `CustomStyleStore`). `warmUp()` fires a throwaway GET at the host root to pre-pool the connection; it never throws and any failure just means the real request pays connection setup as before.
137
137
138
138
The model's limits live in `SyncSTTLimits` (16 kHz sample rate, ~0.1 s–120 s audio, and the auto-release math — the sync STT model behind the dictation service) — the single source shared by the mic, the session, and the request so recorded and declared geometry can't drift.
0 commit comments