v2.6.0 — HUD quick modes, polish v6, unified capture, performance - #37
Merged
Conversation
…very - Recording pill gains sticky mode/translation chips (flow layout, wraps at 400pt); an explicit chip tap forces AI polish past the duration gates, and picking a translation profile with no target turns the shared output language on (last target, English by default). - Completed pill shows the full transcript (content-hugging up to ~7 lines, scrollable viewport beyond), copy/close actions, and re-polish chips that rerun the last dictation with the new mode, updating clipboard and the History row without re-pasting; hover pauses the auto-dismiss. - Menu bar gets quick mode/output-language pickers and an 'improve clipboard by voice' action; overlay pills switch from capsule to continuous rounded rect so multi-line states stop reading as wasted width. - Clipboard edit dictation: Option+Shift+Space reads the copied text, the dictation becomes the instruction, and the rewritten text is delivered via the normal pipeline (no fidelity guards; failures ship the source text unchanged with the error recorded). - Crash recovery: a launch sweep turns abandoned dictation WAVs into failed, retranscribable History rows, repairing the stale RIFF/data sizes an interrupted writer leaves behind; Esc cancel now confirms the audio was saved to History.
… iteration - The overlay never disappears: every dismissed state collapses into a tiny always-visible dock chip at the anchor position, recording morphs out of it, and hovering (or clicking) the chip reopens the last transcription. - Quick chips are now user-pinned profiles (max 3, star toggle in Settings → Prompts; defaults: AI Assistant + Work Message). Chips toggle: deselecting falls back to the base clean-up mode, so 'Automatic' no longer needs a chip. - Completed pill gains a mic button that dictates an instruction applied to the shown text (iterate until it reads right); the refined result updates the clipboard and pill without re-pasting. - Overlay window clamps to the screen's visible frame so tall states never slide off-screen; docked anchor hugs the bottom edge. - Longer waveform (11 bars) with the same outward ripple; taller scroll viewport for long transcripts.
- Replace the measured flow layout with a plain HStack: at most 3 pinned chips plus the translation chip always fit one line, and the custom layout's height could disagree with placement, letting a chip render outside the pill's background. - Dock chip is slimmer and narrower (rectangular shape kept) and expands on click only — hover just highlights it, so a stray mouse pass at the screen edge no longer pops the transcription open.
…tput language Redesign the overlay as a fixed transparent surface: the dock chip is a permanent fixture at the screen edge and every active state is a droplet pill that detaches from the chip and is absorbed back on dismiss, with squash-and-stretch chip feedback. The fixed window also fixes the crash on recording start: content-driven window resizing during transition animations made NSHostingView mutate the window frame inside the AppKit display cycle (updateAnimatedWindowSize), throwing NSInternalInconsistencyException. Clicking outside the result pill (or the dock chip) now collapses it back into the chip via global/local mouse monitors that hit-test the actual content. An explicit output language now bypasses the duration/length polish skip gates (short dictations were shipping untranslated as skipped_duration) and the translation reminder forbids leftover source-language words. Changelog covers the whole branch.
Vocabulary now reaches local STT engines as a canonical glossary prompt (WhisperKit promptTokens, Local AI Server prompt field), and the polish prompt is rebuilt priority-first: output language, user dictionary that maps mishearings and never translates, then fidelity. Recent dictations feed the prompt as continuity context, correction targets anchor the fidelity guard, and AI suggestions stop proposing fragment mappings. - Simplified menu bar popover; About moves to its own window; welcome tour and auto-paste live in Settings (the Settings toggle now works). - Completed pill sizes from real text measurement (no more bottom clipping or empty scroll areas); outside clicks collapse it against the measured content frame; Settings tabs cross-fade with subtle scale. - Translation hardening: the instruction guard skips its cross-language cue check when a target language is set, short dictations always polish with an explicit target, and picking an AI mode promotes the minimum-duration gate to Always.
…plified surface Rebuild AI polish around one benchmarked adaptive prompt: two-tier filler deletion (always-delete vs contextual), duplicated-idea collapse, sacred numbers and ranges, no invented lists, tone preserved. Validated case-by-case against real dictation history on local Qwen 3.5 4B/9B with zero anchor loss. Split transcripts past ~2.2k chars at sentence boundaries and polish each chunk on its own — long rambling dictations now clean up like short ones instead of degrading with input length. Remove the mode picker, prompt-profile CRUD, quick-mode chips, clipboard voice edit (option-shift-space), the completed-pill voice edit, and the duration/short-text skip gates: polish always runs when enabled. The overlay keeps a single translation chip, now inline in the pill headers. Settings -> Prompts becomes personal context plus live preview; the memory manager is reduced to reviewable correction suggestions that feed the dictionary.
- Sentence chunker: enders only close a word (no seams inside 24.7, 10.000, .env, CLAUDE.md), whitespace fallback for punctuation-less transcripts, tiny tails merge back into their neighbor - VocabularyManager: mechanical passes (local regex + Deepgram replace) skip expansion pairs whose key re-matches its own value; they stay AI-prompt-only where context judgment exists - PolishOutputSanitizer strips a leading <think> block - Polish prompt: sentence-edge connector fillers deleted, digits stay digits; document why numbers are not hard-guard anchors - Drop dead storage keys and stale mode-era wording
…ware device HUD Device switches (AirPods connecting, speaker/headset swaps) killed the app: AVFAudio asserts tap formats with Objective-C NSExceptions that Swift cannot catch, and the input preflight warm-up hit them mid-route-transition (three SIGABRT crash logs, all in installTap). Reliability package: - ObjC exception shim + AudioEngineGuard: inputNode/installTap/prepare+start run through a @try/@catch bridge and rethrow as a transient Swift error; applied to preflight warm-up (with retry budget), recorder start/rebuild, streaming capture start/rebuild, and the level monitor. Start paths classify the new error as transient and reuse the existing retry loop. - Bluetooth-aware start timeouts: BT inputs renegotiate A2DP->HFP for 1-3 s, so first-buffer wait and retry budget widen instead of failing mid-handshake. - Recording pill shows "connecting <mic>" with a travelling-wave meter until the first real buffer lands (no more dead flat waveform), suppressing the no-speech hint during the handshake. - Device HUD is phase-aware (connecting -> ready, plus fallback when the preferred mic vanished) with transport-matched glyphs (AirPods/USB/built-in) and in-place phase morphs. - Active-recording markers (PID sidecars) let launch recovery adopt crashed takes instantly instead of after the 60 s age gate; recovery now reports what it adopted. - Continue-previous dictation: cancelled/interrupted/crash-recovered takes become an opt-in chip on the next batch recording; at stop time the WAVs merge (format-converting when needed) into one transcript, superseding the recovered History row. Merge failure never loses the new take. 231 tests green (new: merger, markers, instant recovery); live-validated with 12 real route changes against connected AirPods Pro with zero crashes.
…nput Recording with a mic that is NOT the system default input pays full route setup on every take; on AirPods that is the whole 1-3 s A2DP->HFP handshake, because macOS only keeps the link warm for the default input. When app selection and System Settings agreed, dictation started instantly — the fix is making that agreement automatic: - "Primary microphone" toggle (default ON) under the mic picker for explicit selections: SapoWhisper imposes the chosen mic as the system default input and restores it after every device swap, so connecting AirPods or a headset never steals the mic. Toggling it on re-syncs immediately; off restores the old bind-only behavior (with its Bluetooth start latency). - Pre-capture sync: recording start aligns the system default with the pinned selection synchronously before opening the engine; the route settle window it may open is honored by the existing start delay. 231 tests + ci-check green; live-validated: with Razer selected, forcing the system default to AirPods gets reverted to Razer within ~2.5 s.
The pinned-mic description read as two sentences of system-input jargon; replace it with one plain line in EN/ES: the mic stays fixed when AirPods connect and recordings start instantly.
The output-language tile came from a removed three-pickers row; alone it stretched a mostly empty box across the card. It is now a single inline row (title + fidelity badge + trailing menu picker) whose only extra copy is the animated translation note when a target language is picked. The context card drops its double header (card title + section header) and becomes "Personal context" directly; saving shows a transient green "Saved" check next to the button instead of a permanent footer line, and the shared settings text editor tints its stroke green on focus. Removes the now-unused AIPolishSettingRow tile and orphan localization keys.
… motion Polish pipeline: - Instruction guard rejects only phrasing the model introduced (present in polished, absent from raw, punctuation-tolerant); weak openers must sit at the start of the output; translation keeps only self-reference and math-answer checks. Everyday speech no longer burns the retry budget and ships raw. - Per-chunk salvage: a failed/blocked/timed-out chunk falls back to its own raw text instead of discarding every polished sibling; per-chunk budgets; hosted endpoints polish chunks concurrently. - Overlay countdown uses the same summed per-chunk budget as the processor. - max_tokens on every request; finish_reason=length retries once with a doubled cap, then surfaces a truncation error instead of pasting cut text. - Fidelity guard anchors vocabulary on word boundaries and trims trailing punctuation from URL/email anchors. - Sanitizer: no quote-stripping when the text holds two quoted spans; an unterminated <think> block yields empty output (raw wins). - Dead pipeline removed (length-ratio/dense-script plumbing, unreachable retry tail, duplicate promptInstruction); memory record() early-returns unless a polish was applied. Vocabulary: - Real-word single variants (hit, pug, comet, cloud...) stay out of the deterministic correction pass; bigrams remain mechanical. - Short-token pattern no longer consumes sentence periods. - Cloud keyterm payloads send canonical forms only. Engines: - WhisperKit auto enables detectLanguage (prefill defaulted to <|en|>); vocabulary prompt tokens capped from the front; official large-v3 v20240930 turbo (+626MB variant) added and made the default. - Flux sends language_hint=es&en in auto and coalesces audio to 80ms chunks. - ElevenLabs realtime salvages the pending partial at stop and falls back to batch transcription of the local WAV when the session degraded; file retranscription always uses batch; batch sends temperature=0. Audio chain: - Capture gain applies to the float tap buffer before conversion with a soft-knee limiter (no more int16 hard clipping at high gain). - Sample-rate converters use Mastering algorithm at max quality. - Partial converter buffers on inputRanDry/flush are written, not dropped. - pause/resume run on the setup queue with AudioEngineGuard. - Stored os_unfair_lock instances migrated to OSAllocatedUnfairLock. - Batch transcribers prepare uploads off the main actor; WAV compression processes channels in bulk. Motion/UI: - Semantic animation tokens in Constants.Animation; Reduce Motion respected across the app. - Polish countdown rolls digits; HUD bounces/glows use phaseAnimator/keyframeAnimator; connecting wave runs on TimelineView; audio players animate continuously; General tab reveals animate. - Mic test stops capturing when its Settings tab is deselected; meter render isolated from the 47 Hz level stream; menu bar timer row isolated. - Localized WhisperKit/provider errors (EN/ES).
…ngine Merge AudioRecorder and StreamingAudioCapture (~800 twin lines) into one AudioCaptureEngine with Mode.batch/.streaming — batch is streaming with a nil chunk handler. Both paths now share the strongest machinery from each side: setup-generation cancellation guards and graceful device-bind fallback (from the recorder), on-queue-only engine/url assignment during setup and input-gap diagnostics (from the streaming capture). Log prefixes stay greppable per mode (Recorder/Flux). stopRecording returns a unified AudioCaptureResult (URL + duration + diagnostics).
…ation - One shared start/stop/pause/abort/binding path for the batch recorder, Deepgram Flux, and ElevenLabs realtime behind StreamingDictationSession plus per-engine contexts (replaces three hand-kept copies) - Recording duration ticker moved off @published: 10 Hz ticks no longer re-render every ViewModel observer (Settings tabs included); timer views subscribe locally - WhisperKitTranscriber and the vocabulary/AI-memory/prompt-context managers migrated to @observable; ViewModel mirrors are passthroughs now - Overlay repositions to the mouse screen on each fresh presentation (permanent dock chip had pinned it to the launch screen) - Pausing no longer clears the continue-previous-dictation chip - Transient connectivity flaps (URLError -1009/-1005) retry with the same backoff as transient 5xx across all HTTP clients
…t diff guard Recalibrate filler deletion with dual-use words kept when meaningful, merge repeated ideas, add a same-language dictionary example, and gate every prompt change on a bench against the production model. Structured outputs with a leading filler scan on OpenAI/OpenRouter (plain-text fallback), raw-tail continuity context for chunks 2+, a retry-only content diff guard for lost digits and dropped passages, and a fix for max_tokens never reaching the request body.
…xact model matching, prune dead strings - SpeechConfusionCatalog: single source for brand mishearing tables and spoken-form helpers, shared by VocabularyManager and AIPolishMemoryManager (the AI-memory copy had drifted: missing SapoWhisper variants and 5 brands) - Batch capture records 16 kHz directly for whisper-family engines on the STT-oriented qualities, avoiding the double resample at medium - WhisperKit model folders now match per exact variant: plain substring matching cross-deleted large-v3 siblings and faked download state - Remove 42 dead localization keys (en+es) and localize the remaining hardcoded WhisperKitError messages
The recording meter animated bar heights through the pill's shared drawing layer: every level tick re-ran a window-wide layout pass and re-rendered the flattened layer on the CPU each animation frame, including the pill's text glyphs, whose CoreGraphics bitmap buffers accumulated ~1 MB/s of resident memory per session (reachable, so never reported as leaks). - Render bars at a fixed frame and animate scaleEffect instead of frame height, so ticks no longer invalidate layout. - Wrap the bars in drawingGroup() so their fill/scale animations rasterize in an isolated Metal-backed layer; glyph redraw and the per-session memory growth are gone (draw_glyphs 71+ -> 3 samples). - Sharpen meter response: asymmetric capture smoothing (fast rise, slow fall) and equalizer attack 0.6 -> 0.85, so word onsets land on the next tick at the ~10 Hz level cadence.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release branch for v2.6.0. Highlights (full detail in CHANGELOG.md):
Gates: make ci-check green (253 tests), make release-check green, secrets-scan clean.