Merge pull request #40 from StevenACZ/feat/360-round2

v2.9.0 — 360 round 2: 4-bit local model, Liquid Glass overlay, VoiceOver, paste hardening

Author: StevenACZ

.github/workflows/ci.yml

@@ -10,7 +10,7 @@ jobs:
   ci-check:
     runs-on: macos-15
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
 
       - name: Select Xcode
         run: sudo xcode-select -s /Applications/Xcode.app

AGENTS.md

@@ -20,7 +20,7 @@ addresses, and machine-specific workflow details.
 - `TranscriptionPipeline`: shared transcribe -> polish -> paste -> persist control flow for stop paths; the ViewModel implements `TranscriptionPipelineHost`.
 - Strict concurrency is `complete` on app and test targets. Keep new code warning-free instead of widening unsafe isolation.
 - Engines: MLX Whisper local (default; vendored `LocalPackages/MLXWhisper`), Deepgram Nova-3 batch, Deepgram Flux Live, ElevenLabs Scribe batch/realtime, and Local AI Server batch STT through OpenAI-style endpoints. WhisperKit was removed deliberately (2026-07-06; MLX runs the same weights ~6x faster) — do not reintroduce it, and keep `EnginePortfolioMigration` mapping stored `whisper` selections to `mlx_whisper` and purging the CoreML caches.
-- `LocalPackages/MLXWhisper` is vendored from mlx-audio-swift (MIT, pinned commit in its Package.swift header) trimmed to the Whisper model. Local additions: initial-prompt (`<|startofprev|>`) vocabulary support, real auto language detection, and a downloader with progress; sync upstream fixes manually and keep the pin comment current. Its sources are exempt from repo swift-format lint. Building anything that links mlx-swift needs the Xcode Metal Toolchain component (`xcodebuild -downloadComponent MetalToolchain`) and `-skipPackagePluginValidation` on headless xcodebuild (already in the Makefile); plain `swift build` produces a binary without Metal shaders that dies at MLX init — bench with the package's `mlxwhisper-cli` built via xcodebuild instead (XCTest-host numbers are not representative for the GPU path).
+- `LocalPackages/MLXWhisper` is vendored from mlx-audio-swift (MIT, pinned commit in its Package.swift header) trimmed to the Whisper model. Local additions: initial-prompt (`<|startofprev|>`) vocabulary support, real auto language detection, a downloader with progress, quantized-checkpoint loading (4-bit), a Task-cancellation hook in the decode loop, and HF snapshots pinned to commit shas (bump revisions in `MLXWhisperModel.revision` + `WhisperModelDownloader.tokenizerRepo`); sync upstream fixes manually and keep the pin comment current. Its sources are exempt from repo swift-format lint. Building anything that links mlx-swift needs the Xcode Metal Toolchain component (`xcodebuild -downloadComponent MetalToolchain`) and `-skipPackagePluginValidation` on headless xcodebuild (already in the Makefile); plain `swift build` produces a binary without Metal shaders that dies at MLX init — bench with the package's `mlxwhisper-cli` built via xcodebuild instead (XCTest-host numbers are not representative for the GPU path).
 - Audio capture: one class, `AudioCaptureEngine`, serves every engine. `.batch` records a WAV at `AudioUploadQuality`, except whisper-family targets (MLX Whisper, Local AI Server) on the STT-oriented qualities (ultra-fast, medium), which capture 16 kHz directly — whisper decodes at 16 kHz and a higher-rate capture only adds a second resample. `.streaming` keeps fixed 16 kHz mono int16 for WebSocket compatibility and emits PCM chunks (batch is streaming with a nil chunk handler). Do not reintroduce per-path capture classes.
 - Streaming engines (Flux, ElevenLabs realtime) are driven through `StreamingDictationSession` plus one shared start/stop/pause/abort/binding path in the ViewModel (`StreamingEngineContext`). Do not add per-engine copies of that flow.
 - Observation: `MLXWhisperTranscriber` and the vocabulary/AI-memory/prompt-context managers are `@Observable` — views read them directly; do not reintroduce `@Published` mirrors in the ViewModel. High-frequency tickers (recording duration) stay OFF ObservableObject state: publish through a subject and subscribe locally in the one view that renders them.

CHANGELOG.md

@@ -6,6 +6,39 @@ This project follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+## [2.9.0] - 2026-07-09
+
+### Added
+
+- **Large V3 Turbo (4-bit) local model** — a new quantized tier of the recommended turbo model: 464 MB download instead of 1.5 GB and roughly a third of the RAM while transcribing (~0.5 GB vs ~1.6 GB) at comparable quality. The vendored MLX engine now reads quantized checkpoints natively (packed 4-bit weights, quantized tied embeddings), so more tiers can follow.
+- **Liquid Glass overlay on macOS 26** — the recording pill and the dock chip render in real glass, and they visually fuse when the droplet detaches from or absorbs into the chip. Older systems keep the exact material look they have today.
+- **VoiceOver support across the app** — every icon-only button now has a spoken label (overlay controls, History actions, model rows, vocabulary tools), the overlay announces dictation phases (recording started, transcribing, text pasted, failed), the audio player's scrubber is adjustable with VoiceOver and arrow keys (±5 s), and hover-only affordances like vocabulary chip deletion also reveal on keyboard focus.
+- **Export notice** — the History save panels for audio and text exports now remind that the file will contain dictated content before it leaves the app.
+
+### Changed
+
+- **Retrying and re-transcribing now run through the same pipeline as live dictations** — retry results update the original History row (no duplicate rows), stale results from an abandoned retry can no longer overwrite a newer dictation, and the audio behind a failed dictation is never cleaned up while it can still be retried.
+- **Consistent motion and color language** — animations across Settings, History, and Welcome now use the shared motion tokens (including new transition and shake timings), Welcome steps and pill content swap with a blur-replace transition (Reduce Motion keeps plain crossfades), and green-as-text uses an adaptive darker shade that passes WCAG contrast on light backgrounds. The app also ships a real AccentColor, so selection tint matches the brand in every window.
+- **Cloud batch engines pre-warm their connection** when the recording starts, shaving the DNS/TLS handshake (~100–400 ms) off the stop-to-paste wait on cold starts.
+- The vendored MLX decode loop honors task cancellation between decode steps — groundwork for cancellable re-transcriptions of long recordings.
+
+### Fixed
+
+- **Two latent crash windows in audio setup** — the onboarding microphone probe and the capture-setup cleanup path called AVFAudio APIs outside the engine guard; a device route change at the wrong moment could kill the app with the same uncatchable exception class that caused historic crashes. All engine calls now go through the guard.
+- **Settings window no longer clips its content** — the window was created at 800×560 while the content laid out at 860×620 (diverged in v2.2.0); both now share one constant, and the History window minimum matches its real layout.
+- **Cmd+, opens the real Settings window** — previously a phantom empty settings window could appear while an app window was focused.
+- **History survives database corruption** — a corrupt history database is detected on open (integrity check), sidelined with a timestamped name, and recreated fresh; History keeps working instead of silently doing nothing until reinstall.
+- A race between unloading and loading local models could leave a freshly loaded model unloaded (or two resident at once); model loads are now generation-checked.
+- "now" and the m/h/d suffixes in History timestamps were hardcoded English in the Spanish UI; they are localized.
+- White text on the amber processing color in the menu bar failed contrast badly (≈1.6:1); processing states now use dark text on amber.
+
+### Security
+
+- **Pasted text is sanitized before it reaches the clipboard** — control characters, ANSI escape sequences, bidirectional-override characters, and zero-width characters are stripped from polish output and raw fallbacks alike, so a misbehaving or hostile polish endpoint cannot smuggle terminal escapes or spoofed text into what Cmd+V types.
+- **Clipboard writes are marked concealed** (`org.nspasteboard.ConcealedType`), so clipboard managers and Universal Clipboard treat dictated text as sensitive instead of syncing and indexing it.
+- History storage (database and audio) is created user-only (`0o700`).
+- Supply chain hardening: Hugging Face model downloads are pinned to exact commit revisions, GitHub Actions are pinned by commit SHA, an unused Downloads entitlement was removed, and SECURITY.md documents supported versions and the reporting channel.
+
 ## [2.8.0] - 2026-07-06
 
 ### Added

LocalPackages/MLXWhisper/Package.swift

@@ -7,8 +7,10 @@ import PackageDescription
 // (2026-07-03), trimmed to the Whisper model only so the app does not compile
 // the 18 other STT families, the codecs, or mlx-swift-lm. Local changes on
 // top of upstream: initial-prompt (`<|startofprev|>`) support so the user
-// vocabulary reaches the decoder, and a downloader with real progress
-// reporting instead of prints.
+// vocabulary reaches the decoder, a downloader with real progress reporting
+// instead of prints, 4/8-bit quantized-checkpoint loading (`quantization` in
+// config.json), and a Task-cancellation hook between decode steps so long
+// transcriptions can be aborted.
 let package = Package(
     name: "MLXWhisper",
     platforms: [.macOS(.v14)],

LocalPackages/MLXWhisper/Sources/MLXWhisper/MLXWhisperRuntime.swift

@@ -18,11 +18,12 @@ public enum MLXWhisperRuntime {
 
 extension WhisperModel {
     /// Convenience entry point over raw 16 kHz mono samples, so callers do
-    /// not need to import MLX to build an `MLXArray`.
+    /// not need to import MLX to build an `MLXArray`. Cancellation-aware:
+    /// throws `CancellationError` when the surrounding task is cancelled.
     public func generate(
         samples: [Float],
         generationParameters: STTGenerateParameters
-    ) -> STTOutput {
-        generate(audio: MLXArray(samples), generationParameters: generationParameters)
+    ) throws -> STTOutput {
+        try generateCancellable(audio: MLXArray(samples), generationParameters: generationParameters)
     }
 }

LocalPackages/MLXWhisper/Sources/MLXWhisper/WhisperConfig.swift

@@ -1,5 +1,22 @@
 import Foundation
 
+/// `quantization` block written by mlx-community 4/8-bit checkpoints; absent
+/// on fp16/fp32 checkpoints.
+public struct WhisperQuantization: Codable, Sendable {
+    public var groupSize: Int
+    public var bits: Int
+
+    enum CodingKeys: String, CodingKey {
+        case groupSize = "group_size"
+        case bits
+    }
+
+    public init(groupSize: Int, bits: Int) {
+        self.groupSize = groupSize
+        self.bits = bits
+    }
+}
+
 public struct WhisperConfig: Codable, Sendable {
     public var modelType: String
     public var vocabSize: Int
@@ -23,6 +40,8 @@ public struct WhisperConfig: Codable, Sendable {
 
     public var scaleEmbedding: Bool
 
+    public var quantization: WhisperQuantization?
+
     enum CodingKeys: String, CodingKey {
         case modelType = "model_type"
         case vocabSize = "vocab_size"
@@ -41,6 +60,7 @@ public struct WhisperConfig: Codable, Sendable {
         case padTokenId = "pad_token_id"
         case decoderStartTokenId = "decoder_start_token_id"
         case scaleEmbedding = "scale_embedding"
+        case quantization
 
         // OpenAI / mlx-whisper layout (mlx-community/whisper-*).
         case nMels = "n_mels"
@@ -72,7 +92,8 @@ public struct WhisperConfig: Codable, Sendable {
         eosTokenId: Int = 50257,
         padTokenId: Int = 50257,
         decoderStartTokenId: Int = 50258,
-        scaleEmbedding: Bool = false
+        scaleEmbedding: Bool = false,
+        quantization: WhisperQuantization? = nil
     ) {
         self.modelType = modelType
         self.vocabSize = vocabSize
@@ -91,6 +112,7 @@ public struct WhisperConfig: Codable, Sendable {
         self.padTokenId = padTokenId
         self.decoderStartTokenId = decoderStartTokenId
         self.scaleEmbedding = scaleEmbedding
+        self.quantization = quantization
     }
 
     public init(from decoder: Swift.Decoder) throws {
@@ -137,6 +159,7 @@ public struct WhisperConfig: Codable, Sendable {
         padTokenId = try c.decodeIfPresent(Int.self, forKey: .padTokenId) ?? 50257
         decoderStartTokenId = try c.decodeIfPresent(Int.self, forKey: .decoderStartTokenId) ?? 50258
         scaleEmbedding = try c.decodeIfPresent(Bool.self, forKey: .scaleEmbedding) ?? false
+        quantization = try c.decodeIfPresent(WhisperQuantization.self, forKey: .quantization)
     }
 
     public func encode(to encoder: Swift.Encoder) throws {
@@ -158,6 +181,7 @@ public struct WhisperConfig: Codable, Sendable {
         try c.encode(padTokenId, forKey: .padTokenId)
         try c.encode(decoderStartTokenId, forKey: .decoderStartTokenId)
         try c.encode(scaleEmbedding, forKey: .scaleEmbedding)
+        try c.encodeIfPresent(quantization, forKey: .quantization)
     }
 }