Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 11 additions & 6 deletions .claude/skills/project-guardrails/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,13 @@ one, stop and ask the user first. This is the fast "don't" list; AGENTS.md's

- **No streaming STT.** The AssemblyAI Sync API returns the full transcript in
one response. Overlay goes "Transcribing…" → full text.
- **No separate LLM cleanup pass.** Cleanup rides in the Sync STT request's
`config.prompt` (`TranscriptionPrompt`). No LLM Gateway client, no
`StylerProtocol`, no post-transcription styling stage.
- **No separate/client-side LLM cleanup pass.** Cleanup is the dictation API's
server-side rewrite, requested by the `llm` block on the same `/transcribe`
call — one request, no second round-trip. No LLM Gateway client, no
`StylerProtocol`, no post-transcription styling stage. The block's
`instruction` is now user-editable (`CleanupPromptStore`, sent as
`llm.instruction`; blank = the service default) and requested per-press by the
cleaned trigger, but it is still one server-side request.
- **No local models / model downloads.** Transcription is a remote AssemblyAI
call. No on-device ASR/LLM, no model cache, no download UI.
- Don't reintroduce a "remove filler words (um, uh, like)" directive in the
Expand All @@ -45,9 +49,10 @@ one, stop and ask the user first. This is the fast "don't" list; AGENTS.md's
menu bar, so nothing may depend on it being visible. A menu-bar-_only_ variant
(no Dock icon) was tried and reverted twice for that reason — don't drop the
Dock icon or add `LSUIElement`.
- The dictation trigger is a **single lone modifier** (right ⌘ default), home-
grown via `CGEventTap` + `DictationKeyGate`. No `KeyboardShortcuts` package, no
key+modifier chord.
- The dictation trigger is **two lone modifiers** — a cleaned key (right ⌘
default) and a raw key (right ⌥ default) — home-grown via `CGEventTap` +
`DictationKeyGate` (routed by `DualTriggerRouter`). Two lone-modifier keys are
fine; no `KeyboardShortcuts` package, no key+modifier chord.
- **Updates are download-only** — check → open the DMG in the browser → the user
installs it. The `mxcl/AppUpdater` dependency and its in-place self-updater
were removed; don't reintroduce a self-replacing install path, a timer-driven
Expand Down
141 changes: 80 additions & 61 deletions AGENTS.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion App/Blurt/Blurt/AppCoordinator.swift
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ final class AppCoordinator {
private func startDictationDriver() {
let session = session
keyTap = DictationKeyTap(
onStart: { session.submit(.press) },
onStart: { mode in session.submit(.press(mode)) },
onStop: { session.submit(.release) },
onCancel: { session.submit(.cancel) },
// Recovery-only teardown; `cancelRecording()`'s doc owns the rationale.
Expand Down
94 changes: 60 additions & 34 deletions App/Blurt/Blurt/Hotkey/DictationKeyTap.swift
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,22 @@ import BlurtEngine
import CoreGraphics
import os

/// Drives the single lone-modifier dictation trigger from a `CGEventTap`.
/// Drives the **two** lone-modifier dictation triggers from a `CGEventTap`.
///
/// Watches `flagsChanged` for the bound modifier (e.g. right ⌘, keycode 54) to
/// detect down/up, and `keyDown` for any *other* key to spot a modifier combo
/// (⌘C, ⌘V…). The per-event decision lives in the engine — `DictationKeyRouter`
/// (keycode relevance + down/up edge dedup) over `DictationKeyGate` (tap/hold
/// semantics) — so this type only reduces each `CGEvent` to a router event and
/// owns the tap lifecycle.
/// One key produces the raw verbatim transcript, the other the server-side
/// cleanup rewrite; they share one gate, so only one dictation runs at a time
/// and the key that started it decides the `DictationMode` handed to `onStart`.
/// Watches `flagsChanged` for either bound modifier (e.g. right ⌥ raw, right ⌘
/// cleaned) to detect down/up, and `keyDown` for any *other* key to spot a
/// modifier combo (⌘C, ⌘V…). The per-event decision lives in the engine —
/// `DualTriggerRouter` (keycode relevance, per-key edge dedup, and single-gate
/// ownership) over `DictationKeyGate` (tap/hold semantics) — so this type only
/// reduces each `CGEvent` to a router event and owns the tap lifecycle.
///
/// Unlike the old chord trigger, this **swallows nothing**: a lone modifier
/// types nothing into the focused app, and combos must pass through so normal
/// shortcuts keep working. The tap is therefore created `.listenOnly` — an
/// Like the old single trigger and unlike a chord, this **swallows nothing**: a
/// lone modifier types nothing into the focused app, and combos must pass
/// through so normal shortcuts keep working. The tap is therefore created
/// `.listenOnly` — an
/// active (`.defaultTap`) tap would make macOS synchronously wait on this
/// process before delivering every keystroke system-wide, so any main-thread
/// stall in Blurt would add typing latency in *other* apps.
Expand All @@ -28,7 +32,7 @@ final class DictationKeyTap {
private static let logger = Logger(
subsystem: BlurtIdentity.subsystem, category: "DictationKeyTap")

private let onStart: @Sendable () -> Void
private let onStart: @Sendable (DictationMode) -> Void
private let onStop: @Sendable () -> Void
private let onCancel: @Sendable () -> Void
/// Fired when a *state-recovery* reset (disabled-tap recovery, trigger
Expand All @@ -40,12 +44,17 @@ final class DictationKeyTap {
/// a transcript already in flight — see `DictationSession.cancelRecording`.
private let onRecordingDiscarded: @Sendable () -> Void

/// The engine-side event router (keycode relevance, down/up edge dedup, and
/// the gate's tap/hold state machine — all unit-tested in BlurtEngine).
private var router = DictationKeyRouter(triggerKeyCode: TriggerKey.rightCommand.keyCode)
/// The bound key's device-dependent `CGEventFlags` bit — the one CoreGraphics-
/// The engine-side event router (keycode relevance, per-key down/up edge
/// dedup, single-gate ownership, and the gate's tap/hold state machine — all
/// unit-tested in BlurtEngine). `refreshBinding()` syncs the keycodes to the
/// persisted raw/cleaned stores.
private var router = DualTriggerRouter(
rawKeyCode: TriggerKey.rightOption.keyCode,
cleanedKeyCode: TriggerKey.rightCommand.keyCode)
/// Each bound key's device-dependent `CGEventFlags` bit — the CoreGraphics-
/// typed piece of the binding, so it stays here rather than in the router.
private var triggerFlag = DictationKeyTap.flag(for: .rightCommand)
private var rawFlag = DictationKeyTap.flag(for: .rightOption)
private var cleanedFlag = DictationKeyTap.flag(for: .rightCommand)

/// Monotonic reference; per-event timestamps are `reference.duration(to: now)`.
private let reference = ContinuousClock.now
Expand All @@ -57,7 +66,7 @@ final class DictationKeyTap {
nonisolated(unsafe) private var tap: CFMachPort?

init(
onStart: @escaping @Sendable () -> Void,
onStart: @escaping @Sendable (DictationMode) -> Void,
onStop: @escaping @Sendable () -> Void,
onCancel: @escaping @Sendable () -> Void,
onRecordingDiscarded: @escaping @Sendable () -> Void
Expand Down Expand Up @@ -149,14 +158,18 @@ final class DictationKeyTap {
if router.reset() { onRecordingDiscarded() }
}

/// Re-read the bound trigger key into the router. Call after the user
/// rebinds. The router's reset reports a discarded live recording: rebinding
/// mid-dictation means the old key's up-event will never match, so the capture
/// must be cancelled, not left to run out the auto-release cap.
/// Re-read both bound trigger keys into the router. Call after the user
/// rebinds either. The router's reset reports a discarded live recording:
/// rebinding mid-dictation means the old key's up-event will never match, so
/// the capture must be cancelled, not left to run out the auto-release cap.
func refreshBinding() {
let key = TriggerKeyStore().triggerKey
triggerFlag = Self.flag(for: key)
if router.rebind(triggerKeyCode: key.keyCode) { onRecordingDiscarded() }
let rawKey = RawTriggerKeyStore().triggerKey
let cleanedKey = TriggerKeyStore().triggerKey
rawFlag = Self.flag(for: rawKey)
cleanedFlag = Self.flag(for: cleanedKey)
if router.rebind(rawKeyCode: rawKey.keyCode, cleanedKeyCode: cleanedKey.keyCode) {
onRecordingDiscarded()
}
}

/// Callback entry point (always on the main thread — the tap's source lives on
Expand All @@ -171,8 +184,10 @@ final class DictationKeyTap {
// here would discard speech the user is mid-sentence on. Otherwise the
// trigger's key-up may have been missed; reset, and cancel a recording
// the reset discards rather than leaving the session in .recording until
// the auto-release cap fires and pastes an unprompted transcript.
if CGEventSource.flagsState(.combinedSessionState).contains(triggerFlag) { return }
// the auto-release cap fires and pastes an unprompted transcript. Either
// trigger being held means a live capture whose key-up is still coming.
let held = CGEventSource.flagsState(.combinedSessionState)
if held.contains(rawFlag) || held.contains(cleanedFlag) { return }
if router.reset() { onRecordingDiscarded() }
return
}
Expand All @@ -183,22 +198,29 @@ final class DictationKeyTap {
}

/// Reduces a `CGEvent` to the router's CoreGraphics-free event shape, or nil
/// for event types the trigger doesn't care about.
private func routerEvent(type: CGEventType, event: CGEvent) -> DictationKeyRouter.Event? {
/// for event types the trigger doesn't care about. A `flagsChanged` carries
/// both keys' device-bit states; the router picks the one matching the
/// event's keycode.
private func routerEvent(type: CGEventType, event: CGEvent) -> DualTriggerRouter.Event? {
let keyCode = Int(event.getIntegerValueField(.keyboardEventKeycode))
switch type {
case .flagsChanged:
return .flagsChanged(keyCode: keyCode, triggerFlagIsOn: event.flags.contains(triggerFlag))
return .flagsChanged(
keyCode: keyCode,
rawFlagIsOn: event.flags.contains(rawFlag),
cleanedFlagIsOn: event.flags.contains(cleanedFlag))
case .keyDown:
return .keyDown(keyCode: keyCode)
default:
return nil
}
}

private func dispatch(_ action: DictationKeyGate.Action) {
switch action {
case .start: onStart()
private func dispatch(_ outcome: DualTriggerRouter.Outcome) {
switch outcome.action {
// `.start` carries the owning mode (raw vs cleaned); default to `.cleaned`
// for the impossible nil so the call site stays total.
case .start: onStart(outcome.mode ?? .cleaned)
case .stop: onStop()
case .cancel: onCancel()
case .none: break
Expand Down Expand Up @@ -226,15 +248,19 @@ final class DictationKeyTap {
_ = router.reset()
dispatch(
router.handle(
.flagsChanged(keyCode: router.triggerKeyCode, triggerFlagIsOn: true), at: .seconds(0)))
.flagsChanged(
keyCode: router.cleanedKeyCode, rawFlagIsOn: false, cleanedFlagIsOn: true),
at: .seconds(0)))
}

/// Completes the synthetic cycle as a hold (past the threshold), so the gate
/// emits `.stop` and `onStop` fires.
func simulateReleaseForTesting() {
dispatch(
router.handle(
.flagsChanged(keyCode: router.triggerKeyCode, triggerFlagIsOn: false), at: .seconds(2)))
.flagsChanged(
keyCode: router.cleanedKeyCode, rawFlagIsOn: false, cleanedFlagIsOn: false),
at: .seconds(2)))
}
#endif
}
Expand Down
4 changes: 3 additions & 1 deletion App/Blurt/Blurt/UITestSupport.swift
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,9 @@
/// Stub transcriber: returns the harness's canned transcript, so the "spoken"
/// text is whatever the test set — no network, fully deterministic.
nonisolated struct UITestTranscriber: TranscriberProtocol {
func transcribe(pcm: Data, sampleRate: Int, context: TranscriptionContext?) async throws -> String {
func transcribe(
pcm: Data, sampleRate: Int, context: TranscriptionContext?, cleanup: Bool
) async throws -> String {
await MainActor.run { UITestState.shared.cannedTranscript }
}
}
Expand Down
54 changes: 38 additions & 16 deletions App/Blurt/Blurt/Wizard/SettingsWindowRoot.swift
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ private struct GeneralSettingsTab: View {
}
}

/// The occasional stuff: the enhanced-transcripts switch, checking for an
/// The occasional stuff: the cleanup-instruction editor, checking for an
/// update, and the developer-mode log toggle. Kept out of General so the
/// common pane stays short.
private struct AdvancedSettingsTab: View {
Expand All @@ -82,29 +82,51 @@ private struct AdvancedSettingsTab: View {
}
}

/// The Transcription section of the Settings window: the enhanced-transcripts
/// switch. While on (the default), every dictation request asks AssemblyAI's
/// dictation API for its server-side cleanup rewrite, so the pasted text is
/// the polished version; turned off, the request omits the rewrite and the
/// verbatim transcript is pasted exactly as spoken. The transcriber reads the
/// same default this toggle writes at every request, so a change applies to
/// the next dictation. Settings-only — not a wizard step, since it never
/// gates setup.
/// The Transcription section of the Settings window: the editable cleanup
/// instruction. Whatever is typed here is sent as the dictation request's
/// `llm.instruction` on a *cleaned* dictation (the cleaned-up trigger key), so
/// the user can steer the server-side rewrite. Leaving it blank sends an empty
/// `llm` block, which selects AssemblyAI's own default cleanup rewrite. The
/// transcriber reads the same default this field writes at every request, so a
/// change applies to the next dictation. Settings-only — not a wizard step,
/// since it never gates setup.
private struct TranscriptionSection: View {
@AppStorage(EnhancedTranscriptsStore.defaultsKey) private var enhancedTranscripts = true
// Observe-only: `@AppStorage` re-renders the view when the slot changes (default
// "" so an unset prompt reads as empty). The field writes *through*
// `CleanupPromptStore`, not this slot directly — the store owns the cap and the
// blank → remove-key rule, and is the setter's only production caller (so a
// change to how the prompt is persisted can't leave the setter untested), the
// same pattern `HotkeyStepView` uses for the trigger keys.
@AppStorage(CleanupPromptStore.defaultsKey) private var storedPrompt = ""

private var prompt: Binding<String> {
Binding(
get: { storedPrompt },
set: { CleanupPromptStore().instruction = $0 })
}

var body: some View {
Section {
Toggle(isOn: $enhancedTranscripts) {
Label("Enhanced transcripts", systemImage: "wand.and.stars")
// A vertical-axis TextField grows with content up to `lineLimit`; a
// `@ViewBuilder` label + `labelsHidden()` renders it full-width (like the
// Key Terms field) rather than squeezed into a leading label column.
TextField(
text: prompt,
prompt: Text("Leave blank to use the default cleanup"),
axis: .vertical
) {
Text("Rewrite instruction")
}
.accessibilityIdentifier(UITestIdentifiers.enhancedTranscriptsToggle)
.labelsHidden()
.lineLimit(2...6)
.accessibilityIdentifier(UITestIdentifiers.cleanupPromptField)
} header: {
Text("Transcription")
Text("Cleanup")
} footer: {
Text(
"Polishes each dictation before pasting — removing filler words and fixing punctuation. "
+ "Turn off to paste your words exactly as spoken.")
"The instruction sent to the cleanup rewrite for the cleaned-up dictation key — for "
+ "example, \"Fix punctuation and remove filler words.\" Leave it blank to use "
+ "AssemblyAI's default cleanup.")
}
}
}
Expand Down
Loading