Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/skills/check/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ hides every lint finding behind it — expect to fix a batch, not a queue.
Steps 1–5 are the `--portable` subset. Everything below needs a macOS toolchain:

6. `swift test` with `-warnings-as-errors`
7. engine line-coverage gate (≥80%, `Tests/` excluded — see `MIN_COVERAGE`)
7. engine line-coverage gate (≥88%, `Tests/` excluded — see `MIN_COVERAGE`)
8. ThreadSanitizer + AddressSanitizer test passes
9. xcodegen drift check (regenerating must not change the committed `.pbxproj`)
10. codesign-skipped app build (warnings-as-errors)
Expand Down
12 changes: 11 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ the two checks that genuinely need the build products stay behind them (step 12)
Everything above this line is the `--portable` subset. Everything below needs a macOS toolchain.

6. `swift test` with `-warnings-as-errors`, plus an engine **line-coverage gate** (≥ `MIN_COVERAGE`,
80%, `Tests/` excluded — raise it as coverage grows).
88%, `Tests/` excluded — raise it as coverage grows; `check.sh` records why ~91% is the ceiling).
7. **ThreadSanitizer** and **AddressSanitizer** test passes.
8. **xcodegen drift check** — regenerating must not change the committed `.pbxproj`.
9. **App build** with codesigning skipped (warnings-as-errors via `SWIFT_TREAT_WARNINGS_AS_ERRORS` in
Expand Down Expand Up @@ -859,6 +859,16 @@ weak-reference assertions in `MemoryLeakTests.swift` (`expectNoLeak`) — LeakSa
on Darwin, so AddressSanitizer catches memory _corruption_ but not leaks; `scripts/leaks.sh` covers
the whole app under the Darwin leak detector.

**Mutation testing** — `scripts/mutate.sh`, opt-in and **not** part of `check.sh`. It flips one
operator in a source file, re-runs the suite, and reports whether anything noticed. Reach for it
instead of chasing the coverage number: the engine is near its line-coverage ceiling (the remainder is
Accessibility/TCC/CGEvent code no CI process can run), so the useful question is no longer "did this
line execute?" but "is it _asserted_?" — a surviving mutant is a behaviour change no test objected
to. Its default target list is the files already at or near 100% coverage, which is exactly where the
coverage number has nothing left to say. It stays out of `check.sh` deliberately: a run is minutes,
and survivors need judgement (an _equivalent_ mutant cannot change behaviour, so no test can catch
it), and a required gate that reports unactionable failures is one people learn to skip.

**Swift Testing traps that CI has caught more than once.** Nothing here can be typechecked without a
macOS toolchain, so when you're writing tests from a Linux / web sandbox these are the ones that cost
a red run. Check them by eye before pushing:
Expand Down
4 changes: 3 additions & 1 deletion Sources/BlurtEngine/Config/APIKeyValidator.swift
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,9 @@ public struct APIKeyValidator: Sendable {

var components = URLComponents(
url: baseURL.appendingPathComponent("v2/transcript"),
resolvingAgainstBaseURL: false
// `true` would behave identically here: the URL above is already absolute, so
// there is no base to resolve it against. Equivalent mutant, not a test gap.
resolvingAgainstBaseURL: false // mutate-ok: absolute URL, nothing to resolve
)
components?.queryItems = [URLQueryItem(name: "limit", value: "1")]
guard let url = components?.url else { return .unreachable }
Expand Down
11 changes: 10 additions & 1 deletion Sources/BlurtEngine/FocusCapture/FocusCapture+Editability.swift
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,16 @@ extension FocusCapture {
/// genuinely no editable focus bundles no such framework and correctly falls
/// back to copy.
static func isElectronApp(_ app: NSRunningApplication?) -> Bool {
guard let bundleURL = app?.bundleURL else { return false }
isElectronBundle(app?.bundleURL)
}

/// Pure decision behind `isElectronApp`: does the bundle at `bundleURL` ship the
/// Electron framework? Split from the `NSRunningApplication` wrapper for the same
/// reason as `isBrowserBundleID` — the detection is then unit-testable against a
/// fixture bundle, instead of requiring an Electron app to be installed *and*
/// running on the machine under test.
static func isElectronBundle(_ bundleURL: URL?) -> Bool {
guard let bundleURL else { return false }
let electronFramework = bundleURL.appendingPathComponent(
"Contents/Frameworks/Electron Framework.framework")
return FileManager.default.fileExists(atPath: electronFramework.path)
Expand Down
25 changes: 20 additions & 5 deletions Sources/BlurtEngine/Injection/KeyInjector+SystemActions.swift
Original file line number Diff line number Diff line change
Expand Up @@ -41,17 +41,32 @@ extension KeyInjector {
AXIsProcessTrusted()
}

/// Posts Cmd-V. Returns `false` if the events couldn't be built. The real side
/// effect (a keystroke into the focused app) is why this is the injectable seam
/// tests replace.
static func postCmdV() -> Bool {
/// The Cmd-V key-down/key-up pair, or `nil` when CoreGraphics refuses to build
/// them. Split from `postCmdV` because only the *posting* is untestable: building
/// an event needs no Accessibility trust, while posting one sends a live
/// keystroke into whatever app has focus — not something `swift test` may do to
/// the machine it runs on. So the part carrying an actual invariant (the ⌘ flag
/// on both events and the `kVK_ANSI_V` keycode, which is what makes the paste a
/// paste) is asserted in `KeyInjectorSystemActionsTests`, and only the two
/// `.post` calls below stay covered by running the app.
static func cmdVEvents() -> (down: CGEvent, up: CGEvent)? {
let vKey: CGKeyCode = 0x09 // kVK_ANSI_V
guard let source = CGEventSource(stateID: .combinedSessionState),
let down = CGEvent(keyboardEventSource: source, virtualKey: vKey, keyDown: true),
let up = CGEvent(keyboardEventSource: source, virtualKey: vKey, keyDown: false)
else { return false }
else { return nil }
// Set on both: a key-up carrying no ⌘ reads as the modifier having been
// released mid-chord, which some apps treat as cancelling the shortcut.
down.flags = .maskCommand
up.flags = .maskCommand
return (down, up)
}

/// Posts Cmd-V. Returns `false` if the events couldn't be built. The real side
/// effect (a keystroke into the focused app) is why this is the injectable seam
/// tests replace.
static func postCmdV() -> Bool {
guard let (down, up) = cmdVEvents() else { return false }
// Post to the annotated session tap rather than the HID tap: the session tap
// honors exactly the flags set above instead of OR-ing in the live hardware
// modifier state, so a still-held hotkey modifier can't corrupt Cmd-V into a
Expand Down
2 changes: 1 addition & 1 deletion Sources/BlurtEngine/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,7 @@ await session.press()
await session.release()
```

Run `swift test` for the engine suites (`--filter DictationSessionTests` for one suite). `scripts/check.sh` is the full health gate CI runs — tests with warnings-as-errors, a ≥80% engine coverage gate, TSan/ASan passes, and the linters. On a machine without a macOS toolchain, `scripts/check.sh --portable` verifies docs/scripts/site changes only; the Swift side needs a Mac or CI.
Run `swift test` for the engine suites (`--filter DictationSessionTests` for one suite). `scripts/check.sh` is the full health gate CI runs — tests with warnings-as-errors, a ≥88% engine coverage gate, TSan/ASan passes, and the linters. On a machine without a macOS toolchain, `scripts/check.sh --portable` verifies docs/scripts/site changes only; the Swift side needs a Mac or CI.

## Embedding outside Blurt

Expand Down
10 changes: 10 additions & 0 deletions Tests/BlurtEngineTests/APIKeyDisplayTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,16 @@ struct APIKeyDisplayTests {
}
}

@Test("the not-connected row reads as prose, not as an identifier")
func notConnectedIsProse() {
// Every other `!rendersIdentifier` assertion here is about `.connected(nil)` — a
// short key that gets masked to bare "Connected" — so the `.notConnected` arm was
// unpinned, and flipping it to `true` survived the whole suite
// (`scripts/mutate.sh`). Monospacing "Not connected" would style a sentence as a
// value.
#expect(!APIKeyDisplay.notConnected.rendersIdentifier)
}

@Test("the threshold keeps at least half of any masked key hidden")
func thresholdIsTwiceTheTail() {
#expect(APIKeyDisplay.minimumLengthToMask == APIKeyDisplay.revealedTailLength * 2)
Expand Down
16 changes: 16 additions & 0 deletions Tests/BlurtEngineTests/BlurtErrorTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,22 @@ struct BlurtErrorTests {
#expect(BlurtError.sttFailed(underlying: a) == .sttFailed(underlying: sameIdentityOtherMessage))
}

@Test("wrapped equality requires both domain and code to match, not either")
func wrappedEqualityNeedsBothFields() {
// `wrappedEquality` above varies domain *and* code together, so it cannot tell
// `domain == … && code == …` from `||`: with `||` its unequal pair is still
// `false || false`. Varying one field at a time is what actually pins the
// conjunction. (Found by `scripts/mutate.sh` — the line was fully covered, and
// the mutation to `||` survived the whole suite.)
//
// Worth pinning rather than shrugging at: the engine tests assert error identity
// *through* this `==`, so a too-loose one wouldn't fail here — it would quietly
// weaken every `#expect(phase == .failed(.sttFailed(…)))` elsewhere.
let base = NSError(domain: "X", code: 1)
#expect(BlurtError.sttFailed(underlying: base) != .sttFailed(underlying: NSError(domain: "X", code: 2)))
#expect(BlurtError.sttFailed(underlying: base) != .sttFailed(underlying: NSError(domain: "Y", code: 1)))
}

@Test("wrapping cases of different kinds never compare equal")
func crossKindInequality() {
let e = NSError(domain: "X", code: 1, userInfo: [NSLocalizedDescriptionKey: "same"])
Expand Down
82 changes: 82 additions & 0 deletions Tests/BlurtEngineTests/BrowserBundleIDTests.swift
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import AppKit
import Foundation
import Testing

@testable import BlurtEngine
Expand Down Expand Up @@ -60,3 +62,83 @@ struct BrowserBundleIDTests {
#expect(!FocusCapture.isBrowserBundleID(nil))
}
}

/// The other half of the AX-opaque exemption: Electron detection, and the
/// `isAXOpaqueApp` disjunction the injector actually calls.
///
/// Electron apps are classified by the framework they bundle rather than by
/// bundle ID, because the set is open-ended (every Electron app ever shipped),
/// so the fixtures here are directory trees rather than identifier strings —
/// `isElectronBundle` is split out of the `NSRunningApplication` wrapper for
/// exactly that reason.
@Suite("FocusCapture AX-opaque app classification")
struct AXOpaqueAppTests {

/// An app bundle skeleton in a temp directory, with the Electron framework
/// present or absent. Only the *path* matters to the check — nothing is loaded —
/// so an empty directory at the framework's location is a faithful fixture.
private func makeBundle(withElectron: Bool) throws -> URL {
let bundle = URL.temporaryDirectory.appending(path: "Blurt-\(UUID().uuidString).app")
let contents =
withElectron
? bundle.appending(path: "Contents/Frameworks/Electron Framework.framework")
: bundle.appending(path: "Contents/Frameworks")
try FileManager.default.createDirectory(at: contents, withIntermediateDirectories: true)
return bundle
}

// MARK: isElectronBundle

@Test("a bundle shipping the Electron framework is Electron")
func electronBundleDetected() throws {
let bundle = try makeBundle(withElectron: true)
defer { try? FileManager.default.removeItem(at: bundle) }
// The true arm is what keeps VS Code and Slack on the paste path: their focused
// text fields expose no editable AX signal, so without this they'd fall back to
// copy-only and the user's words would never land.
#expect(FocusCapture.isElectronBundle(bundle))
}

@Test("a native bundle with no Electron framework is not Electron")
func nativeBundleRejected() throws {
let bundle = try makeBundle(withElectron: false)
defer { try? FileManager.default.removeItem(at: bundle) }
// The false arm matters just as much: a native app with genuinely nothing
// editable focused must fall back to copy rather than beep a ⌘V.
#expect(!FocusCapture.isElectronBundle(bundle))
}

@Test("a bundle URL that doesn't exist is not Electron")
func missingBundleRejected() {
#expect(!FocusCapture.isElectronBundle(URL(filePath: "/nonexistent/Ghost.app")))
}

@Test("a nil bundle URL is not Electron")
func nilBundleURLRejected() {
#expect(!FocusCapture.isElectronBundle(nil))
}

// MARK: NSRunningApplication wrappers

@Test("the test host is neither a browser nor Electron")
func testHostIsNotOpaque() {
// The one live `NSRunningApplication` a unit test can count on. Weak as an
// assertion about *this* process, but it pins the wrappers as pass-throughs to
// the two pure checks rather than, say, defaulting to opaque — which would make
// the injector paste into every non-editable target and beep.
let current = NSRunningApplication.current
#expect(!FocusCapture.isBrowserApp(current))
#expect(!FocusCapture.isElectronApp(current))
#expect(!FocusCapture.isAXOpaqueApp(current))
}

@Test("no app at all is not AX-opaque")
func nilAppIsNotOpaque() {
// `KeyInjector` passes its captured target, which is nil when nothing was
// captured — that must not be treated as opaque, or a paste with no known
// target would be attempted anyway.
#expect(!FocusCapture.isBrowserApp(nil))
#expect(!FocusCapture.isElectronApp(nil))
#expect(!FocusCapture.isAXOpaqueApp(nil))
}
}
38 changes: 38 additions & 0 deletions Tests/BlurtEngineTests/DictationLogTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,28 @@ import Testing
private struct DecodedEntry: Decodable {
let transcript: String
let ts: String
/// `turns` and `keyterms` are the two fields `Entry.encode(to:)` writes only when
/// non-empty, so an omitted-key line has to decode rather than throw. Empty, not
/// optional — the repo bans optional collections, and it costs nothing here:
/// "omitted" vs "written as `[]`" is asserted on the raw line by
/// `nilFieldsAreOmitted`, which is the level that contract actually lives at.
let turns: [String]
let keyterms: [String]

/// Spelled out because the custom `init(from:)` below suppresses the synthesis
/// that would otherwise derive these — mirroring `DictationLog.Entry`, which
/// states its own keys for the same reason.
enum CodingKeys: String, CodingKey {
case transcript, ts, turns, keyterms
}

init(from decoder: any Decoder) throws {
let container = try decoder.container(keyedBy: CodingKeys.self)
transcript = try container.decode(String.self, forKey: .transcript)
ts = try container.decode(String.self, forKey: .ts)
turns = try container.decodeIfPresent([String].self, forKey: .turns) ?? []
keyterms = try container.decodeIfPresent([String].self, forKey: .keyterms) ?? []
}
}

/// Each test that genuinely needs a file gets a fresh empty one in a unique temp
Expand Down Expand Up @@ -194,6 +216,22 @@ struct DictationLogTests {
#expect(!line.contains("turns"))
#expect(!line.contains("keyterms"))
}

@Test("a non-empty turns/keyterms list is written, with its values intact")
func conditionalFieldsAreWrittenWhenPresent() throws {
// The other direction of the same contract. `nilFieldsAreOmitted` pins the
// conditional arms' *skip*; without this, `encode(to:)` could stop writing
// either field entirely and only the negative test would still pass — leaving
// the corpus with no record of what steering a request carried.
let url = makeTempLogURL()
let context = TranscriptionContext(
appName: "Mail", priorText: "Hi Sam,", keyTerms: ["AssemblyAI", "LeMUR"])
DictationLog.write(transcript: "p", context: context, to: url, now: Date())

let entry = try #require(firstEntry(in: url))
#expect(entry.keyterms == ["AssemblyAI", "LeMUR"])
#expect(entry.turns == ["Hi Sam,"])
}
}

/// The shared encoder both logs write through.
Expand Down
95 changes: 95 additions & 0 deletions Tests/BlurtEngineTests/KeyInjectorSystemActionsTests.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
import AppKit
import CoreGraphics
import Testing

@testable import BlurtEngine

/// The system side of `KeyInjector`'s seams (`KeyInjector+SystemActions.swift`),
/// covering the parts that can be asserted without changing the state of the
/// machine running the suite.
///
/// The line this suite draws: *reads* of process-global state are fair game
/// (`accessibilityTrusted`, the frontmost-app poll), and so is *building* a
/// CGEvent. What is deliberately left to running the real app is anything that
/// mutates the session — `activate` steals focus, and `postCmdV`'s two `.post`
/// calls would fire a live ⌘V into whatever the developer had open. That is the
/// same reason `check.sh` runs the XCUITest suite on CI only: it commandeers the
/// GUI session.
@Suite("KeyInjector system actions")
struct KeyInjectorSystemActionsTests {

// MARK: - Cmd-V event construction

@Test("cmdVEvents builds a V key-down/key-up pair")
func cmdVEventsBuildsPair() throws {
let events = try #require(KeyInjector.cmdVEvents())

#expect(events.down.type == .keyDown)
#expect(events.up.type == .keyUp)
// 0x09 is kVK_ANSI_V. Asserted numerically because the Carbon constant isn't
// importable here, which is why the source spells it as a literal too — this
// is the check that keeps that literal honest.
#expect(events.down.getIntegerValueField(.keyboardEventKeycode) == 0x09)
#expect(events.up.getIntegerValueField(.keyboardEventKeycode) == 0x09)
}

@Test("both Cmd-V events carry the command flag")
func cmdVEventsCarryCommand() throws {
let events = try #require(KeyInjector.cmdVEvents())

// A ⌘-less key-down is a plain "v" — it types a character into the target
// instead of pasting, which is the visible failure this pins.
#expect(events.down.flags.contains(.maskCommand))
// And a ⌘-less key-up reads as the modifier having been released mid-chord.
#expect(events.up.flags.contains(.maskCommand))
}

@Test("Cmd-V events carry no modifier beyond command")
func cmdVEventsCarryNoOtherModifier() throws {
let events = try #require(KeyInjector.cmdVEvents())

// ⌘⌥V and ⌘⇧V are "paste and match style" in most apps, and ⌃⌘V is bound
// elsewhere again — so a stray extra modifier doesn't fail loudly, it pastes
// the wrong way. `flags` is assigned (not OR-ed) in `cmdVEvents`, and this is
// what keeps it that way.
for flags in [events.down.flags, events.up.flags] {
#expect(!flags.contains(.maskAlternate))
#expect(!flags.contains(.maskShift))
#expect(!flags.contains(.maskControl))
#expect(!flags.contains(.maskSecondaryFn))
}
}

// MARK: - Accessibility trust probe

@Test("accessibilityTrusted reports the process-wide AX trust state")
func accessibilityTrustedMatchesSystem() {
// The test host's trust state isn't ours to set, so the assertable claim is
// that the seam is a pass-through and not, say, a hard-coded `true` that would
// make `KeyInjector` skip its permission check in production.
#expect(KeyInjector.accessibilityTrusted() == AXIsProcessTrusted())
}

// MARK: - Frontmost wait

@Test("waitUntilFrontmost reports failure for an app that never comes frontmost")
func waitUntilFrontmostGivesUp() async {
// The test host is a command-line process with no windows, so the window
// server never reports it frontmost — the deterministic "activation didn't
// land" case. `KeyInjector.activateTargetApp` turns this `false` into
// `.targetAppLost` rather than pasting into the wrong app.
#expect(await KeyInjector.waitUntilFrontmost(.current) == false)
}

@Test("waitUntilFrontmost gives up on a bounded deadline instead of hanging")
func waitUntilFrontmostIsBounded() async {
// Sits on the press→paste path, so an unbounded wait would freeze the paste,
// not just slow it. 350 ms budget; the ceiling leaves room for a loaded CI
// box's scheduling without being loose enough to pass an unbounded loop.
let clock = ContinuousClock()
let elapsed = await clock.measure {
_ = await KeyInjector.waitUntilFrontmost(.current)
}
#expect(elapsed < .seconds(3))
}
}
Loading
Loading