docs: add an evals README, move the engine guide into the package (#137)

Two documentation moves, both about putting the guide next to the thing it
documents.

`evals/` had no directory-level README — only the harness's own, one level
down. Add one that says what the directory is (offline decision support, the
repo's only Python, none of it shipped), indexes the harnesses, and records how
check.sh gates it: ruff format/check over evals/ and pytest over the one named
test file, all three in the --portable subset. That last detail is the trap a
second harness would hit — the pytest step names a file rather than globbing,
so new tests have to be added to check.sh in the same commit or they never run.

`BLURTENGINE.md` becomes `Sources/BlurtEngine/README.md`, so the engine's
developer guide sits in the package it describes and renders as the landing
page for anyone browsing there. Its two outbound links are repointed at the
root, the inbound links in README.md and AGENTS.md follow it, and the
repository map in AGENTS.md gains both files. Package.swift excludes the new
README from the target: SwiftPM has no rule for a stray .md inside a target
directory, so an undeclared one lands in the unhandled-files list.

scripts/check.sh --portable is green (prettier, markdownlint, ruff, pytest);
the Swift half needs a Mac, so CI is the authority there.


Claude-Session: https://claude.ai/code/session_015BBN7mYHeENP9sJc3TNjZW

Co-authored-by: Claude <noreply@anthropic.com>

Author: alexkroman

AGENTS.md

@@ -43,13 +43,14 @@ and the rebindable dictation trigger is home-grown (`CGEventTap` over a pure sta
 package.
 
 For embedding or extending the engine's public API — seam contracts, error table, quick start —
-see [`BLURTENGINE.md`](./BLURTENGINE.md). This file covers repo-wide workflow and the _why_ behind
-the design; BLURTENGINE.md covers the _what_ of the API surface.
+see [`Sources/BlurtEngine/README.md`](./Sources/BlurtEngine/README.md). This file covers repo-wide
+workflow and the _why_ behind the design; the engine's README covers the _what_ of the API surface.
 
 ## Repository map
 
 ```text
 Sources/BlurtEngine/         the engine (dependency-free Swift package)
+  README.md                  the engine's developer guide (quick start, seams, error table)
   Audio/                     MicCapture (+meter), SoundPack/Catalog/Store — record cues
   Config/                    Keychain-backed API key, key terms, developer mode, DefaultsKey +
                              PersistedSettings (every defaults key, and the reset sweep over them)
@@ -79,10 +80,11 @@ scripts/                     check.sh, check-site.sh, bootstrap.sh, dev-build.sh
                              generate-sounds.swift (regenerates the cues AND
                              SoundPackCatalog.swift together)
 Brewfile                     Homebrew-managed check.sh tools — the whole toolchain
-evals/dictation-prompt/      offline DSPy harness for tuning the dictation API's cleanup
-                             instruction — nothing here ships in the app, but check.sh does
-                             lint (ruff), format-check (ruff format), and test (pytest) it
-evals/ruff.toml              ruff config for the above — the repo's only Python config
+evals/                       offline decision support — the repo's only Python, none of it
+                             shipped (see evals/README.md)
+  dictation-prompt/          DSPy harness for tuning the dictation API's cleanup instruction;
+                             check.sh lints (ruff), format-checks (ruff format), and tests it
+  ruff.toml                  ruff config for the above — the repo's only Python config
 site/                        the GitHub Pages site (html/css, sitemap) — formatted by prettier and
                              checked for deployability by scripts/check-site.sh
 .github/workflows/           check.yml (the gate, macos-26),

Package.swift

@@ -9,7 +9,11 @@ let package = Package(
   ],
   targets: [
     .target(
-      name: "BlurtEngine"
+      name: "BlurtEngine",
+      // The engine's developer guide lives next to the code it documents. SwiftPM
+      // has no rule for a stray .md inside a target, so declare it excluded rather
+      // than let it land in the target's unhandled-files list.
+      exclude: ["README.md"]
     ),
     .testTarget(
       name: "BlurtEngineTests",

README.md

@@ -207,8 +207,9 @@ App/Blurt/               AppKit/SwiftUI shell (Xcode project generated by XcodeG
 
 The engine is a standalone package you can embed to build your own dictation
 app — mic capture, dictation-API transcription, and paste-into-the-focused-app behind
-three protocol seams, fully stubbed in tests. [`BLURTENGINE.md`](./BLURTENGINE.md)
-is the developer guide.
+three protocol seams, fully stubbed in tests.
+[`Sources/BlurtEngine/README.md`](./Sources/BlurtEngine/README.md) is the
+developer guide.
 
 Latency note: perceived speed is mostly bookkeeping. `press()` warms up the
 HTTPS connection and kicks off the focused-field context read without awaiting

BLURTENGINE.md Sources/BlurtEngine/README.mdBLURTENGINE.md renamed to Sources/BlurtEngine/README.md

@@ -1,6 +1,6 @@
 # Building on BlurtEngine
 
-BlurtEngine is the Swift package that powers [Blurt](README.md)'s dictation pipeline: capture speech from the microphone, transcribe it in a single AssemblyAI dictation API call (transcription plus a server-side LLM cleanup rewrite; the request carries the audio and nothing about the user's screen), and paste the polished text into the focused app. This guide is for developers embedding the engine in their own macOS app or extending it inside this repository. For repo-wide conventions and agent workflow, see [AGENTS.md](AGENTS.md).
+BlurtEngine is the Swift package that powers [Blurt](../../README.md)'s dictation pipeline: capture speech from the microphone, transcribe it in a single AssemblyAI dictation API call (transcription plus a server-side LLM cleanup rewrite; the request carries the audio and nothing about the user's screen), and paste the polished text into the focused app. This guide is for developers embedding the engine in their own macOS app or extending it inside this repository. For repo-wide conventions and agent workflow, see [AGENTS.md](../../AGENTS.md).
 
 ## What you get
 
@@ -24,7 +24,7 @@ targets: [
 ]
 ```
 
-A local checkout works the same way with `.package(path: "../blurt")`. One thing to know before you pin a version: the tags are **Blurt's app releases**, minted by the DMG pipeline in [RELEASE.md](RELEASE.md), not independent engine releases — a patch bump says nothing about whether the engine changed. Read [Embedding outside Blurt](#embedding-outside-blurt) before shipping it inside another app.
+A local checkout works the same way with `.package(path: "../blurt")`. One thing to know before you pin a version: the tags are **Blurt's app releases**, minted by the DMG pipeline in [RELEASE.md](../../RELEASE.md), not independent engine releases — a patch bump says nothing about whether the engine changed. Read [Embedding outside Blurt](#embedding-outside-blurt) before shipping it inside another app.
 
 Then compose a session:
 
@@ -280,7 +280,7 @@ Two ways out, neither taken yet: drop it from the public product (its own target
 
 ## Invariants — don't break these
 
-Each of these was tried the other way and reverted; the longer stories are in [AGENTS.md](AGENTS.md) and the source comments:
+Each of these was tried the other way and reverted; the longer stories are in [AGENTS.md](../../AGENTS.md) and the source comments:
 
 - **No external SPM dependencies in the engine.** Foundation/Security/AVFoundation only.
 - **No streaming STT, no local models, no client-side LLM cleanup pass.** One dictation request per utterance is the architecture; the cleanup rewrite is server-side (the request's `llm` block), and transcription steering belongs in `ConversationContext`.

evals/README.md

@@ -0,0 +1,50 @@
+# Evals
+
+Offline harnesses that decide things about Blurt's behavior before the decision reaches Swift.
+This is the repo's only Python, and **none of it ships in the app** — a run's output is a measured
+artifact (today: one instruction string) that a human copies into the engine.
+
+| Directory                                           | What it decides                                                                                                                                                                                                              |
+| --------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| [`dictation-prompt/`](./dictation-prompt/README.md) | The dictation API's server-side cleanup instruction — `config.llm.instruction`, the string `CleanupInstruction.text` carries. A DSPy/GEPA search over candidate instructions, scored on disfluent-to-clean transcript pairs. |
+
+Each harness documents its own defaults, corpora, and what its numbers can and cannot establish.
+Read that README before reading a result: the ceilings here are set by the corpus and by the fact
+that we score a stand-in model rather than the service's own rewrite model, and both READMEs say so
+where it matters.
+
+## It is gated like shipped code
+
+`scripts/check.sh` runs three checks over this directory, and CI runs the same ones:
+
+| Check                 | Scope                                 | Fix                       |
+| --------------------- | ------------------------------------- | ------------------------- |
+| `ruff format --check` | `evals/`                              | `ruff format evals/`      |
+| `ruff check`          | `evals/`                              | `ruff check --fix evals/` |
+| `pytest -q`           | `evals/dictation-prompt/test_eval.py` | fix the test or the code  |
+
+All three are platform-independent, so they also run in `scripts/check.sh --portable` — an eval
+change can be verified off-Mac, unlike anything touching Swift. A harness whose own correctness is
+unchecked is a bad instrument, which is why non-shipping code is gated at all.
+
+Two things follow from how that gate is wired:
+
+- **`ruff.toml` is scoped to this directory**, not the repo root — Blurt is a Swift project that
+  happens to contain some Python. Ruff finds it by walking up from each file, so `ruff check evals/`
+  from the repo root picks it up. The config comments explain the line width and the `src` setting.
+- **The pytest step names one file.** A second harness's tests are not picked up by wildcard; add
+  them to the `pytest` invocation in `scripts/check.sh` in the same commit that adds them, or they
+  are decoration.
+
+## Running one
+
+The scripts carry [PEP 723](https://peps.python.org/pep-0723/) headers, so `uv run` installs their
+dependencies into a throwaway environment — nothing to set up, no repo-level lockfile or virtualenv:
+
+```bash
+uv run evals/dictation-prompt/optimize_cleanup_prompt.py --out results.json
+```
+
+The tests and the `--dry-run` paths deliberately import nothing outside the standard library
+(`pytest` aside), so they stay runnable with plain `python3` and no API key. A real run costs paid
+model calls; each harness's README says how many and what the defaults commit you to.