Short, dated notes on what shipped and how it was verified. One entry per merged branch (SPEC §9.4 "demo note"). Newest first.
What: A Data tab — the read-only half of governance: a capture-audit / transparency view showing what the engine has captured recently and its cleanup policy.
How: new adapter getRetentionStatus + retentionStatusSchema (GET /retention/status,
verified live). components/Audit.tsx: a "Last hour / Today" range → screen-frame count, audio
count, active minutes (reuses /search totals + /activity-summary), top apps, and the engine
retention policy read-only. Extracted lib/activity.ts topAppsByMinutes/fmtMinutes shared with
the ActivityStrip (removed the duplication).
Verified: 36 adapter unit tests (1 new); contract suite green live against pinned 0.4.25 incl. a
/retention/status assertion; desktop typecheck + vite build + eslint + prettier clean. Owner ran
tauri:dev and confirmed the Data tab shows real counts + retention status.
Probed for 7b (destructive half): /data/delete-range takes {start, end} → per-table deleted
counts; /retention/configure takes {mode, retention_days, enabled}; /retention/run → {success}.
Next: feat/m1-retention-delete (7b) — confirm-delete a time range + retention controls
(/security-review); finishes M1's "delete a time range confidently" exit criterion.
What: The §2.3 engine capture settings, owned by Siphon and applied as screenpipe record
flags: screen/audio capture toggles (--disable-vision/--disable-audio), video quality
(--video-quality), transcription model (--audio-transcription-engine), and ignored windows
(--ignored-windows). Persisted in the store under engine.config; a "Save & apply" restarts the
engine (if running) to pick them up.
How: engine/capture_config.rs (CaptureConfig + to_record_args, allowlist-validated,
serde-defaulted); supervisor::for_engine appends the flags; start_engine reads engine.config;
new engine_reload command drops + rebuilds the supervisor (restarts only if it was running).
Settings tab gained the "Engine capture" section. Flags only mapped to those verified in 0.4.25's
record --help (see docs/engine-notes.md).
Verified: 5 capture_config unit tests + 31 Rust lib tests; clippy + rustfmt clean; desktop
typecheck + vite build + eslint + prettier clean. Owner ran tauri:dev and confirmed a setting
change restarts the engine (green) and turning off screen capture stops new frames. /security-review
(inline): no findings — argv not shell, allowlist-validated, local-capture-only. /code-review: no
findings.
Deferred to a follow-up (6c): work-hours schedule (auto start/stop the engine) + engine CPU/RAM monitor — a scheduler/OS-sampling concern, not flag-driven.
Next: feat/m1-governance (slice 7) — retention + capture-audit view + confirm-delete range;
finishes the "delete a time range confidently" M1 exit criterion.
What: Siphon's own durable store — the first persistence Siphon owns, separate from the
engine's disposable capture DB (SPEC §2.2 rule 4: wiping ~/.screenpipe must never lose Siphon
data). Plus a Settings tab.
How: src-tauri/src/store — rusqlite (bundled SQLite) at ~/.siphon/siphon.sqlite in WAL mode,
with an append-only versioned-SQL migration runner keyed on PRAGMA user_version (v1 = the
settings(key, value_json) table). Opened + migrated at app setup, held in Tauri state. Commands:
setting_get/setting_set (JSON values) + store_info (path + schema version). The Settings tab
persists an "Open Siphon to" preference (applied on launch) and shows where the store lives.
Foundation for the rest of §2.3 (6b), retention (slice 7), and M2+ speakers/notes/rollups.
Verified: 4 Rust store tests incl. persists-across-reopen with no double-migration; 26 Rust
lib tests; clippy + rustfmt clean; desktop typecheck + vite build + eslint + prettier clean.
Owner ran tauri:dev, changed the pref, fully quit + relaunched, and confirmed it opened to the
saved tab — durable self-owned memory proven. /code-review (1 fix: settings save reverts on
failure) + /simplify: done.
Next: feat/m1-settings-engine (6b) — §2.3 engine-config settings (capture toggles, video
quality/perf, whisper model size, work-hours) plumbed store → supervisor flags → engine restart;
engine CPU/RAM readout. Gets a /security-review.
What: Tagging, shaped around what screenpipe 0.4.25 actually supports (all live-probed): a "Add a tag…" box on the selected timeline frame, and a Tag filter in Search whose results show each frame's tags as removable chips.
Engine reality (probed): tag path is /tags/vision/{frameId} (NOT ocr — /tags/ocr/{id}
400s), body {tags:[…]}, resp {success:true}. /search?tags=X filters and those results populate
content.tags — but a plain search returns tags:[] even for tagged frames, and no endpoint
reads a frame's tags (/frames/{id}/metadata|context|text omit them; GET /tags/vision/{id} is
405). So tags are a label + filter feature, not a per-frame display. The timeline box therefore
shows only tags added this session (add/undo) and doesn't fake a full list — avoiding the
"it vanished" confusion from the earlier stop-behavior. Owner accepted this limitation.
How: adapter addFrameTags/removeFrameTags (POST/DELETE /tags/vision/{id}) + a tags
filter on search(). DELETE-with-body rides the bridge's already-allowlisted DELETE (slice 2) — no
bridge/security change.
Verified: 35 adapter unit tests green (2 new); contract suite green live against pinned 0.4.25
incl. a new tags-filter assertion; desktop typecheck + vite build + eslint + prettier clean.
/code-review + /simplify: no changes.
Next: feat/m1-settings-store (slice 6) — bootstrap the Siphon store (~/.siphon/siphon.sqlite
- migration runner +
settingstable) and the §2.3 settings UI.
What: A compact activity strip above the timeline — total active minutes, frames captured,
and the top apps by time for the selected day, from /activity-summary. Closes slice 4.
How: new adapter getActivitySummary + activitySummarySchema. Live-probing found the endpoint
requires start_time/end_time (HTTP 400 without) — so the params are typed as required and the
contract test passes a range. ActivityStrip fetches the day's summary, aggregates windows → top
apps by minutes, and renders nothing until it has data. Extracted lib/day.ts dayBounds shared by
Timeline + strip (removed the duplication).
Verified: 33 adapter unit tests green (3 new); contract suite green live against pinned 0.4.25
incl. a new /activity-summary assertion (with a range); desktop typecheck + vite build + eslint +
prettier clean. Owner ran tauri:dev and confirmed the strip renders real per-app time.
/code-review + /simplify: no changes.
Next: feat/m1-tags (slice 5) — tag CRUD via engine /tags/* + filter-by-tag (probe the tag
endpoint's content-type path first).
What: A Search view (new tab): full-text query + filters (app, window, URL, date range,
content type) over captured history via the adapter's search() through the key-safe bridge.
Screen-frame (ocr) results have "Open ↗" that deep-links into the Timeline — it opens
that day focused on the exact moment and auto-selects the frame. This completes the "find any
moment from yesterday in <10s" half of the M1 exit criterion.
How: the Timeline gained target/onTargetConsumed — a deep-link loads the window ending at
the target moment (so the frame is on the first page under the engine's newest-first order) and
selects it; "Show full day" clears the focus. No adapter/schema changes — every filter was already
supported (verified live on 0.4.25: q, app_name, date bounds). Split from the activity strip
(4b) to stay reviewable.
Verified: 30 adapter unit tests + typecheck + vite build + eslint + prettier clean. Owner ran
tauri:dev and confirmed search + filters work and "Open ↗" lands on the correct frame (which
also confirms the engine returns results newest-first). /code-review + /simplify: no changes.
Next: feat/m1-activity (4b) — /activity-summary adapter + an activity strip on the timeline
(active minutes + per-app time; shape already probed live).
What: The first feature screen — a Timeline view (Home/Timeline tabs in the shell). Pick a
day; see its captured screen frames as a scrollable list of thumbnails (time + app/window), with
arrow-key navigation and a detail pane (large screenshot + OCR text). Frames are read via the
adapter's search() through the key-safe bridge — no new engine plumbing.
Live-probing paid off (validated against the pinned 0.4.25 before writing any schema):
- 0.4.25 has no
vision/uicontent_type — the engine 400s on them. Screen frames areocr(a frame is an OCR record). Fixed the adapter'sContentTypeunion to the real variants (all|ocr|audio|input|accessibility|memory) and added the observedframe_name/text_sourcefields. This would have broken every frame query on the real machine had we trusted the docs. GET /frames/{id}returns a rawimage/jpeg(not JSON). The timeline sidesteps it by usinginclude_frames=true, whose base64content.framerides the existing JSON bridge unchanged.
Design: time-cursor pagination (end_time = oldest loaded frame), not offset, so "Load older"
is correct regardless of the engine's default ordering; list sorted newest-first client-side;
end-of-data detected from a short page; a request-sequence guard prevents a slow earlier fetch from
clobbering a newer one.
Verified: 30 adapter unit tests green; contract suite green live against pinned 0.4.25 incl. a
new content_type=ocr assertion (locks the value in); desktop typecheck + vite build + eslint +
prettier clean. Owner ran tauri:dev and confirmed real screenshots + OCR text render and
keyboard nav works. /code-review (3 findings, all fixed pre-merge: day-nav race guard,
end-of-data detection, and — owner-reported — keeping the loaded timeline visible when the engine
is stopped instead of wiping it) + /simplify (no changes).
Next: feat/m1-search — search UI + filters (app/window/url/speaker/time) + activity strip,
with results deep-linking into this timeline.
What: The webview now reads the engine through the TS adapter — with the API key staying
entirely in Rust. engine/bridge.rs adds the engine_fetch Tauri command: the webview passes
only method/path/query/body, Rust attaches the supervisor's Keychain SCREENPIPE_API_KEY and
forwards to the loopback engine. Host is pinned to 127.0.0.1 via Url::set_path (SSRF-safe by
construction), methods are allowlisted (GET/POST/PUT/DELETE), timeout is clamped to 60s, and the
command is gated on the engine being Running. The webview transport
(apps/desktop/src/engine/{transport,client}.ts) implements the adapter's Transport over
engine_fetch, so @siphon/engine-adapter stays the one typed boundary and the key never enters
JavaScript (rule #4, §2.3). A "Today" panel in the window reads search() through the adapter
and shows total captured items today + the most recent item.
Why this slice: the security-sensitive key-routing path, reviewable on its own, and the first real engine data in the UI — proves the whole pipe before any feature UI is built on it.
Verified: 6 new bridge unit tests (method allowlist, URL host-pinning, query encoding) +
22 existing Rust tests green; clippy + rustfmt clean. Gated live test
forward_carries_auth_to_live_engine passes against the pinned 0.4.25 binary — correct key →
/search 200, wrong key → 403 (proves the bridge really carries auth). Desktop typecheck +
vite build (adapter bundles into the webview) + eslint + prettier clean. Owner ran tauri:dev
and confirmed the pixel: engine green, Today panel shows "9 captured items today · latest:
siphon-desktop at 6:01:19 AM". /code-review: 1 finding (stale Today count shown after engine
stop) fixed. /security-review: no findings.
Next: feat/m1-timeline — the timeline UI (time-ordered frames, hover thumbnails, keyboard
nav) reading /search?content_type=vision + frame images through this bridge.
What: The read-side of the engine adapter, and the transport seam the whole M1 UI plan
rests on. transport.ts introduces a Transport interface + default FetchTransport; the
EngineClient now owns request assembly, status classification, JSON parsing, and zod
validation, delegating only the network hop. A later slice plugs in a Rust-routed transport so
SCREENPIPE_API_KEY stays in Keychain/Rust and never enters the webview's JavaScript (SPEC
§2.3, rule #4). Adds typed GET /search (endpoints/search.ts + permissive passthrough
schemas/search.ts) with camelCase→snake_case filter mapping — the single entry point timeline
& search read through.
Why this slice: Read-only, no UI — the plumbing every other M1 slice depends on, reviewable on its own (mirrors how M0 landed the adapter before features).
Verified: 30 adapter unit tests pass (mocked fetch + injected-transport tests; the refactor
kept all prior client tests green). Live contract test green against the pinned 0.4.25 binary
(SIPHON_CONTRACT_LAUNCH=1) — /search envelope validated against real engine output.
Discovered live: 0.4.25 leaves /health open but requires the API key on data endpoints, so
the contract harness now spawns with a throwaway SCREENPIPE_API_KEY and authenticates with it.
typecheck (4 projects) + eslint + prettier clean. /code-review (1 finding: dead re-export,
fixed) + inline /security-review (no new key exposure; key location unchanged from M0).
Next: feat/m1-app-bridge — Rust engine_fetch command (attaches the Keychain key) + wire
the adapter's webview transport to it; first real engine data in the window. Gets the full
/security-review (the actual key-routing path).
What: Supervision + health + tray, on top of 3a's acquisition. New Rust modules:
supervisor.rs (spawn screenpipe record --port 3030 --disable-telemetry with
SCREENPIPE_API_KEY in the child's env; bounded-backoff restart on unexpected exit; clean
SIGTERM→SIGKILL stop; logs to ~/.siphon/logs/screenpipe.log; generation-guarded so
stop/start can't orphan a child), health.rs (the sanctioned §2.2 exception — a Rust-side
GET /health liveness probe, loopback + Bearer only, mapped to a Green/Yellow/Red TrayHealth),
tray.rs (a runtime-generated colored status dot + control menu). lib.rs wires a tray, a 5s
health→tray poll loop, engine_start/engine_stop/engine_status commands, and close-to-tray.
The contract harness gained SIPHON_CONTRACT_LAUNCH=1 to launch the pinned binary (capture
disabled) before asserting. The shell gained Start/Stop buttons + a live health dot.
Why this slice: turns the acquired engine into a supervised one and lands the M0 exit
criterion — engine runs supervised, /health green in the tray.
Verified (headless): cargo test 16 unit + fmt/clippy clean — supervisor state machine
(spawn/restart-on-exit/clean-stop) via fake children (sleep/true), a regression test for the
stop-during-spawn race (must not resurrect), and the TrayHealth classifier (healthy→Green,
degraded/5xx→Yellow, 401/403/unreachable→Red). TS typecheck/eslint/prettier + vite build green;
contract harness still skips cleanly with no engine. /code-review (high) → 2 findings
(stop/start orphan race — privacy-relevant; rapid stop→start double-loop) both fixed via an
atomic commit + generation guard, re-verified. /security-review PASS (key via env only, Bearer
to loopback only, never logged, never crosses to the webview).
Owner-run (can't verify from the agent shell — parked no-window bug + macOS capture perms):
cargo test -- --ignored supervises_real_engine (spawns the real engine with capture disabled,
asserts /health reachable, stops it) and the tray pixel-check: pnpm --filter @siphon/desktop tauri:dev in Terminal → click Start engine / see the tray dot go green.
Next: M1 — timeline, search, capture audit, settings.
What: Engine acquisition — the prerequisites for launching the pinned engine, split out
from supervision (3b) for a smaller review. New Rust modules under
apps/desktop/src-tauri/src/engine/: keychain.rs (generate a 256-bit SCREENPIPE_API_KEY and
store it in the macOS Keychain, never in DB/file/logs), pin.rs (parse the engine/VERSION
manifest, embedded at build time via include_str!), download.rs (download the pinned tarball,
verify SHA-256 before extracting — fail closed — then cache the verified bin/ under
~/.siphon/engine/<version>/). Exposed as the engine_ensure_ready Tauri command behind a new
"Download & prepare engine" button in the shell. engine/VERSION moved from UNPINNED to a real
pinned manifest: screenpipe CLI 0.4.25, aarch64-apple-darwin, sha256 recorded from the
actual release.
Ground-truth corrections (from reading the engine source): the CLI is versioned separately
from the desktop app and its macOS binary ships only under a rolling cli-latest tag with no
upstream checksum → we pin by version + our own recorded sha (trust-on-first-use, cached locally).
The env var the engine actually reads is SCREENPIPE_API_KEY (not SCREENPIPE_LOCAL_API_KEY);
corrected the adapter's error strings/docs to match.
Why this slice: Keychain + key-gen + downloading/executing a third-party binary is the
security-sensitive half of the supervisor — reviewed on its own (/security-review PASS: key never
logged/persisted, verify-before-extract, no process execution yet, CSPRNG key).
Verified: cargo test 8 unit + fmt/clippy clean; live --ignored tests prove the real
path headlessly — downloaded ~54 MB, sha256 matched the pin, extracted an executable
bin/screenpipe (+mlx.metallib), second call was a cache hit (~22 s); Keychain round-trip
created com.rushatpeace.siphon/engine.api_key (verified via security find-generic-password,
64-char secret, value never printed). TS typecheck/eslint/prettier/21 tests + workers ruff/pytest
green. /code-review (high) → 2 findings (unbounded download timeout; acquisition not
single-flight) both fixed (bounded reqwest timeouts + async ACQUIRE_LOCK) and re-verified live.
Next: feat/m0-engine-supervise (3b) — spawn + supervise screenpipe record with the key,
Rust /health liveness probe → tray indicator, wire the contract harness to the pinned binary.
Hits the M0 exit criterion.
What: The engine boundary. packages/engine-adapter (TS) — the single place allowed to call
:3030: EngineClient (fetch wrapper: bearer auth, timeout, typed EngineError hierarchy),
zod-validated responses, GET /health endpoint + isHealthy. A thin Python mirror in
workers/src/siphon_workers/engine/ (httpx + pydantic, same shape) for the workers. Contract
smoke-test harness at tests/engine-contract/ that hits a live engine and skips cleanly when none
is running.
Why this slice: This is the update-safety boundary (SPEC §2.2) — engine API churn surfaces here
as an EngineContractError instead of leaking into features. Reviewed on its own.
Verified: pnpm typecheck (4 projects) clean; pnpm test → 21 adapter unit tests pass (mocked
fetch, every error path), contract harness skips with a clear message (no engine yet); workers
ruff clean + pytest 14 pass (mocked httpx transport); eslint + prettier clean.
Next: feat/m0-engine-supervisor — download/supervise the pinned engine, API key → Keychain,
/health → tray. That slice pins engine/VERSION and turns the contract harness live.
What: Monorepo foundation. pnpm workspace (apps/*, packages/*), base TypeScript/ESLint/
Prettier config, Tauri v2 + React/TS desktop skeleton (apps/desktop), Python workers skeleton
(workers/, uv + Python 3.12), engine/VERSION pin placeholder, project-skill stubs.
Why this slice: Pure structure, no engine calls yet — a small, reviewable base before the engine adapter and supervisor land.
Verified: pnpm typecheck / lint / format:check clean; cargo check compiles the Rust
shell (Tauri 2.11.5, ~40s cold); vite build succeeds; uv run ruff check + pytest (2 pass);
python -m siphon_workers starts the APScheduler process and idles. React↔Rust bridge exercised
via the app_info command.
Next: feat/m0-engine-adapter — the typed engine-adapter package + contract smoke tests.