Biometrics durability: tmpfs + export API + push backup

[ng]

Problem

RAW biometric streams (/persistent/*.RAW) currently:

• Live on eMMC. ~1 GB/day write rate (4–11 MB / 15 min depending on activity). Cheap eMMC TBW budgets survive years at this rate, but write-amplification + bursts compress that. Frank firmware prunes to ~24h on-disk by default.
• Have no export path. Users can't pull last week's data; can't share a window for bug forensics; can't take their data with them when switching pods. The Pod 3 dialect bug ((Pod 3) Biometrics services are running, but calibration and data collection not working #395, fixed in fix(pod3): bedTemp/capSense v1 dialect normalizer + streaming resync #486) was diagnosable only because the reporter manually attached a RAW file to a draft PR — that workflow doesn't scale.
• Cannot reach external storage. No USB auto-mount on Pod firmware — only options are pull-via-API or pod-pushes-to-remote.

biometrics.db (the processed HR/HRV/sleep-record metrics that the iOS/web app actually displays) is durable on disk and not affected — pruning RAW does not touch user-facing data. RAW is the upstream signal that sidecar processors consume and discard.

Proposal

Three layers, separately shippable, sequenced for early user value.

Layer 1 — tmpfs + cold archive

frank → /persistent/biometrics/        (tmpfs, 500 MB, hot live RAW)
            ↓ archiver (every 15 min, gzip oldest-first)
        /persistent/biometrics-archive/<seqno>.RAW.gz
            ↓ pruner (every 15 min, runs after archiver)
        deletes the source from tmpfs

• Zero RAW writes hit eMMC for the live stream. Wear approximately eliminated.
• Compressed cold archive on eMMC: ~100–200 MB/day (gzip -6, typical for CBOR with int32 piezo bursts).
• 13 GB free at /persistent → 65–130 days of compressed history before retention triggers.
• Configurable retention; default = "auto-prune oldest until disk < 80% full".

Touches

• /opt/eight/bin/frank.sh — change RAW write path from /persistent to /persistent/biometrics
• new systemd timer + service: archiver
• new systemd timer + service: pruner
• scripts/install — set up tmpfs mount idempotently
• ADR documenting the volatility tradeoff (~30 min raw waveform loss on unclean shutdown; vitals row durability unchanged)

Layer 2 — Pull export API (independent of L1)

• system.exportArchive tRPC endpoint streaming tar.gz over HTTP
• Inputs: startTs, endTs, include: ['raw', 'db', 'logs']
• For DB: sqlite3 biometrics.db .dump | gzip — text SQL, schema-portable
• For raw: tar of compressed archive files in window
• Token-gated auth (env), rate-limited
• iOS/web Export button: date picker + "include raw waveforms" checkbox

Independent of L1 — works against status-quo on-disk RAW, ships export UX immediately for forensics + take-my-data-with-me. Once L1 lands, the same endpoint serves the cold archive.

Layer 3 — Push backup (file but defer)

• SSH push (SFTP/rsync) to user-configured remote
• Config at /etc/sleepypod/archive-push.yml: host, user, path, identity, schedule, include, bandwidth_kbps, retention_remote
• Settings UI: connection test + on-pod-generated ed25519 pubkey copy button
• Build only on user demand. SFTP/rsync chosen because pod already has openssh-client (no new deps).

Out of scope: USB auto-mount (firmware doesn't support), S3 (no AWS CLI in image), WebDAV/SMB.

Open design questions

Q	Lean
Archive granularity (per-file <seqno>.RAW.gz vs daily tarballs)	per-file (simpler windowed export)
Compression (gzip vs zstd)	gzip (universal, sufficient)
Retention default	auto-prune to 80% disk; configurable
Export auth	env-configured token, rate-limited
Export UX	single button + "include raw" checkbox

Sequence

1. L1 — prototype on local pod first, validate (df, mount, sidecars still consuming RAW correctly, no piezo-processor / sleep-detector / environment-monitor regressions), then PR
2. L2 — independent of L1, can land in parallel against status-quo RAW
3. L3 — filed but not claimed until users ask

Acceptance (per layer)

• L1: mount | grep tmpfs shows /persistent/biometrics. After 30 days operation, df /persistent shows < 5 GB used. piezo-processor / sleep-detector / environment-monitor produce DB rows at unchanged cadence. Unclean reboot loses ≤ 30 min of raw waveform; biometrics.db row count for the affected window is unaffected.
• L2: curl <pod>:3000/api/trpc/system.exportArchive?input=... returns valid tar.gz of the requested window. iOS Export button downloads the same archive.
• L3: enabling push → first scheduled run completes → files appear on configured remote. Failure mode (bad host, missing key) does not crash the scheduler.