fix(sp-update): skip biometrics-archiver setup on new (NATS) firmware + diagnostics#594
fix(sp-update): skip biometrics-archiver setup on new (NATS) firmware + diagnostics#594ng wants to merge 5 commits into
Conversation
Pod 5 firmware ca35aafa (April 2026+) refactored away from /opt/eight/bin/frank.sh — biometric frames now go to NATS JetStream at /persistent/jetstream/ instead of /persistent/*.RAW. The helper previously warned and continued, then a subsequent systemctl enable failed under sp-update's set -euo pipefail and rolled back the OTA. Use frank.sh as the canary for old-firmware applicability; return 0 cleanly when absent so OTAs succeed on the new firmware. The defensive warn-and-skip in the patch block stays in as a belt-and-suspenders guard for future code paths.
📝 WalkthroughWalkthroughThe ChangesFirmware Detection for Biometrics Archiver
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~5 minutes Possibly related PRs
Suggested labels
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Distinguishes the expected new-firmware path (frank.sh absent + nats-server active + /persistent/jetstream present) from a broken old-firmware case (frank.sh deleted, nothing replacing it). frank.sh absence remains the gating signal — the patch cannot apply without it — but the log line now gives the operator enough state to diagnose at a glance.
Previously a one-line 'systemctl status sleepypod.service'. Now reports firmware variant (rat_version, frank.sh presence, frank.service state), biometrics pipeline (tmpfs RAW vs NATS JetStream — including TCP probe of 4222, jetstream dir size, 'nats stream info raw' check), .RAW counts at each location, sleepypod modules, biometrics-archiver units, and firmware-side services. Safe on every firmware variant — every probe fails silently to 'missing' so the same command works on Pod 3/4/5 regardless of OTA generation.
One-shot Python recorder that subscribes to a NATS subject filter, dumps every message as NDJSON (subject, headers, payload bytes, CBOR-decoded shape) for a fixed duration, and prints a per-subject histogram on exit. Used to learn what frame types Pod 5 new-firmware publishes before designing the BiometricsSource abstraction in sleepypod-core-56. Bundled deps via uv shebang; pod-side requires the dac user + the nats-server already running on localhost:4222 (default on new firmware).
frank.sh was removed from frankenfirmware ~year ago, so its absence is
NOT a reliable canary for the new NATS firmware (April 2026+). Three
firmware variants coexist in the fleet:
- Old (Pod 3 / Pod 4 / early Pod 5): frank.sh present, .RAW via shim
- Mid (~2025 → April 2026): frank.sh absent, .RAW direct to /persistent
- New (April 2026+, e.g. ca35aafa): frank.sh absent, NATS JetStream
The prior gate ('frank.sh absent → skip') caught mid-era pods too, which
would have regressed their archive/prune rotation. Mid-era pods have been
on the warn-and-continue path successfully since PR #588 — preserve that.
Gate strictly on the new-firmware signals: nats-server.service active
AND /persistent/jetstream directory present. Only those pods skip the
helper. sp-status pipeline detection updated to match the 3-state matrix.
ADR 0018 + biometrics-system wiki + scripts/README updated with the
firmware-variant matrix and applicability notes.
Summary
Pod 5 firmware
ca35aafa(April 2026+) writes biometric CBOR frames to NATS JetStream streamraw(subjectraw.>) at/persistent/jetstream/instead of.RAWfiles. Previouslyinstall_biometrics_archivercontinued past the missingfrank.shshim and trippedsp-update'sset -euo pipefailon a subsequentsystemctl enable --now <timer>, rolling back the OTA. Symptom:Warning: /opt/eight/bin/frank.sh not found — skipping patch (not a Pod?)immediately followed byUpdate failed, rolling back....This PR:
nats-server.serviceactive AND/persistent/jetstreamdirectory present). Only those pods skip; mid-era pods (frank.sh absent, no NATS — exists for ~year now) keep the existing warn-and-continue path so their archive/prune rotation isn't lost.sp-statusfrom a one-liner into a full diagnostic: firmware variant, biometrics pipeline (old/mid/new), NATS probe (TCP 4222 +nats stream info raw), JetStream dir size,.RAWcounts at each location, sleepypod modules, biometrics-archiver units, firmware-side services. Output is paste-friendly for Discord support threads.scripts/probe-nats-capture.py— a one-shot Python recorder (uv shebang, pullsnats-py+cbor2on first run) that subscribes to a NATS subject filter, dumps every message as NDJSON with CBOR-decoded shape, and prints a per-subject histogram on exit. Used to discover what subjects/schemas the new firmware actually publishes before designing the consumer.scripts/README.mddescribe the new data source path and the discovery probe.This is the OTA-unblock stopgap. Actual biometrics ingestion against NATS JetStream is tracked separately in sleepypod-core-54 (epic) with subtasks 55 (discovery via the new probe), 56 (
BiometricsSourceabstraction +nats-pysource), 57 (status surface). Per the saved feedback memory, that work must be additive —RawFileFollowerstays the default and old/mid pods are unaffected.Firmware-variant matrix
frank.shnats-server/persistent/jetstream/persistent/*.RAW/persistent/*.RAWVerification
Local harness with stubbed
systemctl/install/mountpointexercises all four control-flow branches (new fw, mid-era, old fw, anomaly = NATS up but no jetstream dir → conservative fall-through to legacy install). All pass. Shell syntax + shellcheck clean.Test plan
ca35aafa+,nats-serveractive,/persistent/jetstream/populated):sp-updatecompletes without rollback; log showsnew firmware detected (nats-server active; /persistent/jetstream present) — skipping tmpfs setup.sp-statusreportspipeline: NATS JetStream (new firmware, April 2026+).frank.shabsent, no NATS):sp-updatecontinues to install mount + timers + archive loop as it does today — no regression in.RAWrotation at/persistent.sp-statusreportspipeline: direct *.RAW (mid-era firmware; ...).frank.shpresent):sp-updatepatchesfrank.sh, mounts tmpfs, installs timers — completely unchanged path.sp-statusreportspipeline: tmpfs *.RAW via frank.sh shim (old firmware).sleepypod.servicestarts and stays active after the OTA on each pod variant.scripts/probe-nats-capture.py --duration 30produces an NDJSON dump with non-zero message count and per-subject histogram. CBOR decode succeeds for at least one subject.Known follow-up (out of scope)
After this PR, new-firmware pods OTA cleanly but do not ingest biometric data — the NATS consumer is sleepypod-core-54 epic. Discovery captures (sleepypod-core-55) gate the implementation work;
probe-nats-capture.pyexists for that purpose.🛏️ Beam this PR onto a sleepypod ✨
Pick the snippet that matches your pod. Already-installed pods can't use the curl bootstraps below — sleepypod's egress firewall DROPs github.com, and the script that knows how to unblock WAN is the very file curl is trying to fetch. Use
sp-updateinstead; it's on disk and opens WAN as its first step.🔄 Already on a sleepypod (most common — review a PR on a running pod):
🚀 Fresh pod / first install — bootstraps from GitHub, rebuilds every push:
curl -fsSL https://raw.githubusercontent.com/sleepypod/core/worktree-pod5-debug/scripts/install \ | sudo bash -s -- --branch worktree-pod5-debug🎯 Pin to this exact build (run 26003896986 ·
6e3536c) — fresh-pod path, locked to one CI artifact for reproducible review:🧊 Artifact:
sleepypod-core· self-destructs in 30 days 💥