Skip to content

test(golden): capture/replay golden-fixture harness (#41) - #86

Merged
millsmillsymills merged 2 commits into
phase1from
feat/41-golden-fixture-harness
Jun 9, 2026
Merged

millsmillsymills merged 2 commits into
phase1from
feat/41-golden-fixture-harness

Conversation

@millsmillsymills

Copy link
Copy Markdown
Contributor

What

A golden-fixture test harness that captures real Flipper CLI/RPC byte exchanges once and replays them offline in CI, mirroring the workspace VCR-cassette discipline.

  • tests/golden/harness.py — RecordingTransport (wraps a real transport, logs every wire event), ReplayTransport (serves a fixture back offline), and the Fixture JSON format.
  • tests/golden/record.py — captures fixtures from a USB-connected Flipper.
  • tests/golden/test_golden_fixtures.py — replays each fixture in the default not integration suite.
  • tests/golden/fixtures/*.json — six committed fixtures (four captured from real hardware, two no-wire-I/O refusals).
  • Makefile — record-fixtures (hardware) and verify-fixtures (offline drift check).
  • docs/agents/golden-fixtures.md — harness reference.

Replay model

Event-cursor: receive() serves the next rx event only once every tx recorded before it has been sent; otherwise it returns b"". This gates device output behind the matching request, so the time-based drain/read loops in the RPC and CLI code reproduce the recorded exchange faithfully, with no dependence on wall-clock timing or hardware. Sent payloads are recorded for assertions but not matched, so the push-integrity mismatch test can replay the same device MD5 against a corrupted local file.

Coverage (the six launch-gate behaviors)

cli_exec echo/prompt parsing, fs_push MD5 match + mismatch, RPC↔CLI mode switch, shared _io_lock non-interleave, the TX two-gate (enabled + disabled), and WiFi-CLI rejection.

Drift guard

make verify-fixtures (also in the default CI suite) replays every fixture and asserts the committed expect. A parsing/framing/gating change that reinterprets the real captured bytes fails the matching test — signal to re-record on hardware. Fixtures carry a schema_version; a mismatch is a hard load error.

Verification

  • Fixtures captured from a real USB Flipper; full make record-fixtures (capture → verify) loop green.
  • uv run pytest -m "not integration" — 144 passed, coverage 91.73%.
  • ruff check, ruff format --check, ty check src/ all clean.

Closes #41

🤖 Generated with Claude Code

millsmillsymills and others added 2 commits June 9, 2026 00:39
Capture real Flipper CLI/RPC byte exchanges once and replay them offline in
CI, mirroring the workspace VCR-cassette discipline. ReplayTransport serves a
recorded tx/rx event list back through the real client/RPC code; an
event-cursor gates device output behind the matching request so the time-based
drain/read loops reproduce the recorded exchange without hardware.

Covers the six launch-gate behaviors: cli_exec parsing, fs_push MD5
match/mismatch, RPC<->CLI mode switch, shared _io_lock non-interleave, the TX
two-gate (enabled and disabled), and WiFi-CLI rejection. The default suite
replays every fixture and asserts the recorded result, which is the drift
guard (re-record with `make record-fixtures` on hardware).

Closes #41

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@millsmillsymills
millsmillsymills merged commit f4a76a9 into phase1 Jun 9, 2026
5 checks passed
@millsmillsymills
millsmillsymills deleted the feat/41-golden-fixture-harness branch June 9, 2026 07:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

[launch] Golden-fixture test harness

1 participant