test(golden): capture/replay golden-fixture harness (#41) - #86
Merged
Merged
Conversation
Capture real Flipper CLI/RPC byte exchanges once and replay them offline in CI, mirroring the workspace VCR-cassette discipline. ReplayTransport serves a recorded tx/rx event list back through the real client/RPC code; an event-cursor gates device output behind the matching request so the time-based drain/read loops reproduce the recorded exchange without hardware. Covers the six launch-gate behaviors: cli_exec parsing, fs_push MD5 match/mismatch, RPC<->CLI mode switch, shared _io_lock non-interleave, the TX two-gate (enabled and disabled), and WiFi-CLI rejection. The default suite replays every fixture and asserts the recorded result, which is the drift guard (re-record with `make record-fixtures` on hardware). Closes #41 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A golden-fixture test harness that captures real Flipper CLI/RPC byte exchanges once and replays them offline in CI, mirroring the workspace VCR-cassette discipline.
tests/golden/harness.py—RecordingTransport(wraps a real transport, logs every wire event),ReplayTransport(serves a fixture back offline), and theFixtureJSON format.tests/golden/record.py— captures fixtures from a USB-connected Flipper.tests/golden/test_golden_fixtures.py— replays each fixture in the defaultnot integrationsuite.tests/golden/fixtures/*.json— six committed fixtures (four captured from real hardware, two no-wire-I/O refusals).Makefile—record-fixtures(hardware) andverify-fixtures(offline drift check).docs/agents/golden-fixtures.md— harness reference.Replay model
Event-cursor:
receive()serves the nextrxevent only once everytxrecorded before it has been sent; otherwise it returnsb"". This gates device output behind the matching request, so the time-based drain/read loops in the RPC and CLI code reproduce the recorded exchange faithfully, with no dependence on wall-clock timing or hardware. Sent payloads are recorded for assertions but not matched, so the push-integrity mismatch test can replay the same device MD5 against a corrupted local file.Coverage (the six launch-gate behaviors)
cli_exec echo/prompt parsing, fs_push MD5 match + mismatch, RPC↔CLI mode switch, shared
_io_locknon-interleave, the TX two-gate (enabled + disabled), and WiFi-CLI rejection.Drift guard
make verify-fixtures(also in the default CI suite) replays every fixture and asserts the committedexpect. A parsing/framing/gating change that reinterprets the real captured bytes fails the matching test — signal to re-record on hardware. Fixtures carry aschema_version; a mismatch is a hard load error.Verification
make record-fixtures(capture → verify) loop green.uv run pytest -m "not integration"— 144 passed, coverage 91.73%.ruff check,ruff format --check,ty check src/all clean.Closes #41
🤖 Generated with Claude Code