op-reth: add CLI surface snapshot test - #22108
Draft
claude[bot] wants to merge 4 commits into
Draft
Conversation
Renders the full clap command tree (every subcommand and argument, including hidden ones) into a deterministic text snapshot and compares it against a checked-in file, so any CLI surface change — typically inherited silently from a reth pin bump — shows up as a reviewable diff. The renderer is hand-rolled and std-only: it avoids --help output (terminal-width wrapping), long help (embeds the registry-derived built-in chain list), and new dependencies. Machine- and build-specific defaults (platform paths, client version strings, CPU-count-derived values) are normalized to stable placeholders. On mismatch the test prints the full generated snapshot between BEGIN/END markers (recoverable from CI logs) and supports UPDATE_SNAPSHOT=1 to rewrite the file in place. The committed snapshot is a best-effort offline construction from the pinned reth sources; the first CI run is expected to correct it. Refs #21687 Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017oDUppqTaWNyAbxV99uMMY Co-authored-by: Josh Klopfenstein <joshklop@oplabs.co>
Replace the locally seeded CLI surface snapshot with the authoritative version generated by the rust-tests CI job against the pinned reth dependency (f2eecc6). Beyond help-text, alias, possible-value, and positional-arg metadata differences, this corrects two semantic mistakes in the offline approximation: - --gpo.ignoreprice default is 0, not 2: DEFAULT_IGNORE_GAS_PRICE is U256::ZERO at the pinned rev. - --engine.proof-jitter is absent: it is gated behind #[cfg(feature = "trie-debug")], which is not enabled in the CI test build. Machine-specific defaults remain normalized placeholders. Co-Authored-By: Claude <noreply@anthropic.com> Co-authored-by: Josh Klopfenstein <joshklop@oplabs.co>
claude
Bot
force-pushed
the
claude/op-reth-cli-snapshot-test
branch
from
July 29, 2026 17:17
c8aad19 to
8d322b2
Compare
The snapshot renderer emits trim_end() plus exactly one newline; the committed file ended with two, which would still fail the byte-exact comparison. Co-Authored-By: Claude <noreply@anthropic.com> Co-authored-by: Josh Klopfenstein <joshklop@oplabs.co>
rust-clippy runs with -D warnings; bare "CircleCI" in the module doc comment trips clippy::doc_markdown. Co-Authored-By: Claude <noreply@anthropic.com> Co-authored-by: Josh Klopfenstein <joshklop@oplabs.co>
claude
Bot
force-pushed
the
claude/op-reth-cli-snapshot-test
branch
from
July 30, 2026 09:00
6c7c483 to
bf234d1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Josh Klopfenstein · Slack thread
Replaces #22107, which GitHub auto-closed after a transient bad force-push to the branch (the branch is now clean; the diff below is the intended 3-file change).
Before
op-reth inherits its entire CLI surface from the pinned upstream reth crates. A reth pin bump can add, remove, rename, or re-default operator-facing flags without producing any diff in this repository, so those changes ship silently and never get reviewed.
After
A snapshot test renders the full op-reth command tree — every subcommand and every argument, including hidden ones, with aliases, value names, env vars, defaults, possible values, and help text — into a deterministic text file checked in at rust/op-reth/crates/cli/tests/snapshots/cli.snap. Any CLI surface change now fails the test and must be acknowledged by regenerating the snapshot, which makes the change visible in review:
On mismatch the test prints the first differing line, prints the complete generated snapshot between BEGIN/END markers so it can be recovered verbatim from CI logs, and also writes it to the cargo target tmpdir.
How
The renderer in rust/op-reth/crates/cli/tests/cli_snapshot.rs walks the clap command tree built by CommandFactory (the same tree the binary parses) and emits one line per argument in clap definition order. It is deliberately hand-rolled and std-only:
A short section was added to rust/UPDATING-RETH.md telling reth bumpers the snapshot will surface upstream CLI changes and how to regenerate it.
CI
The new test runs in the rust-tests CircleCI job (just test → cargo nextest run --workspace --all-features), which is part of the required-rust-ci gate. The test is gated on the crate dev feature because the dev-only test-vectors subcommand is part of the rendered surface; the --all-features run in CI enables it. Runs without that feature skip the test silently.
Caveats
The committed snapshot is a best-effort construction generated offline from the pinned reth sources — this environment cannot compile the Rust workspace (the reth git dependency is not fetchable here), so the snapshot could not be produced by running the test. The first CI run is expected to fail the new test; the authoritative snapshot will be printed between the BEGIN/END markers in the rust-tests output, and a follow-up commit pastes it into rust/op-reth/crates/cli/tests/snapshots/cli.snap (or a local checkout runs the UPDATE_SNAPSHOT command above), after which the job goes green.
Refs #21687
Generated by Claude Code