ci: add benchmark block presets and measured precompile mix - #657
Open
Qumeric wants to merge 2 commits into
Open
ci: add benchmark block presets and measured precompile mix#657Qumeric wants to merge 2 commits into
Qumeric wants to merge 2 commits into
Conversation
Qumeric
force-pushed
the
valery/bench-block-presets
branch
from
July 10, 2026 14:26
fb66c72 to
4c4c35a
Compare
Guest circuit flamegraph of block 24002549 showed its cost is keccak cells (20.7% of app-proof cells), not bn254 arithmetic (0.31%). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Qumeric
force-pushed
the
valery/bench-block-presets
branch
from
July 14, 2026 13:31
4c4c35a to
2158c48
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #656.
What
.github/benchmark-blocks.json— registry of curated benchmark blocks with provenance notes:p100/p99/p50(from the 16-GPU release run over blocks 24000000..24007199),keccak-heavy(Aztec-pattern rollup block — flamegraph-verified: 20.7% of its cells are keccak permutations, bn254 arithmetic is 0.31%), andp256-heavy(the pre-July-2026 default, kept for precompile-targeted comparisons).block_presetchoice input on Compare Branches and Compare Benchmarks — picking a block is now a dropdown with semantic names; the free-textblock_numberremains as an override. A smallresolve-blockjob (GitHub-hosted, seconds) resolves the preset from the registry so there is a single source of truth.debug_traceBlockByNumber+ callTracer viaRPC_URL_1, public fallback) and writes its precompile call counts to the step summary. This is measured per run, not looked up, so it is correct for arbitrary blocks too. Non-fatal on RPC failure (continue-on-error).Why
The old default block made the tail because of software P256VERIFY (7 calls, ~8% of its instructions), so a p256 optimization showed −9.34% on it while moving fleet percentiles not at all. Making the block's workload mix visible next to every result prevents extrapolating workload-specific wins, and named presets make it obvious how to check a claim against a median or stress block.