perf: hash through an aligned keccak sponge - #668
Closed
Qumeric wants to merge 4 commits into
Closed
Conversation
This was referenced Jul 21, 2026
Qumeric
force-pushed
the
valery/guest-keccak-sponge
branch
from
July 28, 2026 07:05
3010831 to
6d68f09
Compare
Qumeric
changed the base branch from
codex/mpt-common-rlp-headers
to
develop-v2.1.0
July 28, 2026 07:05
Qumeric
marked this pull request as ready for review
July 28, 2026 08:53
The length and alignment assertions sat inside the guest-only `xorin`, so host test runs never checked the contract the instruction depends on. Hoist them into a shared wrapper above the target-gated bodies.
`put_u8` writes straight into the staged block, bounded only by the `flush_if_full` above it. Nothing checked that, so removing or reordering the flush would write one byte past the block instead of failing.
Mutation testing found three changes the suite did not notice: admitting a misaligned block to the aligned XORIN path, dropping the pad-to-8 before the final absorb, and removing put_u8's flush. The first two were unchecked host-side preconditions; these sweeps cover the third, by enumerating fill against absorb length rather than sampling uniform chunk sizes.
Contributor
Author
|
Closing this as this is only 1% improvement in presence of related OpenVM Keccak improvement and considering the fact that this is not going to be a canonical Ethereum guest going forward and we do not want to introduce additional risk |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an 8-byte-aligned guest Keccak sponge that emits XORIN/KECCAKF directly and implements
BufMut, allowing MPT reference RLP to stream into the hash withoutrlp_scratch. MPT decode/update hashing and witness-db address/slot hashing use the same one-shot path, avoiding the checked wrapper's alignment allocations and copies. The sponge maintains the native instruction alignment and length requirements internally.The win is per hash call, not per permutation — the permutation count is unchanged, so this does not touch
KeccakfPermAir. What it removes is the software bookkeeping around every absorb: the 200-byte state reset, the staging copies the checked wrapper makes when a pointer or length is not 8-aligned, and that wrapper's own alignment checks. With hundreds of thousands of hashes per block, those per-call constants dominate.Differential coverage compares known answers, boundary lengths, unaligned inputs, streaming writes, and the raw
BufMutpath againsttiny-keccak, plus a sweep enumerating the absorb state machine's transition table — every stagedfillagainst every following input length — since sampling uniform chunk sizes only ever visits fills in an arithmetic progression.That coverage was checked by mutation rather than assumed: each guard was disabled in turn to see whether the suite noticed. Three changes initially slipped through. Two were unchecked preconditions — the XORIN length and alignment assertions sat inside the guest-only
xorin, so host runs never exercised the contract the instruction depends on, and they now live in a shared wrapper above the target-gated bodies. The third wasput_u8, whose write into the staged block is bounded only by the flush above it, which no test had presented with a full block. Eight of nine mutations are now caught; the one survivor absorbs an exact-fit block eagerly instead of lazily, which no observer can distinguish. Both fixes aredebug_asserts, so the measured numbers below are unaffected.See Claude's explainer artifact for more details.
Benchmark results
Both blocks measured against
develop-v2.1.0at the same commit, execute-metered.Block 24001988
develop-v2.1.0vs this PR:develop-v2.1.0execute_metered_insnsmetered_rows_unpaddedmetered_main_cells_unpaddedmetered_interaction_cells_unpaddedmetered_memory_unpadded_bytesBlock 24002549
develop-v2.1.0vs this PR:develop-v2.1.0execute_metered_insnsmetered_rows_unpaddedmetered_main_cells_unpaddedmetered_interaction_cells_unpaddedmetered_memory_unpadded_bytesThe two blocks agree to within 0.02% on instructions and each drop exactly two segments, so the win does not depend on block shape. The memory reduction is the checked wrapper's staging buffers going away: they were allocated through the guest bump allocator, where deallocation is a no-op, so every unaligned absorb permanently grew the heap.
Relationship to openvm#3070
openvm#3070 rewrites
native_xorin's unaligned fallback so it allocates nothing. The two changes are complementary rather than overlapping: this PR takes the MPT's hashing off the wrapper entirely, while openvm#3070 improves the wrapper for everything still going through it — the EVMKECCAK256opcode and transaction/header hashing inside revm, which reach it via the alloynative-keccakhook. Neither blocks the other, and openvm#3070's contract is unchanged, so landing it needs only a patch revision bump here.