Skip to content

perf: hash through an aligned keccak sponge - #668

Closed
Qumeric wants to merge 4 commits into
develop-v2.1.0from
valery/guest-keccak-sponge
Closed

perf: hash through an aligned keccak sponge#668
Qumeric wants to merge 4 commits into
develop-v2.1.0from
valery/guest-keccak-sponge

Conversation

@Qumeric

@Qumeric Qumeric commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Adds an 8-byte-aligned guest Keccak sponge that emits XORIN/KECCAKF directly and implements BufMut, allowing MPT reference RLP to stream into the hash without rlp_scratch. MPT decode/update hashing and witness-db address/slot hashing use the same one-shot path, avoiding the checked wrapper's alignment allocations and copies. The sponge maintains the native instruction alignment and length requirements internally.

The win is per hash call, not per permutation — the permutation count is unchanged, so this does not touch KeccakfPermAir. What it removes is the software bookkeeping around every absorb: the 200-byte state reset, the staging copies the checked wrapper makes when a pointer or length is not 8-aligned, and that wrapper's own alignment checks. With hundreds of thousands of hashes per block, those per-call constants dominate.

Differential coverage compares known answers, boundary lengths, unaligned inputs, streaming writes, and the raw BufMut path against tiny-keccak, plus a sweep enumerating the absorb state machine's transition table — every staged fill against every following input length — since sampling uniform chunk sizes only ever visits fills in an arithmetic progression.

That coverage was checked by mutation rather than assumed: each guard was disabled in turn to see whether the suite noticed. Three changes initially slipped through. Two were unchecked preconditions — the XORIN length and alignment assertions sat inside the guest-only xorin, so host runs never exercised the contract the instruction depends on, and they now live in a shared wrapper above the target-gated bodies. The third was put_u8, whose write into the staged block is bounded only by the flush above it, which no test had presented with a full block. Eight of nine mutations are now caught; the one survivor absorbs an exact-fit block eagerly instead of lazily, which no observer can distinguish. Both fixes are debug_asserts, so the measured numbers below are unaffected.

See Claude's explainer artifact for more details.

Benchmark results

Both blocks measured against develop-v2.1.0 at the same commit, execute-metered.

Block 24001988

develop-v2.1.0 vs this PR:

metric develop-v2.1.0 this PR delta
execute_metered_insns 588,618,511 575,132,344 −13,486,167 (−2.291%)
metered_rows_unpadded 844,070,559 820,155,810 −23,914,749 (−2.833%)
metered_main_cells_unpadded 43,260,383,096 42,177,994,061 −1,082,389,035 (−2.502%)
metered_interaction_cells_unpadded 13,194,454,796 12,875,875,956 −318,578,840 (−2.414%)
metered_memory_unpadded_bytes 713,961,114,662 699,779,237,587 −14,181,877,075 (−1.986%)
app segments 65 63 −2

Block 24002549

develop-v2.1.0 vs this PR:

metric develop-v2.1.0 this PR delta
execute_metered_insns 527,179,427 515,190,361 −11,989,066 (−2.274%)
metered_rows_unpadded 759,879,792 737,715,207 −22,164,585 (−2.917%)
metered_main_cells_unpadded 39,410,659,691 38,402,408,673 −1,008,251,018 (−2.558%)
metered_interaction_cells_unpadded 11,846,946,557 11,563,307,133 −283,639,424 (−2.394%)
metered_memory_unpadded_bytes 646,441,525,015 633,407,491,239 −13,034,033,776 (−2.016%)
app segments 59 57 −2

The two blocks agree to within 0.02% on instructions and each drop exactly two segments, so the win does not depend on block shape. The memory reduction is the checked wrapper's staging buffers going away: they were allocated through the guest bump allocator, where deallocation is a no-op, so every unaligned absorb permanently grew the heap.

Relationship to openvm#3070

openvm#3070 rewrites native_xorin's unaligned fallback so it allocates nothing. The two changes are complementary rather than overlapping: this PR takes the MPT's hashing off the wrapper entirely, while openvm#3070 improves the wrapper for everything still going through it — the EVM KECCAK256 opcode and transaction/header hashing inside revm, which reach it via the alloy native-keccak hook. Neither blocks the other, and openvm#3070's contract is unchanged, so landing it needs only a patch revision bump here.

@Qumeric
Qumeric force-pushed the valery/guest-keccak-sponge branch from 3010831 to 6d68f09 Compare July 28, 2026 07:05
@Qumeric
Qumeric changed the base branch from codex/mpt-common-rlp-headers to develop-v2.1.0 July 28, 2026 07:05
@Qumeric
Qumeric requested a review from shuklaayush July 28, 2026 08:53
@Qumeric
Qumeric marked this pull request as ready for review July 28, 2026 08:53
Qumeric added 3 commits July 28, 2026 18:33
The length and alignment assertions sat inside the guest-only `xorin`, so host test runs
never checked the contract the instruction depends on. Hoist them into a shared wrapper
above the target-gated bodies.
`put_u8` writes straight into the staged block, bounded only by the `flush_if_full` above
it. Nothing checked that, so removing or reordering the flush would write one byte past the
block instead of failing.
Mutation testing found three changes the suite did not notice: admitting a misaligned block to
the aligned XORIN path, dropping the pad-to-8 before the final absorb, and removing put_u8's
flush. The first two were unchecked host-side preconditions; these sweeps cover the third, by
enumerating fill against absorb length rather than sampling uniform chunk sizes.
@Qumeric

Qumeric commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Closing this as this is only 1% improvement in presence of related OpenVM Keccak improvement and considering the fact that this is not going to be a canonical Ethereum guest going forward and we do not want to introduce additional risk

@Qumeric Qumeric closed this Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant