Skip to content

perf(mpt): specialize the RLP shapes the witness contains, behind an rlp module - #684

Draft
Qumeric wants to merge 4 commits into
develop-v2.1.0from
valery/mpt-rlp-module
Draft

perf(mpt): specialize the RLP shapes the witness contains, behind an rlp module#684
Qumeric wants to merge 4 commits into
develop-v2.1.0from
valery/mpt-rlp-module

Conversation

@Qumeric

@Qumeric Qumeric commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Replaces #664, #666 and #667, which were three stacked PRs expressing one idea: reach the RLP encodings a trie witness actually contains without walking alloy_rlp's general header space. Splitting one idea across three stacked PRs cost more to review than it saved, so they are collapsed here.

Stacked on #658.

Two commits:

  1. Specialize the RLP shapes — the union of the three PRs, with the unrelated storage-trie cache (now perf: cache repeated storage trie lookups #683) removed from the ancestry.
  2. Move RLP handling into its own module, with differential tests — the specializations were inline fast paths and match arms scattered through a 1129-line trie.rs. decode_rlp_item, decode_node_header, is_null_ref, encode_header, encode_slice, advance_unchecked and NULL_NODE_REF_SLICE move to crates/mpt/src/rlp.rs; bytes_eq and branch_child_id stay, since they are not RLP.

Why the module matters

Hand-specialized RLP invites exactly one question — does it agree with the general implementation on every input? Previously that was only covered indirectly, through trie round-trips. Behind a module boundary each specialization is compared against alloy_rlp directly:

  • decode_node_header_matches_alloy / decode_rlp_item_matches_alloy sweep every leading byte 0x00..=0xff — the dimension that selects which specialized arm runs — against payload lengths straddling the 1-, 2- and 3-byte length-prefix boundaries, and against three fillers for the bytes after the lead. Most combinations are invalid RLP, which is the point: the specialized decoder must reject exactly what alloy_rlp rejects.

    The filler dimension matters more than it looks. For the header forms carrying a multi-byte length, the bytes after the lead are the declared length, so a single filler pins each of those arms to one length value. Sweeping all-zero and all-ones fillers alongside the pattern is what reaches the canonicality rules — a length with a leading zero, a long-form length below 56, and a maximal one. I checked this holds by mutation: deleting the 0xf9 leading-zero rejection leaves every test passing under a fixed filler, and fails decode_node_header_matches_alloy with the sweep.

  • encode_header_matches_alloy covers payload lengths 0..600 plus 65_535, 65_536 and 1 << 20, for both lists and strings.

  • encode_slice_matches_alloy sweeps all 256 single-byte strings, the only shape with a special encoding.

  • Round-trip and is_null_ref tests cover the remainder.

Errors are compared as accept-or-reject rather than by variant: the contract is that invalid input is rejected, not which variant surfaces. On success the tests assert both the decoded value and the buffer advance.

Benchmark results

Marginal against its own parent, #658, on block 24001988, execute-metered — #658 vs this PR:

metric #658 this PR delta
execute_metered_insns 573,518,844 560,139,143 −13,379,701 (−2.333%)
metered_rows_unpadded 821,986,167 805,256,166 −16,730,001 (−2.035%)
metered_main_cells_unpadded 42,655,380,312 42,160,281,907 −495,098,405 (−1.161%)
metered_interaction_cells_unpadded 12,920,344,836 12,680,532,190 −239,812,646 (−1.856%)
metered_memory_unpadded_bytes 705,340,794,254 697,234,594,891 −8,106,199,363 (−1.149%)
app segments 63 62 −1

Cumulative for the chain (#647 + #658 + this PR) against develop-v2.1.0:

metric block 24001988 block 24002549
execute_metered_insns 588,618,511 → 560,139,143 (−4.838%) 527,179,427 → 501,891,666 (−4.797%)
metered_rows_unpadded −4.598% −4.240%
metered_main_cells_unpadded −2.543% −2.336%
metered_interaction_cells_unpadded −3.895% −3.830%
metered_memory_unpadded_bytes −2.343% −1.955%
app segments 65 → 62 (−3) 59 → 57 (−2)

Runs: block 24001988 develop-v2.1.0 / chain; block 24002549 develop-v2.1.0 / chain.

Notes

decode_node_header and decode_rlp_item keep alloy_rlp as the fallback for anything uncommon, so behaviour is unchanged and only the common paths get shorter.

The unresolved-node fast path compares the witness digest against the parent's expected reference with digest_eq, whose length is fixed at compile time so the comparison stays inline as whole-word loads. That comparison is the single hottest operation the fast path performs — a slice comparison of runtime length here costs more than the generic header decode the fast path exists to avoid.

…l tests

The specializations were spread through trie.rs as inline fast paths and match arms. Collecting
them behind a module boundary keeps the trie code about tries, and gives the specializations a
place to be checked: each one is now compared against alloy_rlp across the header-form
boundaries, which is the property that makes them safe to have.
@Qumeric
Qumeric force-pushed the valery/mpt-rlp-module branch from 327eb0a to cc42675 Compare July 28, 2026 14:31
@Qumeric
Qumeric marked this pull request as draft July 28, 2026 16:53
@Qumeric
Qumeric force-pushed the valery/mpt-nonzero-children branch from 7d01991 to cab15d0 Compare July 29, 2026 07:30
Base automatically changed from valery/mpt-nonzero-children to develop-v2.1.0 July 29, 2026 07:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant