Skip to content

Refresh benchmark comparison against current alloy 1.6/2.0#77

Merged
koko1123 merged 1 commit into
mainfrom
chore/alloy-bench-update
Jun 10, 2026
Merged

Refresh benchmark comparison against current alloy 1.6/2.0#77
koko1123 merged 1 commit into
mainfrom
chore/alloy-bench-update

Conversation

@koko1123

@koko1123 koko1123 commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

What

Re-runs the eth-zig vs alloy comparison against current alloy (primitives/sol-types/dyn-abi 1.6.0, consensus/signer/network 2.0.5) on Zig 0.16.0, and updates every public claim to the new honest numbers.

New score: 18/26 wins, 5 losses, 3 ties (was 23/26 vs March's alloy 0.8/0.6)

Alloy improved on mid-size keccak (256B/1KB now ~tied), hex/address parsing, and uniswapv4 swap. eth-zig's headline wins grew: RLP EIP-1559 encode 24.33x, ABI dynamic decode 8.41x, secp256k1 sign+recover 4.23x, sign 2.34x, u256 div 3.43x.

Changes

  • bench/alloy-bench/Cargo.toml: major-version bumps (benches compiled with zero source changes)
  • bench/RESULTS.md: full new 26-row table with versions, date (2026-06-10), machine; March run preserved as a dated historical note
  • README hero/Why/tables, docs benchmarks/introduction/comparison/faq pages, and the SEO description in layout.tsx (still claimed 19/26) all updated — no stale ratio left outside immutable CHANGELOG history
  • CHANGELOG Unreleased: Changed bullet

Verification

  • compare.sh + compare_u256.sh complete runs (logs preserved); numbers copied verbatim
  • zig build test (0.16.0) and pnpm -C docs build pass

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation

    • Refreshed benchmark comparison documentation with updated performance measurements.
    • Updated metrics for cryptographic operations: secp256k1 signing, ABI encoding/decoding, Keccak hashing, u256 arithmetic, hex operations, and transaction hashing.
    • Revised library performance standings in comparison tables.
  • Chores

    • Upgraded Alloy ecosystem dependencies to newer stable releases for benchmarking.

The published numbers dated from March against alloy 0.8/0.6. Upgrade
the bench harness (alloy-primitives/sol-types/dyn-abi 1.6.0,
alloy-consensus/signer/network 2.0.5, alloy-rlp 0.3.15; no source
changes needed) and re-run the full comparison on Zig 0.16.0.

New honest score: eth-zig 18/26, alloy 5/26, 3 tied (was 23/26 --
alloy improved on mid-size keccak, hex/address parsing, and the
uniswapv4 swap path). Biggest verified wins: RLP EIP-1559 encode
24.33x, ABI dynamic decode 8.41x, secp256k1 recovery 4.23x, signing
2.34x, u256 div 3.43x.

Updated every public claim: README hero/tables, docs benchmarks,
introduction, comparison, faq pages, the SEO description in
layout.tsx (still said 19/26), and bench/RESULTS.md (full new table,
versions, date, machine; March run kept as a dated historical note).
@vercel

vercel Bot commented Jun 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
eth-zig Ready Ready Preview, Comment Jun 10, 2026 3:15pm

Request Review

@coderabbitai

coderabbitai Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

Pull request was closed or merged during review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2d5b1f9d-b176-4712-9d24-72ffaa1d26e2

📥 Commits

Reviewing files that changed from the base of the PR and between e28b44b and 1ce0f86.

📒 Files selected for processing (9)
  • CHANGELOG.md
  • README.md
  • bench/RESULTS.md
  • bench/alloy-bench/Cargo.toml
  • docs/content/docs/benchmarks.mdx
  • docs/content/docs/comparison.mdx
  • docs/content/docs/faq.mdx
  • docs/content/docs/introduction.mdx
  • docs/src/app/layout.tsx

📝 Walkthrough

Walkthrough

This PR refreshes eth.zig's public benchmark positioning by upgrading the benchmark harness to newer Alloy crate versions (1.6/2.0), running a new comparative benchmark, and updating all marketing, documentation, and metadata to reflect the refreshed results and performance claims (18 of 26 benchmarks won).

Changes

Benchmark Results Refresh

Layer / File(s) Summary
Benchmark harness dependency upgrade
bench/alloy-bench/Cargo.toml
Alloy ecosystem crate versions upgraded from 0.6/0.8 to 1.6/2.0, enabling benchmarks to run against the latest stable releases.
Benchmark results and analysis
bench/RESULTS.md
Benchmark run recorded with 18/26 wins; all comparison table metrics (secp256k1, ABI, RLP, u256, hex, tx hash), score summary, key optimizations, and the expanded "Where alloy.rs Wins" list are updated to reflect new measurements.
README positioning
README.md
Main project README updated with new benchmark win count, revised performance claims, refreshed "Performance vs alloy.rs" section with updated operation timings/speedups, and new comparison table ranges.
Documentation portal
docs/content/docs/benchmarks.mdx, docs/content/docs/introduction.mdx, docs/content/docs/comparison.mdx, docs/content/docs/faq.mdx
All documentation pages updated with new benchmark scores, revised performance claims, updated operation timings and multipliers, and expanded "Where alloy.rs Wins" coverage.
Changelog and metadata
CHANGELOG.md, docs/src/app/layout.tsx
Changelog entry and SEO description metadata updated to reflect 18/26 benchmark win count.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Poem

🐰 A fresh new run with alloy bright,
Eighteen wins shining in the light,
From docs to READMEs, numbers spread,
Our benchmarks gleaming, burnished red!
eth.zig bounds ahead!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title "Refresh benchmark comparison against current alloy 1.6/2.0" directly and concisely describes the main objective of the changeset, which is to update benchmark comparisons against newer Alloy crate versions (1.6 and 2.0).
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore/alloy-bench-update

Warning

Review ran into problems

🔥 Problems

Git: Failed to clone repository. Please run the @coderabbitai full review command to re-trigger a full review. If the issue persists, set path_filters to include or exclude specific files.


Comment @coderabbitai help to get the list of available commands and usage tips.

@koko1123 koko1123 closed this Jun 10, 2026
@koko1123
koko1123 deleted the chore/alloy-bench-update branch June 10, 2026 15:20
@koko1123
koko1123 restored the chore/alloy-bench-update branch June 10, 2026 15:22
@koko1123 koko1123 reopened this Jun 10, 2026
@koko1123
koko1123 merged commit 2d74584 into main Jun 10, 2026
13 of 14 checks passed
@koko1123
koko1123 deleted the chore/alloy-bench-update branch June 10, 2026 15:22
@coderabbitai coderabbitai Bot mentioned this pull request Jun 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant