Skip to content

fix: record actual bench phase durations instead of TimerInfo start instants - #346

Open
0oyun wants to merge 1 commit into
axiom-crypto:mainfrom
0oyun:fix/bench-timer-duration
Open

fix: record actual bench phase durations instead of TimerInfo start instants#346
0oyun wants to merge 1 commit into
axiom-crypto:mainfrom
0oyun:fix/bench-timer-duration

Conversation

@0oyun

@0oyun 0oyun commented Jul 1, 2026

Copy link
Copy Markdown

Problem

BaseTester::bench_builder (in halo2-base/src/utils/testing.rs) stores
ark_std TimerInfo values in BenchStats:

pub struct BenchStats {
    pub vk_time: TimerInfo,
    pub pk_time: TimerInfo,
    pub proof_time: TimerInfo,
    pub verify_time: TimerInfo,
    ...
}

A TimerInfo only holds the Instant at which the timer started. The
downstream benches read the timing later like this:

stats.proof_time.time.elapsed(),
stats.verify_time.time.elapsed(),

Because .elapsed() is called on the start instant at report time, each
value measures "phase start → report time", not the duration of the phase
itself. Concretely:

  • proof_time.time.elapsed() includes the whole verification step (and any
    work between proving and the CSV write).
  • Earlier phases are inflated by everything that ran after them.

So the timings written to the *.csv bench outputs are wrong (overlapping and
inflated).

Affected bench writers: bn254/tests/{msm, fixed_base_msm, ec_add, pairing}.rs
and secp256k1/tests/ecdsa.rs.

Fix

Measure each phase with an explicit Instant and store std::time::Duration
in BenchStats:

let vk_timer = start_timer!(|| "Generating vkey");
let vk_start = Instant::now();
let vk = keygen_vk(&params, &builder).unwrap();
let vk_time = vk_start.elapsed();
end_timer!(vk_timer);

start_timer!/end_timer! are kept purely for their log output; the recorded
value is now the actual operation time. Downstream writers use the fields
directly (stats.proof_time / stats.verify_time), which are still
Duration, so the {:?} formatting is unchanged.

Testing

cargo check -p halo2-base and cargo check -p halo2-ecc --tests both pass.

…nstants

`BaseTester::bench_builder` stored ark_std `TimerInfo` values in `BenchStats`.
A `TimerInfo` only holds the `Instant` at which the timer *started*, so the
downstream benches that later read `stats.proof_time.time.elapsed()` measured
"phase start -> report time" rather than the phase's own duration. In practice
`proof_time` ended up including the subsequent verification (and every earlier
phase was inflated by all the work that ran after it), making the CSV timings
wrong.

Measure each phase with an explicit `Instant` and store `std::time::Duration`
in `BenchStats`; update the bn254/secp256k1 bench writers to use the fields
directly. `start_timer!`/`end_timer!` are kept purely for their log output.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 1, 2026 04:48

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes benchmark timing collection in BaseTester::bench_builder by recording per-phase durations instead of storing ark_std::TimerInfo start instants (which caused inflated/overlapping timings when .elapsed() was evaluated later).

Changes:

  • Switch BenchStats timing fields from TimerInfo to std::time::Duration, measured via Instant::now() + .elapsed() around each phase.
  • Keep start_timer!/end_timer! for log output, but decouple recorded stats from TimerInfo.
  • Update downstream bench CSV writers to use stats.proof_time / stats.verify_time directly.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
halo2-base/src/utils/testing.rs Record true phase durations (vk/pk/prove/verify) into BenchStats as Duration values.
halo2-ecc/src/secp256k1/tests/ecdsa.rs Write Duration fields directly to CSV instead of calling .time.elapsed() on stored start instants.
halo2-ecc/src/bn254/tests/pairing.rs Same CSV writer update for proof/verify timing fields.
halo2-ecc/src/bn254/tests/msm.rs Same CSV writer update for proof/verify timing fields.
halo2-ecc/src/bn254/tests/fixed_base_msm.rs Same CSV writer update for proof/verify timing fields.
halo2-ecc/src/bn254/tests/ec_add.rs Same CSV writer update for proof/verify timing fields.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants