fix(bench): pin single-threaded recipes to -j 1 by olantwin · Pull Request #51 · ShipSoft/aegir

olantwin · 2026-07-03T09:46:23Z

The geant4 module now derives its worker count from the framework parallelism, so the gun_st recipes must say -j 1 explicitly to stay single-threaded.

Only -j 1 runs are bitwise reproducible: at higher -j, events reach the worker kernel(s) in nondeterministic order and consume a continuous per-kernel RNG stream, even when the module's concurrency is 1 (measured and recorded in results/NOTES.md). Per-event seeding is the proper fix and follows in a separate PR.

Part 6 of the series (base: perf/rntuple-bind).

Summary by CodeRabbit

Documentation
- Added guidance on reproducible benchmark results and the effects of parallel execution.
- Documented when to use single-threaded runs for exact physics comparisons versus statistical comparisons.
Chores
- Updated benchmark and profiling commands to run single-threaded for consistent, reproducible results.

coderabbitai · 2026-07-03T09:46:30Z

Warning

Review limit reached

@olantwin, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 13 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 1575a9de-49ae-4678-9751-85b88f8a8213

📥 Commits

Reviewing files that changed from the base of the PR and between 19ae236 and 9feddd2.

📒 Files selected for processing (2)

docs/geant4-perf-notes.md
justfile

📝 Walkthrough

Walkthrough

The PR documents nondeterminism caused by multithreaded worker-kernel and RNG-stream assignment, and adds -j 1 to benchmark and profiling phlex commands.

Changes

Reproducible benchmark execution

Layer / File(s)	Summary
Document reproducibility behavior `docs/geant4-perf-notes.md`	Documents threading-related reproducibility differences and validation rules for exact or statistical comparisons.
Pin benchmark and profiling commands `justfile`	Adds `-j 1` to Geant4 benchmark, noop, flamegraph, and trace `phlex` invocations.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Possibly related PRs

ShipSoft/aegir#12: Introduced the profiling and benchmark workflows whose commands are pinned here.
ShipSoft/aegir#46: Uses benchmark measurements across different phlex concurrency settings.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description does not follow the repository template and omits the required checklist items.	Add the Checklist section with all three items and confirm the commit message, pre-commit, and CI status.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: pinning benchmark recipes to -j 1.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch fix/st-recipes-j1

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-07-22T12:04:06Z

Physics Metrics Comparison

1 configuration(s) have differences

Configurations compared: 1
With differences: 1

Configurations with Differences

`comparison_physics-smoke`

Files compared: 1
Histograms compared: 8
Fits compared: 0
Total differences: 4

Show differences

gun_st_nofield_smoke_validation.root:
  h_hit_edep.integral: 891 → 914 (Δ=2.581%)
  h_hit_z.integral: 891 → 914 (Δ=2.581%)
  h_hit_detector.integral: 891 → 914 (Δ=2.581%)
  h_hit_momentum.integral: 891 → 914 (Δ=2.581%)

About this comparison

Physics metrics are extracted from ROOT files and compared.
Default comparison uses 5% tolerance for histogram statistics
and 3σ for fit parameters.

Metrics stored in git notes: refs/notes/ci/physics-metrics/<config>

olantwin · 2026-07-22T15:49:03Z

@coderabbitai review

coderabbitai · 2026-07-22T15:49:11Z

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/geant4-perf-notes.md`:
- Around line 59-60: Update the phlex -j 1 reproducibility statement in the
documentation to refer specifically to the checked validation histograms
h_mc_multiplicity, h_mc_momentum, and h_mc_pdg being reproducible, rather than
claiming the full output is bitwise identical.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 2af2b9ce-440d-47a1-9367-56d342e3ab7a

📥 Commits

Reviewing files that changed from the base of the PR and between d94d5f6 and 19ae236.

📒 Files selected for processing (2)

docs/geant4-perf-notes.md
justfile

The geant4 module now derives its worker count from the framework parallelism, so the gun_st recipes must say -j 1 explicitly to stay single-threaded. Only -j 1 runs are bitwise reproducible: at higher -j, TBB hands successive events to different pool threads, each with its own lazily-built worker kernel and RNG stream, even when the module's concurrency is 1 (recorded in docs/geant4-perf-notes.md). This covers the profile-flamegraph and profile-trace recipes too: they default to the gun_st_bench workflow, so they need -j 1 to profile the single-threaded path rather than a multi-threaded run.

olantwin · 2026-07-23T09:33:22Z

@coderabbitai review

coderabbitai · 2026-07-23T09:33:30Z

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

olantwin force-pushed the fix/st-recipes-j1 branch from 6df4050 to 3d85317 Compare July 7, 2026 12:23

olantwin changed the base branch from perf/rntuple-bind to perf/hit-buffer-reserve July 7, 2026 12:23

olantwin force-pushed the fix/st-recipes-j1 branch from 3d85317 to 27ff670 Compare July 8, 2026 11:59

olantwin force-pushed the perf/hit-buffer-reserve branch 2 times, most recently from 0cb35be to 4c812ed Compare July 14, 2026 16:47

olantwin force-pushed the fix/st-recipes-j1 branch from 27ff670 to 4c2fb47 Compare July 15, 2026 13:06

olantwin force-pushed the perf/hit-buffer-reserve branch from 4c812ed to d41b3d2 Compare July 20, 2026 18:38

olantwin force-pushed the fix/st-recipes-j1 branch from 4c2fb47 to 48e43fc Compare July 20, 2026 18:38

olantwin force-pushed the perf/hit-buffer-reserve branch from d41b3d2 to 22e4daa Compare July 22, 2026 10:14

Base automatically changed from perf/hit-buffer-reserve to main July 22, 2026 11:43

olantwin force-pushed the fix/st-recipes-j1 branch from 48e43fc to 19ae236 Compare July 22, 2026 12:02

coderabbitai Bot reviewed Jul 22, 2026

View reviewed changes

Comment thread docs/geant4-perf-notes.md Outdated

olantwin force-pushed the fix/st-recipes-j1 branch from 19ae236 to 9feddd2 Compare July 23, 2026 08:10

olantwin merged commit 79ac2f5 into main Jul 23, 2026
7 checks passed

olantwin deleted the fix/st-recipes-j1 branch July 23, 2026 09:39

coderabbitai Bot mentioned this pull request Jul 23, 2026

fix(geant4): fail cleanly on unknown physics lists, count worker kernels #52

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(bench): pin single-threaded recipes to -j 1#51

fix(bench): pin single-threaded recipes to -j 1#51
olantwin merged 1 commit into
mainfrom
fix/st-recipes-j1

olantwin commented Jul 3, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Jul 3, 2026 •

edited

Loading

Review limit reached

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

github-actions Bot commented Jul 22, 2026 •

edited

Loading

Uh oh!

olantwin commented Jul 22, 2026

Uh oh!

coderabbitai Bot commented Jul 22, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

olantwin commented Jul 23, 2026

Uh oh!

coderabbitai Bot commented Jul 23, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

olantwin commented Jul 3, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jul 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

github-actions Bot commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Physics Metrics Comparison

Configurations with Differences

comparison_physics-smoke

Uh oh!

olantwin commented Jul 22, 2026

Uh oh!

coderabbitai Bot commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

olantwin commented Jul 23, 2026

Uh oh!

coderabbitai Bot commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

olantwin commented Jul 3, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 3, 2026 •

edited

Loading

github-actions Bot commented Jul 22, 2026 •

edited

Loading

`comparison_physics-smoke`

coderabbitai Bot commented Jul 22, 2026 •

edited

Loading

coderabbitai Bot commented Jul 23, 2026 •

edited

Loading