perf: share calibrated work across fresh benchmark processes - #1605
Draft
mrousavy wants to merge 1 commit into
Draft
perf: share calibrated work across fresh benchmark processes#1605mrousavy wants to merge 1 commit into
mrousavy wants to merge 1 commit into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Independent calibration lets a slower binary perform fewer operations and accumulate less runtime state. This derives one per-case work plan from base, discards calibration processes, then runs fresh base/head processes with identical iteration counts, chunk sizes, five warmup batches and twenty ordered measurements. Head slowdowns never shorten its work. Incompatible counts fail explicitly.
Changed suite/runner definitions now produce two head-only baseline measurements, without running an incompatible base or inventing a Bencher baseline. This also provides a clean first rollout of the calibration protocol. Suite hashes remain the source of comparability. Calibration results are explicitly separate and contain no measured samples. The receiver now finishes its accepted HTTP response before closing, fixing a reproduced connection-reset race.
Validation: 59 host tests; tooling and benchmark TypeScript checks; benchmark lint; actionlint. A real controller/HTTP-receiver integration fixture launches 10 distinct processes for shared ABBA calibration/measurement and 6 for a changed-suite baseline, verifies reversed case mapping and identical work, and fails if the old base receives the new protocol. Runner tests preserve the full work at 2脳 and 10脳 slowdown and retain slow samples. These are host/process tests, not a claim that simulator performance is stable or that shared counts eliminate drift.
Stacked on #1604. Merge predecessors first; the trusted publisher and changed-suite raw contract roll out with this PR. CI is intentionally cancelled immediately after PR creation to conserve Actions minutes; remote CI is not claimed to pass. No merge requested.