mbo/hash: per-machine LFS measurement bundles + verifiable charts; complete benchmark sizes - #245
Merged
Merged
Conversation
…charts Authoritative perf/quality numbers cannot come from CI (noisy shared runners), so measurements are contributed from real machines. Each run is now packed into one Git-LFS .tgz per machine under data/<os>-<arch>-<cpu-brand>/, holding the whole dataset (canonical results.json + raw.json.gz + smhasher.json + logs). Plain git keeps only the published charts, which `verify` regenerates from the bundle and diffs - so every published figure is provably derived from committed data and cannot be hand-edited. - hash_benchmark_report.py: `_platform_slug` + richer `_machine_augment` (cpu_model, compiler); `bundle` (pack a run into data/<slug>/<slug>_<cores>c_<gitsha8>_<stamp>.tgz) and `verify` (unpack, re-render the SVGs, diff vs the committed charts) subcommands. `_svg_plot` now writes a trailing newline so its output matches the end-of-file-fixer'd committed form (verify is exact). - run_measurements.py: bundles + verifies at the end; prints LFS commit hints. - .gitattributes: LFS-track mbo/hash/measurements/data/**/*.tgz; data/.gitignore keeps loose staging out and allows the bundles. - Migrated the #243 authoritative M5 Pro perf data into the first bundle (macos-arm64-apple-m5-pro), removing the loose hash_benchmark_results.json and data/*_raw.json.gz. Perf-only until a SMHasher battery run adds to it. - README: rewrote storage/regenerate/output/open-items for the bundle + verify + contribution policy.
…full Adds 5 (small-range density), 38 (mid-chain fill), and 127/128 (bracketing the chain->bulk 128-byte window edge) to the throughput size sets, so every dispatch-tier boundary is sampled. A constexpr static_assert enforces kReadmeSizes subset of kFullSizes, so every README-table row has a matching point on the full-set chart and a full run subsumes the README run (the tables are a filtered view of the full dataset).
helly25
enabled auto-merge (squash)
July 10, 2026 22:13
Fab-Cat
approved these changes
Jul 10, 2026
helly25
added a commit
that referenced
this pull request
Jul 11, 2026
…source of truth) Delete the Python _README_SIZES list. _throughput_table now reads its sizes from the data's own buckets (like _latency_table already did), so there is no second size list to drift from the C++ kReadmeSizes - which is the sole source of truth. run_measurements runs a FAST sweep (buckets == kReadmeSizes) for the README tables and the FULL sweep for the ns-vs-length chart, bundling both. This closes the drift that let #245's C++ size additions (5/38/127/128) silently miss the tables.
helly25
added a commit
that referenced
this pull request
Jul 11, 2026
…e of truth) (#249) * mbo/hash/measurements: sync _README_SIZES with kReadmeSizes (add 5/38/127/128) #245 completed the C++ kReadmeSizes (5, 38, 127/128) but not the Python _README_SIZES mirror that filters the README tables, so the published tables would silently omit those four sizes. Bring the mirror back in sync. * mbo/hash/measurements: read README-table sizes from the data (single source of truth) Delete the Python _README_SIZES list. _throughput_table now reads its sizes from the data's own buckets (like _latency_table already did), so there is no second size list to drift from the C++ kReadmeSizes - which is the sole source of truth. run_measurements runs a FAST sweep (buckets == kReadmeSizes) for the README tables and the FULL sweep for the ns-vs-length chart, bundling both. This closes the drift that let #245's C++ size additions (5/38/127/128) silently miss the tables.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Measurement-data infrastructure for
0.13.0, so perf/quality numbers can be contributed from real machines (CI runners are too noisy) and every published figure is verifiable against committed data.Per-machine LFS bundles
.tgzper run underdata/<os>-<arch>-<cpu-brand>/, holding the whole dataset (canonicalresults.json, raw*_raw.json.gz,smhasher.json+ logs)..gitattributestracksdata/**/*.tgz; loose run outputs stay git-ignored staging.hash_throughput_*.svg).hash_benchmark_report.py verify --bundle <tgz>regenerates them from the bundled data and diffs, so a published chart is provably derived from a committed dataset and cannot be hand-edited (_svg_plotis deterministic). README tables are a filtered view of the same canonical data.bundle/verifysubcommands +_platform_slug+ richer provenance (cpu_model,compiler);run_measurements.pybundles + verifies at the end.Completed benchmark size set
5(small-range density),38(mid-chain fill), and127/128(bracketing the chain->bulk 128-byte-window edge), so every dispatch-tier boundary and SSO cap is sampled.4096stays the ceiling (by ~1-4 KB both single- and multi-lane hashes are at steady-state throughput).static_assert(ReadmeSizesAreSubsetOfFull())enforceskReadmeSizes ⊆ kFullSizes: every README-table row has a matching chart point, and a full run subsumes the README run (the tables are just a filtered view). Negative-tested (it fails the build on a violation).Note: LFS endpoint
Activating LFS: the
http://remote works for git (redirects) but hangs on the LFS batch API; LFS must use HTTPS (git config lfs.url https://.../info/lfs, or an HTTPS/SSH remote as on normal clones).Before merge (WIP)
The committed M5 Pro bundle is placeholder #243 data (old 18-size sweep). The authoritative runs on an idle Apple M5 Pro and AMD Zen5 with the new sizes will replace it + add the Zen5 bundle + refresh the charts/tables. Those land on this PR before merge; then
0.13.0(version triple already aligned) gets tagged.