mbo/hash: per-machine LFS measurement bundles + verifiable charts; complete benchmark sizes

.gitattributes

@@ -0,0 +1,7 @@
+# Git LFS tracks the per-machine mbo/hash measurement bundles (one .tgz per run,
+# holding the canonical results JSON, raw benchmark JSON, and SMHasher logs).
+# This directory is dev-only and .bazelignore'd, so it is never part of the
+# helly25_mbo module or any BCR offering; the generated BCR source archive
+# carries LFS pointer files here, which is harmless (nothing builds them).
+# See mbo/hash/measurements/README.md.
+mbo/hash/measurements/data/**/*.tgz filter=lfs diff=lfs merge=lfs -text

mbo/hash/hash_benchmark.cc

@@ -45,21 +45,44 @@ constexpr uint64_t kSeed = 5'381;
 // FAST set (default, CI, and the README tables): a dense set straddling every
 // dispatch-tier boundary and SSO cutoff so the small-key cliffs are visible -
 // 7/8 the fully-unrolled <=8 path, 15/16 the <=16 path (15 = libstdc++ SSO
-// cap), 22 the libc++ SSO cap, 47/48 and 63/64 the short-chain steps, with
-// 3/11/19/27 filling the small range and 256/1024/4096 the bulk.
-constexpr std::array<int, 18> kReadmeSizes = {1,  3,  7,  8,  11, 15, 16,  19,    22,
-                                              27, 32, 47, 48, 63, 64, 256, 1'024, 4'096};
+// cap), 22 the libc++ SSO cap, 38/47/48 and 63/64 the short-chain steps, 127/128
+// bracketing the chain->bulk 128-byte-window edge, with 3/5/11/19/27 filling the
+// small range and 256/1024/4096 the bulk.
+constexpr std::array<int, 22> kReadmeSizes = {
+    1, 3, 5, 7, 8, 11, 15, 16, 19, 22, 27, 32, 38, 47, 48, 63, 64, 127, 128, 256, 1'024, 4'096,
+};
 
 // FULL set (MBO_HASH_BENCHMARK_FULL=1): ~3x denser, a slow exponential (ratio
 // ~1.2) from 1..4096 unioned with the boundary set above, so the ns-vs-length
 // curve is smooth and the tier edges stay sampled. For the complete dataset /
 // graph, not for the README tables.
-constexpr std::array<int, 51> kFullSizes = {
-    1,   2,   3,   4,   5,   6,   7,   8,     9,     11,    13,    15,    16,    18,    19,    22,    27,
-    32,  38,  46,  47,  48,  55,  63,  64,    66,    79,    95,    114,   137,   165,   198,   237,   256,
+constexpr std::array<int, 53> kFullSizes = {
+    1,   2,   3,   4,   5,   6,   7,   8,     9,     11,    13,    15,    16,    18,    19,    22,    27,    32,
+    38,  46,  47,  48,  55,  63,  64,  66,    79,    95,    114,   127,   128,   137,   165,   198,   237,   256,
     285, 342, 410, 492, 591, 709, 851, 1'021, 1'024, 1'225, 1'470, 1'764, 2'116, 2'540, 3'048, 3'657, 4'096,
 };
 
+// Table/chart consistency rule: kReadmeSizes (README tables) must be a subset of
+// kFullSizes (the throughput chart), so every table row has a matching point on
+// the curve. Add a README size to the full set too. Enforced at compile time.
+constexpr bool ReadmeSizesAreSubsetOfFull() {
+  for (const int want : kReadmeSizes) {
+    bool found = false;
+    for (const int have : kFullSizes) {
+      if (have == want) {
+        found = true;
+        break;
+      }
+    }
+    if (!found) {
+      return false;
+    }
+  }
+  return true;
+}
+
+static_assert(ReadmeSizesAreSubsetOfFull(), "every kReadmeSizes entry must also appear in kFullSizes");
+
 // The active throughput size set, selected once by the environment.
 std::span<const int> ThroughputSizes() {
   const char* const full = std::getenv("MBO_HASH_BENCHMARK_FULL");  // NOLINT(concurrency-mt-unsafe): startup only

mbo/hash/measurements/README.md

@@ -36,8 +36,9 @@ mbo/hash/measurements/
   hash_benchmark_report.py     # run / store / tables / plot / smhasher (stdlib only)
   build_smhasher3.sh           # reproducible SMHasher3 build (clone + fixes + install plugin + container gcc)
   smhasher3/mbohash.cpp        # in-house mumbo/jumbo and dumbo SMHasher3 registration (includes the real headers)
-  hash_benchmark_results.json  # canonical distilled results (committed provenance for the README tables)
-  data/                        # complete raw datasets + SMHasher3 logs (see "Data storage")
+  hash_throughput_64.svg       # published 64-bit throughput chart (committed; verifiable from data/)
+  hash_throughput_128.svg      # published 128-bit throughput chart (committed; verifiable from data/)
+  data/<os-arch-cpu>/*.tgz     # per-machine measurement bundles, Git LFS (see "Data storage")
 ```
 
 The C++ benchmark itself (`mbo/hash/hash_benchmark.cc`,
@@ -90,30 +91,41 @@ plus `min`, `median`, `mean` (full-set, for context) and the repetition count.
 The README tables use `best`; `best_cv` flags a case whose fast tail is itself
 noisy and worth re-measuring.
 
-## Data storage, compression, commit policy
-
-Measured sizes (fast/18-size set; the full/51-size set is ~2.8x): raw
-google/benchmark JSON is **1.1 MB** (fast) / **~3 MB** (full) - too large and
-too noisy (per-iteration timings) to text-dump; gzip takes it to **123 KB** /
-**~340 KB**. The distilled canonical JSON is **38 KB** (fast) / **~100 KB**
-(full), and with sorted keys it diffs cleanly. So:
-
-- **Canonical results** (`hash_benchmark_results.json`): distilled per-case
-  stats + context. Small and human-diffable - **committed as text**; it is the
-  provenance for the README tables and the input to the plotter. This is the
-  answer to "just text-dump them": the _canonical_ JSON, yes; the _raw_, no.
-- **Complete raw datasets** (`data/`): google/benchmark's full JSON (every
-  repetition), `data/<date>-<host>-<gitsha>.json.gz`, **gzip-compressed**. Only
-  needed for a `compare.py` U-test. Policy: commit the gzipped raw only for an
-  **authoritative** dataset (a clean `main` checkout - see the workflow note),
-  so history stays reproducible without accumulating a raw dump per dev push.
-  Dev-branch raws stay git-ignored (`data/` ignores everything but `.gitkeep`).
-
-## Regenerate
-
-The one-shot runner does all three steps (perf sweep, chart, SMHasher3 battery)
-and prints exactly what to commit. Run it from a clean `main` checkout so the
-provenance is authoritative:
+## Data storage, commit policy, contributions
+
+Authoritative numbers cannot come from CI (shared, virtualized runners with CPU
+scaling on - see "CI"), so measurements are contributed from real machines
+(Apple M5 Pro, AMD Zen5, Intel, ...). Each run produces a fair amount of data:
+the raw google/benchmark JSON gzips to ~123 KB (fast) / ~340 KB (full), the
+distilled canonical JSON is ~38 KB / ~100 KB, plus the per-algorithm SMHasher3
+logs. Across several machines that accumulates, so:
+
+- **Per-machine bundle, Git LFS**
+  (`data/<os>-<arch>-<cpu-brand>/<slug>_<cores>c_<gitsha8>_<stamp>.tgz`): one
+  gzipped tarball per run holds the _whole_ dataset - the canonical
+  `results.json`, the raw `*_raw.json.gz` (for `compare.py` U-tests), and the
+  `smhasher.json` + per-algorithm logs. It is Git-LFS-tracked (`.gitattributes`),
+  so the blobs stay out of the main pack; unpack to inspect or A/B. Keep the
+  current bundle per machine (replace on re-measure); LFS history retains olds.
+- **Published charts, plain git** (`hash_throughput_64.svg` / `_128.svg`): the
+  human-facing output embedded in the hash README. Small, and - the point -
+  _verifiable_: `hash_benchmark_report.py verify --bundle <tgz>` regenerates them
+  from the bundled canonical data and fails on any mismatch, so every published
+  figure is provably derived from a committed dataset and cannot be hand-edited
+  (`_svg_plot` is deterministic: geometry keyed only off the data, no wall-clock).
+  The README perf tables are likewise rendered from the canonical data (unpack a
+  bundle and run `tables`).
+- The loose run outputs in `data/` are staging only and git-ignored; `bundle`
+  packs them into the `.tgz`. This directory is dev-only and `.bazelignore`'d, so
+  it never enters the module or BCR (the release archive carries LFS pointer
+  files here, which is harmless).
+
+## Regenerate / contribute a machine
+
+Run the one-shot runner on the machine you want to add, from the repo root and a
+clean `main` checkout (so the provenance is authoritative). It runs the perf
+sweep, renders + verifies the charts, runs the SMHasher3 battery, packs the
+per-machine LFS bundle, and prints exactly what to commit:
 
 ```sh
 # Everything, SMHasher3 batteries 4-at-a-time (perf runs first and alone):
@@ -129,31 +141,32 @@ load-independent, so `--jobs` runs several concurrently - trading cores for
 wall-clock, not accuracy. SMHasher3 is built and run inside a container
 (`build_smhasher3.sh`), so the runner invokes its Linux binary via `docker run`.
 
-Or drive the steps individually:
+Or drive the steps individually (`bundle` packs a run; `verify` audits the
+published charts against a bundle; unpack a bundle for its canonical JSON):
 
 ```sh
-# From the repository root (the script shells out to `bazel run` there):
+# Perf sweep -> staged canonical results.json + raw.json.gz (from the repo root):
 python3 mbo/hash/measurements/hash_benchmark_report.py run \
     --reps 20 --warmup 0.05 \
-    --raw mbo/hash/measurements/data/$(date +%Y%m%d)-raw.json \
-    --out mbo/hash/measurements/hash_benchmark_results.json --tables
-
-# Re-render the README tables from the committed canonical JSON (no bazel):
-python3 mbo/hash/measurements/hash_benchmark_report.py tables \
-    --results mbo/hash/measurements/hash_benchmark_results.json
-
-# Plot ns-vs-length curves (dependency-free SVG):
-python3 mbo/hash/measurements/hash_benchmark_report.py plot \
-    --results mbo/hash/measurements/hash_benchmark_results.json --out /tmp/hash.svg
-
-# SMHasher3 quality over all algorithms (needs a built SMHasher3, see above):
-python3 mbo/hash/measurements/hash_benchmark_report.py smhasher \
-    --smhasher3 /path/to/SMHasher3 --algos all --out mbo/hash/measurements/smhasher_results.json
+    --raw mbo/hash/measurements/data/raw.json.gz \
+    --out mbo/hash/measurements/data/results.json --tables
+
+# Pack a run's staged outputs into the per-machine Git-LFS bundle:
+python3 mbo/hash/measurements/hash_benchmark_report.py bundle \
+    --results mbo/hash/measurements/data/<stamp>_results.json \
+    --include mbo/hash/measurements/data/<stamp>_raw.json.gz \
+             mbo/hash/measurements/data/<stamp>_smhasher.json
+
+# Verify the committed charts match a bundle's data (unpacks, re-renders, diffs):
+python3 mbo/hash/measurements/hash_benchmark_report.py verify \
+    --bundle mbo/hash/measurements/data/<slug>/<bundle>.tgz
+
+# Re-render the README tables / charts from a bundle's canonical JSON:
+tar xzOf <bundle>.tgz results.json > /tmp/results.json
+python3 mbo/hash/measurements/hash_benchmark_report.py tables --results /tmp/results.json
+python3 mbo/hash/measurements/hash_benchmark_report.py plot --results /tmp/results.json --out /tmp/hash.svg
 ```
 
-Output filenames are timestamp-prefixed (see below), so `--raw` / `--out` land
-as `data/YYYYMMDD_HHMMSS_...`.
-
 ## SMHasher3 quality (`smhasher`)
 
 `hash_benchmark_report.py smhasher --algos all --smhasher3 <path>` runs the
@@ -180,10 +193,12 @@ Last verified run (2026-07): **mumbo-64/jumbo-128** and **dumbo-64** all PASS
 
 ## Output filenames
 
-Every file the tool writes is prefixed `YYYYMMDD_HHMMSS_` (local wall clock, one
-stamp per invocation), so runs never overwrite each other and the filename
-records when it was produced. The committed canonical results file is therefore
-a specific timestamped file; `tables`/`plot` take an explicit `--results` path.
+Loose staging files the tool writes are prefixed `YYYYMMDD_HHMMSS_` (local wall
+clock, one stamp per invocation) so runs never overwrite each other. The
+committed artifact is the per-machine bundle
+`data/<os>-<arch>-<cpu-brand>/<slug>_<cores>c_<gitsha8>_<stamp>.tgz` - the slug
+derived from the dataset's own `uname` + CPU brand, the SHA from its provenance -
+so a bundle is self-identifying and collision-free across machines.
 
 ## CI
 
@@ -195,18 +210,18 @@ for architecture/compiler shape comparison, not a gate.
 
 ## Open items
 
-- **Authoritative `smhasher.json` not yet committed**: the in-house plugin now
-  builds cleanly at the pinned commit `6ab4343` (an include-order bug in
-  `smhasher3/mbohash.cpp` - `Hashlib.h` ahead of `Platform.h`, which defines
-  `seed_t` and the `FLAG_IMPL_*` enums - is fixed and pinned against clang-format;
-  the pin is correct, no re-pin needed). Run `run_measurements.py` on a clean
-  `main` checkout to land the machine-readable record; the README SMHasher scores
-  already stand from verified runs.
-- Optional guard: a check that the README tables match the committed canonical
-  JSON so they cannot silently drift (they match as of the latest authoritative
-  run below).
-
-Done: the authoritative perf dataset + gzipped raw are committed
-(`hash_benchmark_results.json`, `data/*_raw.json.gz`), and the log-log
-throughput curves (`hash_throughput_64.svg` / `_128.svg`) are committed and
-embedded in the hash README.
+- **Only the reference machine (Apple M5 Pro) has a committed bundle so far, and
+  it is perf-only.** Contribute other machines (AMD Zen5, Intel, ...) by running
+  `run_measurements.py` there; and re-run with the SMHasher3 battery to add
+  `smhasher.json` + logs to the M5 Pro bundle (the include-order fix makes that
+  build reproducible now; the README SMHasher scores already stand from verified
+  runs).
+- `verify` audits the published _charts_ against a bundle; the README perf
+  _tables_ are not yet auto-checked (they render from a bundle's `results.json`
+  via `tables`). A marker-delimited tables region + a `verify` extension would
+  close that.
+
+Done: per-machine Git-LFS bundles (canonical + raw + SMHasher in one `.tgz`) with
+a `verify` audit that regenerates the published charts from the bundled data;
+this replaced committing loose canonical/raw files. The SMHasher3 build is
+reproducible again (include-order fix).

mbo/hash/measurements/data/.gitignore

@@ -1,5 +1,9 @@
-# Raw datasets are large; commit policy is an open item (see ../README.md).
-# Keep the directory, ignore its contents for now.
+# Loose run outputs (results.json, raw.json.gz, smhasher.json, logs) are staging
+# only - they get packed into a per-machine LFS bundle `<slug>/<...>.tgz` by
+# `hash_benchmark_report.py bundle`. Commit the bundles (LFS-tracked, see
+# ../../../.gitattributes), not the loose files. See ../README.md.
 *
 !.gitkeep
 !.gitignore
+!*/
+!*/*.tgz