feat(hash): add compare command, bundle inputs, and plot --kind/--scale - #258
Merged
Conversation
hash_benchmark_report.py:
- compare: per-case Δ% (with a geomean summary) between two datasets.
- tables/plot/compare/quality accept a bundle .tgz or a results JSON
interchangeably (positionally or via --results/--bundle).
- plot --kind {throughput,latency,all} and --scale {log-log,linear-log};
x-axis stays log. log-log output is byte-identical (verify passes).
hash_benchmark.cc:
- expanded kFullSizes (rationale comments) and made the latency benchmark
sweep the full size set, so latency renders as a dense ns-vs-length curve.
Fab-Cat
approved these changes
Jul 13, 2026
- Add OS name to machine label (e.g. "macOS" instead of "Darwin"). - Add space between number and unit in size labels (e.g. "1 KiB" instead of "1KiB"). - Align all columns to the right in throughput and latency tables
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Measurement-tooling additions to
hash_benchmark_report.py, a denser latency benchmark, and reporting/consistency polish.New commands & inputs
compare— per-case Δ% (with a geomean summary row) between two datasets; negative = B faster (lower ns). Shows both machine labels; only cases present in both datasets are compared.tables/plot/compare/qualityaccept a bundle.tgzor a canonical results JSON interchangeably, positionally or via--results/--bundle(auto-detected by extension). Existing flag invocations still work.plot--kind{throughput,latency,all}— latency is now dense enough to render as a log-log curve.--scale{log-log,linear-log}—linear-log= linear y / log x (nice 0-based ticks) to read absolute ns gaps; x stays log always.Benchmark (
hash_benchmark.cc)kFullSizes(with rationale comments on the boundaries) and made the latency benchmark sweep the full size set, so latency plots as a dense ns-vs-length curve.Reporting polish
publishorders sections and the manifest by the generated header (machine label), so the README is stable regardless of the order bundles are passed.readme_sizessubset, like the throughput tables (no longer dumps every length now that latency sweeps the full set).B/KiBsuffix (e.g.1 KiB,4 KiB); the latency column now uses the same labels instead of raw numbers.MacOS/Linux).Fixes
%as%%in thecomparehelp string — a literal%is a printf format for argparse, which Python 3.14 validates eagerly at parser-construction time, so the tool crashed on 3.14 before this.Compatibility / follow-up
plotlog-log output is byte-identical to before (the tool's ownverifypassed on the prior chart set).publishregen (done separately with the final bundle set);verifycovers it.