Skip to content

feat(hash): add compare command, bundle inputs, and plot --kind/--scale - #258

Merged
helly25 merged 3 commits into
mainfrom
feat/hash-measurements-compare-plot
Jul 13, 2026
Merged

feat(hash): add compare command, bundle inputs, and plot --kind/--scale#258
helly25 merged 3 commits into
mainfrom
feat/hash-measurements-compare-plot

Conversation

@helly25

@helly25 helly25 commented Jul 13, 2026

Copy link
Copy Markdown
Owner

Summary

Measurement-tooling additions to hash_benchmark_report.py, a denser latency benchmark, and reporting/consistency polish.

New commands & inputs

  • compare — per-case Δ% (with a geomean summary row) between two datasets; negative = B faster (lower ns). Shows both machine labels; only cases present in both datasets are compared.
  • Unified inputstables/plot/compare/quality accept a bundle .tgz or a canonical results JSON interchangeably, positionally or via --results/--bundle (auto-detected by extension). Existing flag invocations still work.

plot

  • --kind {throughput,latency,all} — latency is now dense enough to render as a log-log curve.
  • --scale {log-log,linear-log}linear-log = linear y / log x (nice 0-based ticks) to read absolute ns gaps; x stays log always.

Benchmark (hash_benchmark.cc)

  • Expanded kFullSizes (with rationale comments on the boundaries) and made the latency benchmark sweep the full size set, so latency plots as a dense ns-vs-length curve.

Reporting polish

  • publish orders sections and the manifest by the generated header (machine label), so the README is stable regardless of the order bundles are passed.
  • Latency table uses the curated readme_sizes subset, like the throughput tables (no longer dumps every length now that latency sweeps the full set).
  • Size labels unified everywhere (tables + chart x-axis): consistent B/KiB suffix (e.g. 1 KiB, 4 KiB); the latency column now uses the same labels instead of raw numbers.
  • Machine label / headers now include the OS (MacOS/Linux).
  • Throughput and latency table columns are right-aligned.

Fixes

  • Escaped % as %% in the compare help string — a literal % is a printf format for argparse, which Python 3.14 validates eagerly at parser-construction time, so the tool crashed on 3.14 before this.

Compatibility / follow-up

  • plot log-log output is byte-identical to before (the tool's own verify passed on the prior chart set).
  • The OS/label and size-label changes alter chart subtitles and table text, so the committed README perf section + charts need a publish regen (done separately with the final bundle set); verify covers it.

hash_benchmark_report.py:
- compare: per-case Δ% (with a geomean summary) between two datasets.
- tables/plot/compare/quality accept a bundle .tgz or a results JSON
  interchangeably (positionally or via --results/--bundle).
- plot --kind {throughput,latency,all} and --scale {log-log,linear-log};
  x-axis stays log. log-log output is byte-identical (verify passes).

hash_benchmark.cc:
- expanded kFullSizes (rationale comments) and made the latency benchmark
  sweep the full size set, so latency renders as a dense ns-vs-length curve.
@helly25
helly25 requested a review from Fab-Cat July 13, 2026 21:22
helly25 added 2 commits July 13, 2026 23:15
- Add OS name to machine label (e.g. "macOS" instead of "Darwin").
- Add space between number and unit in size labels (e.g. "1 KiB" instead of "1KiB").
- Align all columns to the right in throughput and latency tables
@helly25
helly25 enabled auto-merge (squash) July 13, 2026 23:06
@helly25
helly25 merged commit cc23c11 into main Jul 13, 2026
22 checks passed
@helly25
helly25 deleted the feat/hash-measurements-compare-plot branch July 13, 2026 23:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants