From d03580bf362fbb28a7de802861fc85d9651135ce Mon Sep 17 00:00:00 2001 From: helly25 <6420169+helly25@users.noreply.github.com> Date: Sun, 12 Jul 2026 17:49:57 +0100 Subject: [PATCH 1/8] mbo/hash: generate mangle-seed header per build; dumbo/fnv1a in Starlark The checked-in mangle-seed fallback carried a version-derived constant, so every version bump made it stale and broke hash_mangle_seed_default_test (the CI failure on release PRs, e.g. #252). Generate the header per build and stop committing it: remove internal/hash_mangle_seed.h.in and the diff test; hash_mangle.h includes the generated header directly and #errors when it is missing (clangd falls back to a stable constant under -DIS_CLANGD). A version bump no longer needs a committed regeneration, and the --//mbo/hash:mangle_seed* flags are unaffected. Also switch the version/seed fold from FNV-1a to the in-house dumbo hash (SMHasher3-proven) and add Starlark ports of dumbo and fnv1a in //mbo/hash:hash.bzl (public hash struct), kept byte-for-byte identical to the C++ prime implementation and verified against it by hash_bzl_vs_cpp_dumbo_test and hash_bzl_vs_cpp_fnv1a_test via the new //mbo/hash:hash_tool CLI. README and CHANGELOG updated. Verified: bumping 0.13.1 -> 0.13.3 rotates the generated constant and all hash tests pass; reverting restores it and all tests pass. Also condensed the 0.13.0 CHANGELOG entries to the terse, verb-first house style (presentation only; released content unchanged). --- CHANGELOG.md | 81 ++++++++-------- mbo/hash/BUILD.bazel | 51 ++++++++-- mbo/hash/README.md | 78 ++++++++++----- mbo/hash/hash.bzl | 124 ++++++++++++++++++++++++ mbo/hash/hash_mangle.h | 18 ++-- mbo/hash/hash_tool.cc | 82 ++++++++++++++++ mbo/hash/internal/hash_bzl_verify.bzl | 93 ++++++++++++++++++ mbo/hash/internal/hash_mangle_seed.bzl | 37 +++---- mbo/hash/internal/hash_mangle_seed.h.in | 50 ---------- 9 files changed, 460 insertions(+), 154 deletions(-) create mode 100644 mbo/hash/hash.bzl create mode 100644 mbo/hash/hash_tool.cc create mode 100644 mbo/hash/internal/hash_bzl_verify.bzl delete mode 100644 mbo/hash/internal/hash_mangle_seed.h.in diff --git a/CHANGELOG.md b/CHANGELOG.md index 575c987..645f08b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,45 +1,50 @@ # 0.13.1 +- Switched the build-seed mangle constant header to per-build generation (no longer committed): removed `internal/hash_mangle_seed.h.in` and `hash_mangle_seed_default_test`, so a version bump no longer needs a committed regeneration. `hash_mangle.h` includes the generated header directly (missing is an `#error`; clangd falls back under `-DIS_CLANGD`). +- Added Starlark ports of the `dumbo` and `fnv1a` hashes in `//mbo/hash:hash.bzl` (the public `hash` struct), byte-identical to C++ and verified against it. +- Switched the mangle version/seed fold from FNV-1a to the in-house `dumbo` hash. +- Added `//mbo/hash:hash_tool`, a minimal ` []` hash CLI (plain `GetHash64`, no mangle). + # 0.13.0 -- Added `DiffOptions::ignore_missing_final_newline` (the `mbo::diff` `--ignore_missing_final_newline` flag and the `diff_test` `ignore_missing_final_newline` attribute): a file with and one without a trailing newline compare equal (the `\ No newline at end of file` marker is suppressed). It only ignores the terminator, so an empty file stays distinct from a single empty line. Honored identically by all three algorithms (shared `Data` preprocessing). -- `mbo::diff` now treats an empty `DiffOptions::time_format` as "omit the timestamp": the unified/context file header becomes a git-style `--- name` / `+++ name` (name only, no per-file mtime), so the output is reproducible across machines and time zones. A non-empty `time_format` is unchanged, and `time_format` stays library-only. -- Added a Myers diff algorithm (`mbo::diff::DiffMyers`, "An O(ND) Difference Algorithm and Its Variations", the algorithm behind GNU diff and git) and made it the default (`DiffOptions::Algorithm::kMyers`, `--algorithm=myers`). Lines are interned into integer tokens (honoring all ignore/strip/replace options), the linear-space middle-snake divide and conquer produces _minimal_ diffs, and past a cost cap of max(64, sqrt(L+R)) a git-style furthest-reaching split bounds pathological inputs. Compared to the previous default the common scattered-edits case is >2x faster and the disjoint-files worst case drops from ~16s to <1ms (2k lines, `//mbo/diff:diff_benchmark`). -- Sped up the `myers` tokenizer: interning keys are now `std::string_view`s into the preprocessed line cache hashed with `mbo::hash::GetHash64` (previously every line was copied into a `std::string`-keyed map), and `ignore_case` folds each line once into a reused buffer with stable storage only for distinct lines. Tokenize-heavy inputs (20k distinct 120-char lines) run ~16% faster; see the `tokenize_*` cases in `//mbo/diff:diff_benchmark` (whitespace-ignore variants included; `ignore_all_space` now also builds its stripped line in place instead of via an extra copy). -- The `mbo::diff` binary follows the POSIX diff exit code contract: 0 = equal, 1 = different, **2 = trouble** (unreadable input, internal error, bad usage; previously conflated with 1). The bazel `diff_test` rule gained the `minimal` attribute, and `mbo/diff/TODO.md` records the remaining optional features and explicit non-goals. -- Rounded out the `mbo::diff` CLI/test surface: new `--minimal` flag (`DiffOptions::minimal`) guaranteeing minimal `myers` diffs by disabling the cost cap (like GNU `diff --minimal`); the usage message names all algorithms and formats; the bazel `diff_test` rule gained `width` and `skip_left_deletions` attributes; and a CLI bashtest now checks the full algorithm x format matrix against per-engine expected outputs, so an untested or unsupported combination fails by name. `max_diff_chunk_length` and `time_format` intentionally stay library-only. -- Renamed the previous default `mbo::diff` algorithm from `unified` to what it is: `naive` (`DiffOptions::Algorithm::kNaive`, `mbo::diff::DiffNaive`, `mbo/diff/impl/diff_naive.*`). It greedily resynchronizes on the closest matching line and does not produce minimal diffs. The flag/attribute value `unified` remains supported as a deprecated alias that now selects `myers` - matching its historic "like `diff -u`" promise - and enforces `--format=unified`. -- Verified the `mbo::diff` feature/algorithm support matrix with tests: all comparison options (`ignore_case`, `ignore_all_space`, `ignore_consecutive_space`, `ignore_trailing_space`, `ignore_blank_lines`, `ignore_matching_lines`, `strip_comments`, `regex_replace_*`) work identically under `naive`, `myers` and `direct`, and all three output formats work with every algorithm. One documented corner case: combining `ignore_case` with a case-sensitive `ignore_matching_lines` expression - a matching and a non-matching line differing only in case compare equal under `naive` but not under the token-based `myers`; write such expressions case-insensitively (`(?i)...`). `max_diff_chunk_length` only applies to `naive` (`myers` uses an internal cost cap, `direct` needs no bound). -- Added `mbo::diff` output formats: next to the default unified format, `DiffOptions::output_format` selects context format (`diff -c`) or normal format (plain `diff`). The `diff` binary and the `diff_test` bazel rule gained the matching `--format=unified|context|normal` flag / `format` attribute (normal format defaults `--context` to 0 and emits no file headers; context format uses `***`/`---` file headers). Both new formats reproduce GNU diff output byte for byte and apply cleanly with `patch`. -- Fixed `mbo::diff` unified output for empty ranges (pure insertions or deletions, visible with `--context=0`): the chunk header now references the line _preceding_ the gap (e.g. `@@ -2,0 +3 @@` instead of `@@ -3,0 +3 @@`), matching GNU diff. Previously `patch` applied such hunks one line too late. -- Split the `mbo::diff` chunk rendering out of `mbo/diff/internal/chunk.cc` into `mbo/diff/internal/output.{h,cc}` (`diff_internal::AppendChunk`); `Chunk` now only accumulates and filters. -- Added `mbo::hash::GetHash64(std::string_view)` / `GetHash128(std::string_view)` and the `mbo::hash::Hash128` result type - constexpr-safe, non-cryptographic hashing; the default algorithm is the in-house `mumbo` (see below). -- Renamed the legacy in-house hash `mbo::hash::simple` to `mbo::hash::dumbo` and **redesigned** it into a compact single-lane MUM hash. Nothing-up-my-sleeve constants (golden ratio, sqrt-prime fractions), a widening `Mul128Fold64` step over 8-byte words, and a two-multiply seed-injected finalizer take it from the legacy SMHasher3 40/188 to a clean **PASS 188/188** (all three in-house hashes are now clean), ~2-3x faster than the legacy hash for >=8 B, and the fastest hash in the suite for tiny keys - single-lane, so it slows on large keys; a deliberately minimal companion to `mumbo`, not a replacement (see `mbo/hash/README.md` for the measured design iterations). The deprecated `simple::GetHash` wrapper and the `mbo::hash::simple` namespace are **removed** outright - pre-1.0, so no compatibility alias is kept and existing `mbo::hash::simple::*` users must migrate. Use `mbo::hash::GetHash64` (mumbo) as the default; `mbo::hash::dumbo::GetHash64(data, seed)` is the seeded, SMHasher3-clean minimal option. Values are not stable across library versions. -- All `mbo::hash` entry points (`GetHash64`, `GetHash128`, `GetHash`) are now templates over an algorithm struct (default `DefaultHashAlgorithm`, i.e. `mbo::hash::mumbo`). Every algorithm provides a `::Algorithm` struct with static `GetHash64`/`GetHash128` members; the concepts `HasGetHash64`/`HasGetHash128`/`IsHashAlgorithm` detect what is available, and `Hasher` completes partial algorithms with fallbacks (128->64 fold; for a missing 128 two decorrelated 64-bit passes where the second skips the first up-to-8 bytes and injects them via the seed, so both lanes cover every byte and differ even for seed-ignoring algorithms). -- Added constexpr-safe, canonical implementations of further hash algorithms, all usable as algorithm structs with `GetHash64`: `mbo::hash::fnv1a::GetHash64` (FNV-1a 64), `mbo::hash::xxh64::GetHash64` (XXH64), `mbo::hash::xxh3::GetHash64` (XXH3 64-bit, scalar), and `mbo::hash::murmur3::GetHash64/GetHash128` (MurmurHash3 x64 128). All produce the published reference values on every platform (little-endian defined). -- Exposed `mbo::hash::Hash128To64(Hash128)`: folds a 128-bit hash into a well-mixed 64-bit one, e.g. to derive a fold-mixed 64-bit value from `murmur3::GetHash128` (whose `GetHash64` is the canonical `h1` truncation instead). -- Added the build-seed mangle as its own entry point `mbo/hash/hash_mangle.h` (`//mbo/hash:hash_mangle_cc`): `GetHash` and `MangledHasher` XOR ONE build-selected constant into `GetHash64` values (still constexpr), so values deliberately do not compare across independently configured builds; plain `hash.h` / `:hash_cc` stays fully deterministic and is never exposed to (or rebuilt for) seed variation. The constant lives in one generated header per configuration (consistent per program by construction), folding the module's own version (read from `MODULE.bazel` via `native.module_version()` - no duplicated version declaration, correct also when consumed via BCR - so every release rotates mangled values at zero marginal cost: a release recompiles dependents anyway) with the custom Bazel flags `--//mbo/hash:mangle_seed` (any printable-ASCII string; folded to a bucket inside the header-generation rule, so build/remote caches see a bounded set of variants no matter what rotates through the flag) and `--//mbo/hash:mangle_seed_buckets` (default `8`; `0` disables the mangle making `GetHash == GetHash64`, `1` pins one stable constant across releases and seeds). -- Runtime loads use `memcpy` (gcc never folded the byte-assembly loads - roughly 3x on gcc for **all** algorithms) and tail loads use branch-lite overlapping reads; constant evaluation keeps the byte-assembly path (values identical, guarded by tests). -- `Hasher` is also a transparent functor, usable directly as the hash parameter of `absl`/`std` hash containers with heterogeneous `std::string_view` lookup. -- Added `mbo::hash::CombineHashes(uint64_t, uint64_t)` (order-dependent, well-mixed combine) and `hash_internal::Mul128Fold64` (constexpr 64x64->128 fold, xxh3/wyhash family core). -- Test framework additions: seed-bit avalanche (SMHasher-style; skipped for seedless algorithms) and structured/sparse-key distinctness (all-zero lengths, single-bit keys, cyclic patterns). -- Added canonical, constexpr-safe `mbo::hash::xxh3::GetHash128` (`XXH3_128bits[_withSeed]`, the modern fast file-checksum format), verified against reference vectors and differentially against libxxhash; `xxh3::Algorithm` is now 128-bit native. -- Added canonical, constexpr-safe `mbo::hash::rapidhash::GetHash64` (rapidhash V3, wyhash family - best small-key latency) and `mbo::hash::siphash::GetHash64` / `SipHash` (SipHash-2-4/-1-3, the keyed hash-flooding-resistant PRF), both verified against reference vectors. -- Added a differential test comparing the xxh64/xxh3 implementations bit-for-bit against the actual reference library (test-only `xxhash` archive) over randomized inputs, seeds, and lengths. -- Added `mbo::hash::Hash64To32(uint64_t)`: XOR-fold shrink to 32 bits (all 64 bits contribute; the official FNV shrinking recommendation, safe for every algorithm). -- Added `mbo::hash::GetHash32(data, seed)` and `Hasher::GetHash32` with the `HasGetHash32` concept: algorithms may provide a native 32-bit variant; otherwise the XOR-fold of the 64-bit hash is synthesized. -- Added a mixed-length latency benchmark (`BmHash64Latency`): unpredictable key sizes with a serialized dependency chain, measuring what hash-table workloads actually pay. -- Added streaming/incremental hashing: the `HasStreaming` concept and `Streamer` wrapper (`Update(...).Finalize()`, non-destructive, constexpr-safe), with chunked results guaranteed equal to the one-shot value. Implemented for `mumbo`, `xxh64` (canonical streaming semantics), and `siphash`; `rapidhash` has no canonical streaming form and honestly opts out. -- Added a repository-root `NOTICE` file reproducing the upstream notices of the transcribed algorithms (rapidhash MIT, xxHash BSD-2, MurmurHash3/SipHash/FNV public domain or CC0); README links it. -- Added **`mumbo`**, the library's own hash algorithm and the default behind `GetHash64`/`GetHash128`/`GetHash32`/`GetHash` and streaming: built on the widening 64x64->128 multiply ("MUM" - one multiply absorbs 16 bytes and diffuses full-width both directions), with fully unrolled per-length small-key loads (data in both product operands), a 128-byte 8-chain bulk fetch window, a native dual-lane 128-bit form, and a finalizer that keeps both widening-product halves with the length folded into the product operands (which is also what enables streaming). Secrets are the sqrt fractions of the first 16 primes. **SMHasher3: PASS 188/188 in both widths** - the only clean native-128 result measured on our rig - plus best-in-class mixed-length latency at <= 16 bytes and bulk throughput (measured design iterations and full tables: `mbo/hash/README.md`). -- Split the NOTICE-bearing transcriptions into **`//mbo/hash:hash_extra_cc`** (`mbo/hash/hash_extra.h`): `rapidhash` (MIT) and `xxh64`/`xxh3` (BSD-2-Clause) now require an explicit dependency (and shipping the repository-root NOTICE); the default `//mbo/hash:hash_cc` contains only notice-free code. All extras remain fully supported `IsHashAlgorithm` plug-ins. -- Hash values are not guaranteed stable across library versions and are not intended for persistence or cryptographic use. -- Added the **`mbo/digest`** library (charter: `mbo/digest/README.md`): spec-transcribed, constexpr-safe message digests with identical compile-time and runtime values (`static_assert`-proven). Algorithms: **SHA-224/256/384/512, SHA-512/224, SHA-512/256** (FIPS 180-4; the SHA-512/t IVs rederived programmatically per the spec), **SHA3-224/256/384/512** and the **SHAKE128/256** XOFs (FIPS 202; `Digest` for any output length; different lengths share their prefix), **BLAKE2b**/`blake2b_256` (RFC 7693, incl. native keying via `DigestKeyed`/`StreamInitKeyed` - BLAKE2 is its own MAC), **BLAKE3** with the full Merkle-tree structure (plain/`DigestXof`, keyed, and `DeriveKey` KDF modes; pinned against the official test-vector suite, all 35 lengths x 3 modes), and **SHA-1** + **MD5** for legacy interop (both loudly marked collision-broken). Every value pinned against independently generated reference vectors (FIPS/RFC examples, per-algorithm padding boundaries, million-byte inputs). -- The digest API mirrors mbo/hash's plug-in architecture: per-algorithm `Algorithm` structs with the `IsDigestAlgorithm`/`HasStreaming` concepts, the incremental `mbo::digest::Streamer` (peekable finalize), **`Hmac`/`HmacStreamer`** (RFC 2104, generic over any streaming digest incl. HMAC-SHA3 with rate-sized blocks), and `ToHexString`. Digests take no seed - keying is native (BLAKE2b/BLAKE3) or HMAC's job. -- Added the `digest` binary (`//mbo/digest:digest`): checksum-style ` ` lines (`sha256sum`/`shasum` format, byte-compatible; `--reverse` swaps the columns), `-a`/`--algorithm` selects any of the 17 library algorithms (default sha256), `-` reads stdin, directories are errors (`-d`/`--ignore_directories` skips them silently); streaming chunked reads (no whole-file buffering). Tested by a bashtest matrix diffing every algorithm's output against independently generated expected files. -- Added `--check` (short: `-c`) to the `digest` binary: verifies checksum files (`OK` / `FAILED` per listed file, coreutils-style warnings and exit codes; accepts `*` binary markers and uppercase hex, so sum files are interchangeable with `sha256sum`/`shasum` in both directions), with the companions `--quiet`, `--status`, `--ignore_missing`, and `--strict`. -- Added the bazel `verify_digest_test` rule (`//mbo/digest:digest.bzl`): given an `algorithm` and files mapped to a saved digest - either `digests` (a checksum sidecar file) or `checksums` (an inline hex digest in the BUILD file) - it re-digests each file with `//mbo/digest:digest --check`, so a file may only change when its saved digest is updated in the same commit and nothing drifts unnoticed. The `digests` sidecar form is preferred: the sidecar stays an independent, externally verifiable artifact anyone can re-check with stock `sha256sum -c`, no Bazel required. It pins `mbo/hash/hash_test_vectors.inc` (the generated known-answer vectors) alongside a `diff_test` that fails if that file is stale versus its generator. -- Factored the shared load primitives into the new `//mbo/hash:hash_internal_util` target (used by `mbo/digest`) and added the big-endian `hash_internal::Load32BE`/`Load64BE` loads (digest specifications are big-endian) with the same `memcpy`-based runtime path as the little-endian loads. +- Added `DiffOptions::ignore_missing_final_newline` (the `--ignore_missing_final_newline` flag and the `diff_test` attribute): a file with and one without a trailing newline compare equal. +- Made an empty `DiffOptions::time_format` omit the file-header timestamp (git-style `--- name` / `+++ name`), so output is reproducible across machines and time zones. +- Added a Myers diff algorithm (`DiffMyers`, `--algorithm=myers`) and made it the default: minimal diffs, >2x faster on scattered edits, and the disjoint-files worst case dropped from ~16s to <1ms. +- Sped up the `myers` tokenizer (`string_view` interning keys hashed with `mbo::hash::GetHash64`, single-fold `ignore_case`): ~16% faster on tokenize-heavy inputs. +- Made the `mbo::diff` binary follow the POSIX exit-code contract (0 equal, 1 different, 2 trouble) and gave `diff_test` a `minimal` attribute. +- Rounded out the `mbo::diff` CLI/test surface: `--minimal`, algorithm/format names in the usage message, `diff_test` `width`/`skip_left_deletions` attributes, and a CLI bashtest over the algorithm x format matrix. +- Renamed the previous default `mbo::diff` algorithm `unified` to `naive` (`DiffNaive`); `unified` stays a deprecated alias that now selects `myers`. +- Verified the `mbo::diff` feature/algorithm support matrix with tests across `naive`, `myers`, and `direct`. +- Added `mbo::diff` context (`diff -c`) and normal (plain `diff`) output formats (`--format=unified|context|normal`), byte-compatible with GNU diff and `patch`. +- Fixed `mbo::diff` unified output for empty ranges so chunk headers match GNU diff (and apply cleanly with `patch`). +- Split `mbo::diff` chunk rendering into `mbo/diff/internal/output.{h,cc}` (`AppendChunk`). +- Added `mbo::hash::GetHash64` / `GetHash128` and the `Hash128` type: constexpr-safe, non-cryptographic hashing with the in-house `mumbo` as default. +- Renamed the legacy `mbo::hash::simple` to `mbo::hash::dumbo` and redesigned it into a compact single-lane MUM hash (SMHasher3 PASS 188/188); removed the `simple` namespace outright (pre-1.0, no alias). +- Made all `mbo::hash` entry points templates over an algorithm struct (per-algorithm `Algorithm`, the `HasGetHash*`/`IsHashAlgorithm` concepts, and `Hasher` fallbacks). +- Added constexpr-safe `fnv1a`, `xxh64`, `xxh3` (64-bit), and `murmur3` (64/128), all matching published reference values. +- Exposed `mbo::hash::Hash128To64(Hash128)` to fold a 128-bit hash into a mixed 64-bit one. +- Added the build-seed mangle entry point `hash_mangle.h` (`GetHash` / `MangledHasher`): XORs one build-selected constant (from the module version and the `--//mbo/hash:mangle_seed*` flags) into `GetHash64` values, so they deliberately do not compare across independently configured builds. +- Switched runtime loads to `memcpy` (~3x faster on gcc) with branch-lite overlapping tail reads; kept the byte-assembly path for constant evaluation. +- Made `Hasher` a transparent functor, usable directly by `absl`/`std` hash containers with heterogeneous `string_view` lookup. +- Added `mbo::hash::CombineHashes` and `hash_internal::Mul128Fold64`. +- Added seed-bit avalanche and structured/sparse-key distinctness tests. +- Added constexpr-safe `xxh3::GetHash128` (`XXH3_128bits`), verified against reference vectors and libxxhash. +- Added constexpr-safe `rapidhash::GetHash64` (rapidhash V3) and `siphash::GetHash64` / `SipHash`, verified against reference vectors. +- Added a differential test comparing `xxh64`/`xxh3` bit-for-bit against the reference library over randomized inputs. +- Added `mbo::hash::Hash64To32` (XOR-fold to 32 bits). +- Added `GetHash32` / `Hasher::GetHash32` and the `HasGetHash32` concept (native variant or synthesized fold). +- Added a mixed-length latency benchmark (`BmHash64Latency`). +- Added streaming hashing (the `HasStreaming` concept and `Streamer`) for `mumbo`, `xxh64`, and `siphash`. +- Added a repository-root `NOTICE` reproducing the upstream notices of the transcribed algorithms. +- Added `mumbo`, the library's own MUM-based hash and the default behind `GetHash*`: SMHasher3 PASS 188/188 in both widths, with best-in-class small-key latency and bulk throughput. +- Split the NOTICE-bearing transcriptions into `//mbo/hash:hash_extra_cc` (`rapidhash`, `xxh64`/`xxh3`); the default `//mbo/hash:hash_cc` stays notice-free. +- Documented that hash values are not stable across library versions and are not for persistence or cryptographic use. +- Added the `mbo/digest` library: constexpr-safe message digests (SHA-2 family, SHA-3/SHAKE, BLAKE2b, BLAKE3, and legacy SHA-1/MD5), each pinned against reference vectors. +- Mirrored mbo/hash's plug-in architecture in `mbo/digest` (per-algorithm `Algorithm` structs, `Streamer`, `Hmac`, `ToHexString`). +- Added the `digest` binary (`//mbo/digest:digest`): `sha256sum`-compatible output, `-a` selects any of the 17 algorithms, reads stdin, streaming reads. +- Added `--check` to the `digest` binary (coreutils-compatible verification with `--quiet`, `--status`, `--ignore_missing`, and `--strict`). +- Added the bazel `verify_digest_test` rule (`//mbo/digest:digest.bzl`): re-digests files against a saved sidecar or inline checksum so they only change when the digest is updated too. +- Factored the shared load primitives into `//mbo/hash:hash_internal_util` and added the big-endian `Load32BE` / `Load64BE` loads. # 0.12.0 diff --git a/mbo/hash/BUILD.bazel b/mbo/hash/BUILD.bazel index d78cb5d..0b3dcfa 100644 --- a/mbo/hash/BUILD.bazel +++ b/mbo/hash/BUILD.bazel @@ -18,6 +18,7 @@ load("@bazel_skylib//rules:common_settings.bzl", "int_flag", "string_flag") load("@rules_cc//cc:defs.bzl", "cc_binary", "cc_library", "cc_test") load("//mbo/diff:diff.bzl", "diff_test") load("//mbo/digest:digest.bzl", "verify_digest_test") +load(":internal/hash_bzl_verify.bzl", "hash_bzl_verify") load(":internal/hash_mangle_seed.bzl", "mangle_seed_gen") package(default_visibility = ["//visibility:private"]) @@ -41,11 +42,33 @@ int_flag( visibility = ["//visibility:private"], ) +# Build-time (Starlark) ports of a subset of the hash algorithms, kept identical +# to the C++ prime implementation (see hash.bzl; verified by the diff tests below). +bzl_library( + name = "hash_bzl", + srcs = ["hash.bzl"], + visibility = ["//visibility:public"], +) + bzl_library( name = "hash_mangle_seed_bzl", srcs = ["internal/hash_mangle_seed.bzl"], visibility = ["//visibility:private"], - deps = ["@bazel_skylib//rules:common_settings"], + deps = [ + ":hash_bzl", + "@bazel_skylib//rules:common_settings", + ], +) + +bzl_library( + name = "hash_bzl_verify_bzl", + srcs = ["internal/hash_bzl_verify.bzl"], + visibility = ["//visibility:private"], + deps = [ + ":hash_bzl", + "//mbo/diff:diff_bzl", + "@bazel_skylib//rules:write_file", + ], ) mangle_seed_gen( @@ -54,15 +77,6 @@ mangle_seed_gen( visibility = ["//visibility:private"], ) -# Pins the non-Bazel fallback: under default flags the generated header must -# be byte-identical to the checked-in template. Fails when defaults or the -# fold change without updating `internal/hash_mangle_seed.h.in`. -diff_test( - name = "hash_mangle_seed_default_test", - file_new = "hash_mangle_seed_gen.h", - file_old = "internal/hash_mangle_seed.h.in", -) - cc_library( name = "hash_internal_util", hdrs = [ @@ -230,3 +244,20 @@ verify_digest_test( name = "hash_test_vectors_digest_test", digests = {"hash_test_vectors.inc": "hash_test_vectors.inc.sha256"}, ) + +# Minimal hash CLI (` []`, plain GetHash64, no mangle). Doubles as +# the C++ prime reference that the Starlark ports in hash.bzl are diffed against. +cc_binary( + name = "hash_tool", + srcs = ["hash_tool.cc"], + visibility = ["//visibility:public"], + deps = [ + ":hash_cc", + "//mbo/container:limited_map_cc", + "@abseil-cpp//absl/strings:str_format", + ], +) + +# Verifies the bzl hash ports against the C++ prime (hash_tool), one diff_test +# per offered algorithm: hash_bzl_vs_cpp_dumbo_test, hash_bzl_vs_cpp_fnv1a_test. +hash_bzl_verify(name = "hash_bzl_vs_cpp") diff --git a/mbo/hash/README.md b/mbo/hash/README.md index 469e2c4..a43304e 100644 --- a/mbo/hash/README.md +++ b/mbo/hash/README.md @@ -67,27 +67,33 @@ Three entry points, split by contract: ## Algorithm overview -| Algorithm | Widths | Available via | NOTICE | Seeded | Streaming | SMHasher3 | -| ----------- | ------ | --------------------------------- | ----------------------- | ------ | --------- | -------------- | -| `mumbo` | 64 | `hash.h` (default 64/32) | none (in-house) | yes | yes | PASS | -| `jumbo` | 128 | `hash.h` (default 128) | none (in-house) | yes | yes (64) | PASS | -| `murmur3` | 64/128 | `hash.h` | none (public domain) | yes | no | FAIL (123) | -| `siphash` | 64 | `hash.h` | none (CC0) | keyed | yes | PASS (186) | -| `fnv1a` | 64 | `hash.h` | none (public domain) | yes | no | FAIL (7!) | -| `dumbo` | 64 | `hash.h` | none (in-house) | yes | no | PASS | -| `rapidhash` | 64 | `hash_extra.h` + `:hash_extra_cc` | **MIT - ship NOTICE** | yes | no | PASS | -| `xxh64` | 64 | `hash_extra.h` + `:hash_extra_cc` | **BSD-2 - ship NOTICE** | yes | yes | FAIL (181) | -| `xxh3` | 64/128 | `hash_extra.h` + `:hash_extra_cc` | **BSD-2 - ship NOTICE** | yes | no | FAIL (166/162) | - -Notes: `fnv1a` is the algorithm family many `std::hash` implementations use -(e.g. MSVC) - included as the familiar baseline. `siphash` is a keyed PRF: -the DoS-resistant choice when the seed is a secret. `dumbo` is the compact -single-lane member of the MUM family: the fastest hash here for tiny keys and -SMHasher3-clean (188/188, see the design iterations), but single-lane (so it +This is the at-a-glance map; the `SMHasher3` column is a PASS/FAIL summary only. +For the exact score and the failing families see [Quality: SMHasher3](#quality-smhasher3). + +| Algorithm | Widths | Available via | Starlark | NOTICE | Seeded | Streaming | SMHasher3 | +| ----------- | ------ | --------------------------------- | -------- | ----------------------- | ------ | --------- | --------- | +| `mumbo` | 64 | `hash.h` (default 64/32) | no | none (in-house) | yes | yes | PASS | +| `jumbo` | 128 | `hash.h` (default 128) | no | none (in-house) | yes | yes (64) | PASS | +| `murmur3` | 64/128 | `hash.h` | no | none (public domain) | yes | no | FAIL | +| `siphash` | 64 | `hash.h` | no | none (CC0) | keyed | yes | PASS | +| `fnv1a` | 64 | `hash.h` | yes | none (public domain) | yes | no | FAIL | +| `dumbo` | 64 | `hash.h` | yes | none (in-house) | yes | no | PASS | +| `rapidhash` | 64 | `hash_extra.h` + `:hash_extra_cc` | no | **MIT - ship NOTICE** | yes | no | PASS | +| `xxh64` | 64 | `hash_extra.h` + `:hash_extra_cc` | no | **BSD-2 - ship NOTICE** | yes | yes | FAIL | +| `xxh3` | 64/128 | `hash_extra.h` + `:hash_extra_cc` | no | **BSD-2 - ship NOTICE** | yes | no | FAIL | + +Notes: the **Starlark** column marks the hashes also implemented at build time +in [`hash.bzl`](hash.bzl) (`hash.dumbo` and `hash.fnv1a`), kept byte-for-byte +identical to the C++ prime and verified against it (`hash_tool`); mumbo and +jumbo stay C++-only. `fnv1a` is the algorithm family many `std::hash` +implementations use (e.g. MSVC) - included as the familiar baseline. `siphash` +is a keyed PRF: the DoS-resistant choice when the seed is a secret. `dumbo` is +the compact single-lane member of the MUM family: the fastest hash here for tiny +keys and SMHasher3-clean (see the design iterations), but single-lane (so it slows on large keys) - a deliberately minimal companion to `mumbo`, not a -replacement for it. Linking `:hash_extra_cc` requires -shipping the repository-root [NOTICE](../../NOTICE) (see "Third-party -components" in the [repository README](../../README.md)). +replacement for it. Linking `:hash_extra_cc` requires shipping the +repository-root [NOTICE](../../NOTICE) (see "Third-party components" in the +[repository README](../../README.md)). ## Build-seed mangle (`hash_mangle.h` / `:hash_mangle_cc`) @@ -147,6 +153,28 @@ shape the implementation: target split completes the containment: plain hash users sit entirely outside the churn. +### The fold is the in-house `dumbo` hash, ported to Starlark + +The version-and-seed fold uses the in-house `dumbo` hash (SMHasher3-proven - +`fnv1a` is not), implemented in Starlark in `hash.bzl` so the build-time fold +and the shipped library agree on the algorithm. C++ is the prime +implementation; the Starlark port is kept byte-for-byte identical to it, +verified by `//mbo/hash:hash_bzl_vs_cpp_dumbo_test` (the bzl output diffed +against the `hash_tool` C++ binary). `dumbo` is the only in-house hash available +in Starlark - mumbo and jumbo stay C++-only - and the standard `fnv1a` is +offered there as well, likewise verified against C++. + +Call the ports from your own rules via `@helly25_mbo//mbo/hash:hash.bzl` (at load +time; input is a printable-ASCII string or a list of byte values `0..255`, plus +an optional seed; returns the 64-bit hash as an `int`): + +```starlark +load("@helly25_mbo//mbo/hash:hash.bzl", "hash") + +_key = hash.dumbo("my build-time key") # dumbo, seed 0 +_fnv = hash.fnv1a([0x61, 0x62, 0x63]) # fnv1a, seed = FNV offset basis +``` + ## Configuration All configuration lives on the mangle - the deterministic `hash.h` and @@ -174,11 +202,11 @@ build --//mbo/hash:mangle_seed=alice ``` Under default flags the constant still rotates once per release (the library -version is folded in). Non-Bazel builds fall back to the checked-in -`internal/hash_mangle_seed.h.in`, which carries the default-flag constant for -the current version; `//mbo/hash:hash_mangle_seed_default_test` keeps it -byte-identical to the generated header, and a version bump regenerates it -(command in the file's header comment). +version is folded in). The header is generated per build and never committed, +so nothing can drift out of sync and a version bump needs no manual step. A +build that lacks the generated header is a hard error, not a silent fallback; +for clangd (which has no generated header) `hash_mangle.h` uses a stable +fallback constant under `-DIS_CLANGD` purely so the editor can parse it. ## Abseil interop diff --git a/mbo/hash/hash.bzl b/mbo/hash/hash.bzl new file mode 100644 index 0000000..501a0d0 --- /dev/null +++ b/mbo/hash/hash.bzl @@ -0,0 +1,124 @@ +# SPDX-FileCopyrightText: Copyright (c) The helly25 authors (helly25.com) +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Build-time (Starlark) ports of a subset of the `mbo/hash` algorithms. + +The single home for the bzl hash offerings, exposed via the public `hash` +struct. C++ is the prime implementation; each offering here is kept byte-for-byte +identical to its C++ counterpart, verified by `//mbo/hash:hash_bzl_vs_cpp_*_test` +(bzl output diffed against the `hash_tool` C++ binary). The design is +algorithm-count agnostic: offering more (or fewer) bzl hashes only widens or +narrows that comparison. + +Of the in-house mumbo/jumbo and dumbo family only `dumbo` is ported (single +accumulator, no lanes/streaming/128-bit form - see `hash_dumbo.h`); mumbo and +jumbo stay C++-only. The standard `fnv1a` is also provided (it is what the mangle +seed generation historically folded). + +Inputs are either a printable-ASCII string or a list of byte values (0..255); +strings are converted with the printable-ASCII table below (Starlark has no +`ord`). +""" + +_MASK64 = (1 << 64) - 1 + +# Starlark has no `ord`; map printable ASCII (0x20 .. 0x7E) via a table. +_PRINTABLE = " !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~" +_ASCII = {char: index + 32 for index, char in enumerate(_PRINTABLE.elems())} + +def _to_bytes(data): + """Returns `data` as a list of byte values (0..255). + + A list is returned as-is (already byte values); a string is folded through + the printable-ASCII table and fails on any non-printable input, matching the + fold-input contract of the mangle seed generation. + """ + if type(data) != "string": + return data + result = [] + for char in data.elems(): + if char not in _ASCII: + fail("hash input must be printable ASCII, got %r." % char) + result.append(_ASCII[char]) + return result + +def _load_le(data, start, count): + """Assembles `count` bytes of `data` from `start` as a little-endian uint64 (see `Load64`/`LoadTail`).""" + result = 0 + for i in range(count): + result |= data[start + i] << (i * 8) + return result & _MASK64 + +def _mult128(lhs, rhs): + """Full 64x64->128 multiply; returns `(low, high)` (see `hash_internal::Mult128`).""" + product = (lhs & _MASK64) * (rhs & _MASK64) + return (product & _MASK64, (product >> 64) & _MASK64) + +def _mul128_fold64(lhs, rhs): + """64x64->128 multiply folded to 64 bits by XORing the halves (see `hash_internal::Mul128Fold64`).""" + product = (lhs & _MASK64) * (rhs & _MASK64) + return ((product & _MASK64) ^ (product >> 64)) & _MASK64 + +# dumbo constants (see `hash_dumbo.h`): golden-ratio reciprocal and the sqrt(3) +# / sqrt(5) fractional parts (SHA-512 initial values, FIPS 180-4). +_DUMBO_INIT = 0x9E3779B97F4A7C15 +_DUMBO_WORD = 0xBB67AE8584CAA73B +_DUMBO_STATE = 0x3C6EF372FE94F82B + +def _dumbo_mum_step(state, word): + return _mul128_fold64((word ^ _DUMBO_WORD) & _MASK64, (state ^ _DUMBO_STATE) & _MASK64) + +def _dumbo_finalize(state, seed, length): + low, high = _mult128((state ^ _DUMBO_WORD ^ length) & _MASK64, (seed ^ _DUMBO_STATE) & _MASK64) + return _mul128_fold64((low ^ _DUMBO_STATE ^ length) & _MASK64, (high ^ _DUMBO_WORD ^ seed) & _MASK64) + +def _dumbo(data, seed = 0): + """dumbo 64-bit hash of `data`; identical to C++ `mbo::hash::dumbo::GetDumboHash`.""" + data = _to_bytes(data) + length = len(data) + hash_value = (seed ^ _DUMBO_INIT) & _MASK64 + ptr = 0 + + # C++ loops `while (end - ptr) > 8`; Starlark has no `while`, so bound the + # range (each pass advances 8 bytes) and break on the same condition. + for _ in range(length // 8 + 1): + if length - ptr <= 8: + break + hash_value = _dumbo_mum_step(hash_value, _load_le(data, ptr, 8)) + ptr += 8 + + # Final 1..8 bytes (also the whole key when <= 8) as one overlapping load. + hash_value = _dumbo_mum_step(hash_value, _load_le(data, ptr, length - ptr)) + return _dumbo_finalize(hash_value, seed, length) + +# fnv1a constants (see `hash_fnv1a.h`): the offset basis is the canonical default +# seed, so `hash.fnv1a(data)` matches published FNV-1a 64 reference values. +_FNV_OFFSET_BASIS = 0xCBF29CE484222325 +_FNV_PRIME = 0x100000001B3 + +def _fnv1a(data, seed = _FNV_OFFSET_BASIS): + """FNV-1a 64 of `data`; identical to C++ `mbo::hash::fnv1a::GetHash64`.""" + data = _to_bytes(data) + hash_value = seed & _MASK64 + for byte in data: + hash_value = ((hash_value ^ byte) * _FNV_PRIME) & _MASK64 + return hash_value + +# The public bzl hash offerings. Each takes a printable-ASCII string or a byte +# list and an optional seed, and returns the 64-bit hash as an integer. +hash = struct( + dumbo = _dumbo, + fnv1a = _fnv1a, +) diff --git a/mbo/hash/hash_mangle.h b/mbo/hash/hash_mangle.h index aabbaca..0816c42 100644 --- a/mbo/hash/hash_mangle.h +++ b/mbo/hash/hash_mangle.h @@ -47,15 +47,21 @@ #include "mbo/hash/hash.h" +// The build-selected constant lives in a header generated per build by +// `//mbo/hash:hash_mangle_seed_gen` (from the module version and the +// `--//mbo/hash:mangle_seed*` flags); it is never committed. A real build that +// lacks it is an error, not a silent fallback. clangd has no generated header, +// so under `-DIS_CLANGD` (set in the clangd config) it falls back to a stable +// constant purely so this low-level header still parses in the editor. #if __has_include("mbo/hash/hash_mangle_seed_gen.h") # include "mbo/hash/hash_mangle_seed_gen.h" // IWYU pragma: export +#elif defined(IS_CLANGD) +namespace mbo::hash { +// The buckets=1 "one stable constant" value (see internal/hash_mangle_seed.bzl). +inline constexpr uint64_t kMangleConstant = 0x45828334C99AF44FULL; +} // namespace mbo::hash #else -# include "mbo/hash/internal/hash_mangle_seed.h.in" // IWYU pragma: export -# if __STDC_VERSION__ >= 202'311L -# if !defined(IS_CLANGD) -# warning "The correctly generated header is not available. Falling back to template." -# endif // !defined(IS_CLANGD) -# endif // __STDC_VERSION__ >= 202311L +# error "mbo/hash/hash_mangle_seed_gen.h is missing; it is generated by //mbo/hash:hash_mangle_seed_gen." #endif namespace mbo::hash { diff --git a/mbo/hash/hash_tool.cc b/mbo/hash/hash_tool.cc new file mode 100644 index 0000000..15c5e40 --- /dev/null +++ b/mbo/hash/hash_tool.cc @@ -0,0 +1,82 @@ +// SPDX-FileCopyrightText: Copyright (c) The helly25 authors (helly25.com) +// SPDX-License-Identifier: Apache-2.0 +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// A minimal hash CLI: `hash_tool []`. It prints the plain 64-bit +// hash (`GetHash64`, each algorithm's canonical default seed) as 16 uppercase +// hex digits; the build-seed mangle is deliberately NOT applied. With it +// hashes that one argument; without, it reads stdin line by line and prints one +// hash per line. It is the C++ reference (the prime implementation) that the +// Starlark ports in `//mbo/hash:hash.bzl` are diffed against, and a small +// standalone tool. It does nothing else. + +#include +#include +#include +#include +#include +#include +#include + +#include "absl/strings/str_format.h" +#include "mbo/container/limited_map.h" +#include "mbo/hash/hash.h" + +namespace { + +using HashFn = uint64_t (*)(std::string_view); + +// Plain `GetHash64` per algorithm, each with its own canonical default seed +// (dumbo 0, fnv1a offset basis, ...). Captureless lambdas decay to `HashFn`. +constexpr auto kAlgorithms = mbo::container::ToLimitedMap({ + {"dumbo", [](std::string_view data) { return mbo::hash::dumbo::Algorithm::GetHash64(data); }}, + {"fnv1a", [](std::string_view data) { return mbo::hash::fnv1a::Algorithm::GetHash64(data); }}, + {"mumbo", [](std::string_view data) { return mbo::hash::mumbo::Algorithm::GetHash64(data); }}, + {"murmur3", [](std::string_view data) { return mbo::hash::murmur3::Algorithm::GetHash64(data); }}, + {"siphash", [](std::string_view data) { return mbo::hash::siphash::Algorithm::GetHash64(data); }}, +}); + +std::optional Lookup(std::string_view algo) { + const auto it = kAlgorithms.find(algo); + if (it == kAlgorithms.end()) { + return std::nullopt; + } + return it->second; +} + +} // namespace + +int main(int argc, char** argv) { + const std::span args(argv, static_cast(argc)); + if (args.size() < 2 || args.size() > 3) { + std::cerr << absl::StreamFormat( + "Usage: %s [] (data read from stdin, one per line, if omitted)\n", + args.empty() ? "hash_tool" : args[0]); + return 2; + } + const std::optional hash_fn = Lookup(args[1]); + if (!hash_fn.has_value()) { + std::cerr << absl::StreamFormat("Unknown algo '%s'; known: dumbo, fnv1a, mumbo, murmur3, siphash.\n", args[1]); + return 2; + } + if (args.size() == 3) { + std::cout << absl::StreamFormat("%016X\n", (*hash_fn)(args[2])); + return 0; + } + std::string line; + while (std::getline(std::cin, line)) { + std::cout << absl::StreamFormat("%016X\n", (*hash_fn)(line)); + } + return 0; +} diff --git a/mbo/hash/internal/hash_bzl_verify.bzl b/mbo/hash/internal/hash_bzl_verify.bzl new file mode 100644 index 0000000..d057111 --- /dev/null +++ b/mbo/hash/internal/hash_bzl_verify.bzl @@ -0,0 +1,93 @@ +# SPDX-FileCopyrightText: Copyright (c) The helly25 authors (helly25.com) +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Verifies the `//mbo/hash:hash.bzl` ports against the C++ prime implementation. + +C++ is the source of truth: for a fixed set of inputs, the Starlark hash (folded +at load time) must equal what the `hash_tool` C++ binary prints. A `diff_test` +per offered algorithm compares the two. The design is algorithm-count agnostic - +this only covers the algorithms bzl actually offers. +""" + +load("@bazel_skylib//rules:write_file.bzl", "write_file") +load("//mbo/diff:diff.bzl", "diff_test") +load("//mbo/hash:hash.bzl", "hash") + +# Printable ASCII (0x20 .. 0x7E) - the fold-input alphabet. Used to build inputs +# that span a range of byte values as well as a range of lengths. +_PRINTABLE = " !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~" + +# Lengths cross dumbo's 8-byte block boundary and every tail size (1..8), and +# extend past a single/multiple blocks for good measure. +_LENGTHS = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 15, 16, 17, 23, 24, 25, 31, 32, 33, 63, 64, 65, 100, 127, 128, 129, 255, 256, 300] + +# A handful of concrete strings, including mangle fold inputs. +_EXTRA = ["0.13.0|", "0.13.1|", "hello world, this is dumbo!", "|"] + +_FNS = {"dumbo": hash.dumbo, "fnv1a": hash.fnv1a} + +def _pattern(length): + return "".join([_PRINTABLE[i % len(_PRINTABLE)] for i in range(length)]) + +def _hex16(value): + # Starlark `%` has no width/zero-pad; pad manually to match C++ `%016X`. + digits = "%X" % value + return ("0" * (16 - len(digits))) + digits + +def hash_bzl_verify(name, algos = ["dumbo", "fnv1a"], tool = "//mbo/hash:hash_tool"): + """Wires up the bzl-vs-C++ diff tests for each `algo` in `algos`. + + Args: + name: Base name for the generated inputs/vectors/test targets. + algos: Offered algorithms to verify (each must be a key of `_FNS`). + tool: Label of the C++ reference binary (`hash_tool`). + """ + inputs = [_pattern(length) for length in _LENGTHS] + _EXTRA + + # The shared inputs, one per line (raw; fed to `hash_tool` on stdin). + write_file( + name = name + "_inputs", + out = name + "_inputs.txt", + content = inputs, + newline = "unix", + ) + + for algo in algos: + hash_fn = _FNS[algo] + + # The Starlark result: one 16-hex hash per input, folded at load time. + write_file( + name = "%s_%s_bzl" % (name, algo), + out = "%s_%s_bzl.txt" % (name, algo), + content = [_hex16(hash_fn(data)) for data in inputs], + newline = "unix", + ) + + # The C++ prime result over the same inputs. + native.genrule( + name = "%s_%s_cpp" % (name, algo), + srcs = [name + "_inputs.txt"], + outs = ["%s_%s_cpp.txt" % (name, algo)], + cmd = "$(execpath %s) %s < $(execpath %s_inputs.txt) > $@" % (tool, algo, name), + tools = [tool], + ) + + diff_test( + name = "%s_%s_test" % (name, algo), + file_old = "%s_%s_cpp.txt" % (name, algo), + file_new = "%s_%s_bzl.txt" % (name, algo), + # write_file omits the trailing newline that hash_tool prints. + ignore_missing_final_newline = True, + ) diff --git a/mbo/hash/internal/hash_mangle_seed.bzl b/mbo/hash/internal/hash_mangle_seed.bzl index 8d59c91..82146f5 100644 --- a/mbo/hash/internal/hash_mangle_seed.bzl +++ b/mbo/hash/internal/hash_mangle_seed.bzl @@ -16,6 +16,7 @@ """Generator for `//mbo/hash:hash_mangle_seed_gen.h` (see `hash_mangle.h`).""" load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") +load("//mbo/hash:hash.bzl", "hash") # The bucket-0 mangle constant and the full-width odd multiplier that spreads # a bucket index across all 64 bits. The expansion is deliberately independent @@ -25,20 +26,6 @@ _BASE = 0x45828334C99AF44F _SPREAD = 0x9E3779B97F4A7C15 _MASK64 = (1 << 64) - 1 -# Starlark has no `ord`; map printable ASCII via a table (0x20 .. 0x7E). -_PRINTABLE = " !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~" -_ASCII = {char: index + 32 for index, char in enumerate(_PRINTABLE.elems())} - -def _fnv1a_64(text): - """FNV-1a 64 over the ASCII bytes of `text` (any uniform fold works here).""" - hash_value = 0xCBF29CE484222325 - for char in text.elems(): - if char not in _ASCII: - fail("The mangle fold input (module version + --//mbo/hash:mangle_seed) must be printable ASCII, got %r." % - char) - hash_value = ((hash_value ^ _ASCII[char]) * 0x100000001B3) & _MASK64 - return hash_value - def mangle_constant(version, seed, buckets): """Returns the mangle constant selected by (`version`, `seed`, `buckets`). @@ -65,7 +52,11 @@ def mangle_constant(version, seed, buckets): fail("--//mbo/hash:mangle_seed_buckets must be >= 0, got %d." % buckets) if buckets == 0: return 0 - bucket = _fnv1a_64(version + "|" + seed) % buckets + + # Fold through the in-house dumbo hash (SMHasher3-proven; see `hash.bzl` and + # `hash_dumbo.h`). The bzl port is kept identical to the C++ one, so the + # build-time fold and the shipped library agree on the algorithm. + bucket = hash.dumbo(version + "|" + seed) % buckets constant = _BASE ^ ((bucket * _SPREAD) & _MASK64) if constant == 0: # Invariant: enabled (buckets >= 1) implies an observable, nonzero @@ -76,10 +67,9 @@ def mangle_constant(version, seed, buckets): return constant # The generated header. `@MANGLE_CONSTANT@` is the only substitution (plain -# string replace: the C++ braces rule out Starlark `format`). The rendering -# under default flags is checked in as `internal/hash_mangle_seed.h.in` (the -# non-Bazel fallback), kept byte-identical by -# `//mbo/hash:hash_mangle_seed_default_test`. +# string replace: the C++ braces rule out Starlark `format`). It is generated +# per build into `hash_mangle_seed_gen.h` and never committed; `hash_mangle.h` +# errors if it is absent (no committed fallback to drift out of sync). _HEADER = """\ // SPDX-FileCopyrightText: Copyright (c) The helly25 authors (helly25.com) // SPDX-License-Identifier: Apache-2.0 @@ -120,12 +110,9 @@ namespace mbo::hash { // flags become `--@helly25_mbo//mbo/hash:mangle_seed` and // `--@helly25_mbo//mbo/hash:mangle_seed_buckets`. // -// This file doubles as the non-Bazel fallback, so the value below is the -// default-flag result for the current library version; -// `//mbo/hash:hash_mangle_seed_default_test` keeps it byte-identical to the -// generated header. On a version bump regenerate it with: -// bazel build //mbo/hash:hash_mangle_seed_gen -// cp bazel-bin/mbo/hash/hash_mangle_seed_gen.h mbo/hash/internal/hash_mangle_seed.h.in +// This header is generated per build and is not committed, so it always matches +// the version and flags it was built with; `hash_mangle.h` requires it (a +// missing header is a build error, never a silent fallback). inline constexpr uint64_t kMangleConstant = @MANGLE_CONSTANT@; } // namespace mbo::hash diff --git a/mbo/hash/internal/hash_mangle_seed.h.in b/mbo/hash/internal/hash_mangle_seed.h.in deleted file mode 100644 index 8524008..0000000 --- a/mbo/hash/internal/hash_mangle_seed.h.in +++ /dev/null @@ -1,50 +0,0 @@ -// SPDX-FileCopyrightText: Copyright (c) The helly25 authors (helly25.com) -// SPDX-License-Identifier: Apache-2.0 -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -#ifndef MBO_HASH_HASH_MANGLE_SEED_GEN_H_ -#define MBO_HASH_HASH_MANGLE_SEED_GEN_H_ - -// IWYU pragma: private, include "mbo/hash/hash_mangle.h" -// IWYU pragma: friend "mbo/hash/hash_mangle.h" - -#include - -namespace mbo::hash { - -// The build-selected constant XORed into `GetHash` values (see `hash_mangle.h`). -// -// Selected by folding the module's own version (from MODULE.bazel via -// `native.module_version()` - every release rotates the constant by -// construction) with the custom Bazel flags -// `--//mbo/hash:mangle_seed` (any printable ASCII string, folded to a bucket -// inside the generation rule, see `internal/hash_mangle_seed.bzl`) and -// `--//mbo/hash:mangle_seed_buckets` (`0` disables the mangle - constant 0, -// `GetHash == GetHash64`; `1` pins one stable nonzero constant across -// releases; `N` bounds variation to `N` constants so build caches converge). -// When the library is built as a dependency (e.g. as `helly25_mbo`), the -// flags become `--@helly25_mbo//mbo/hash:mangle_seed` and -// `--@helly25_mbo//mbo/hash:mangle_seed_buckets`. -// -// This file doubles as the non-Bazel fallback, so the value below is the -// default-flag result for the current library version; -// `//mbo/hash:hash_mangle_seed_default_test` keeps it byte-identical to the -// generated header. On a version bump regenerate it with: -// bazel build //mbo/hash:hash_mangle_seed_gen -// cp bazel-bin/mbo/hash/hash_mangle_seed_gen.h mbo/hash/internal/hash_mangle_seed.h.in -inline constexpr uint64_t kMangleConstant = 0xF0CE596C32241C31ULL; - -} // namespace mbo::hash - -#endif // MBO_HASH_HASH_MANGLE_SEED_GEN_H_ From ad72f6997ac23d0cdf130c542e7c0d3ea481e780 Mon Sep 17 00:00:00 2001 From: helly25 <6420169+helly25@users.noreply.github.com> Date: Sun, 12 Jul 2026 18:00:27 +0100 Subject: [PATCH 2/8] release: drop dev-only files from the release archive Add .trunk, .gitattributes, .gitignore (and mbo/hash/measurements) to the release_prep.sh EXCLUDES so the released tarball carries only what a consumer needs to build the library. Kept bazelmod (useful to copy for other libs), .clang-format (mope formats generated output with it), the other lint/format configs, and compile_commands-update.sh (a courtesy for consumers). --- .github/workflows/release_prep.sh | 4 ++++ CHANGELOG.md | 1 + 2 files changed, 5 insertions(+) diff --git a/.github/workflows/release_prep.sh b/.github/workflows/release_prep.sh index 4d24570..ebd6630 100755 --- a/.github/workflows/release_prep.sh +++ b/.github/workflows/release_prep.sh @@ -66,9 +66,13 @@ done # Exclude some dev stuff from the archive. EXCLUDES=( ".bcr" + ".gitattributes" ".github" + ".gitignore" ".pre-commit" ".pre-commit-config.yaml" + ".trunk" + "mbo/hash/measurements" "tools" ) { diff --git a/CHANGELOG.md b/CHANGELOG.md index 645f08b..000b3d1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ - Added Starlark ports of the `dumbo` and `fnv1a` hashes in `//mbo/hash:hash.bzl` (the public `hash` struct), byte-identical to C++ and verified against it. - Switched the mangle version/seed fold from FNV-1a to the in-house `dumbo` hash. - Added `//mbo/hash:hash_tool`, a minimal ` []` hash CLI (plain `GetHash64`, no mangle). +- Excluded dev-only files from the release archive (`mbo/hash/measurements`, `.trunk`, and git metadata). # 0.13.0 From 1c246ffb8e4d5b12704037861c025fc9577d3571 Mon Sep 17 00:00:00 2001 From: helly25 <6420169+helly25@users.noreply.github.com> Date: Sun, 12 Jul 2026 18:04:15 +0100 Subject: [PATCH 3/8] ci: remove the informational hash benchmark job It never gated (continue-on-error), ran only on main when mbo/hash changed, did no baseline comparison, and produced noisy shared-runner artifacts nothing consumed. Reliable, comparable numbers now come from the out-of-band mbo/hash/measurements bundles. The hash_benchmark binary stays for those and for manual runs; only the CI job and its done-gate wiring are removed. --- .github/workflows/main.yml | 49 +------------------------------------- 1 file changed, 1 insertion(+), 48 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index f25391a..cb5d5b0 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -59,53 +59,6 @@ jobs: - uses: pre-commit-ci/lite-action@v1.0.2 if: always() - benchmark: - needs: [trunk, pre-commit] - # Informational only (hence continue-on-error): shared runners are noisy, so - # these numbers are for architecture/compiler shape comparisons (x86_64 gcc - # vs arm64 Apple clang) and gross regressions, not precise gating. Results - # appear in the job log and as a JSON artifact per OS. - # - # Runs only on `main`, and only when something under mbo/hash actually - # changed - there is nothing to compare otherwise, and no reason to spend - # runner time or risk noise on unrelated pushes. - if: github.ref == 'refs/heads/main' - continue-on-error: true - strategy: - matrix: - os: [ubuntu-latest, macos-26] - runs-on: ${{matrix.os}} - steps: - - uses: actions/checkout@v6 - - uses: dorny/paths-filter@v3 - id: changes - with: - filters: | - hash: - - 'mbo/hash/**' - - uses: bazelbuild/setup-bazelisk@v3 - if: steps.changes.outputs.hash == 'true' - - name: Run hash benchmark - if: steps.changes.outputs.hash == 'true' - # Fast (README) size set; the tool's precautions (interleaving + warmup) - # and 9 repetitions so the artifact is comparable with google/benchmark's - # compare.py. The full-size dataset is measured out of band (see - # mbo/hash/measurements/). - run: | - bazel run -c opt //mbo/hash:hash_benchmark -- \ - --benchmark_min_time=0.2s \ - --benchmark_repetitions=9 \ - --benchmark_min_warmup_time=0.05s \ - --benchmark_enable_random_interleaving=true \ - --benchmark_report_aggregates_only=true \ - --benchmark_out="${GITHUB_WORKSPACE}/hash_benchmark.json" \ - --benchmark_out_format=json - - uses: actions/upload-artifact@v4 - if: steps.changes.outputs.hash == 'true' - with: - name: hash-benchmark-${{matrix.os}} - path: hash_benchmark.json - test-gcc: needs: [trunk, pre-commit] secrets: inherit @@ -205,7 +158,7 @@ jobs: bazel_config: ${{ matrix.bazel_config }} done: - needs: [trunk, pre-commit, benchmark, test-gcc, test-clang, test-bcr] + needs: [trunk, pre-commit, test-gcc, test-clang, test-bcr] if: always() runs-on: ubuntu-latest steps: From 03a7ca0e7ee9616cdd7304f2e3069e1a8c3b7894 Mon Sep 17 00:00:00 2001 From: helly25 <6420169+helly25@users.noreply.github.com> Date: Sun, 12 Jul 2026 18:06:00 +0100 Subject: [PATCH 4/8] ci: test one Bazel version per supported major (drop 9.0.2) Policy is the last three majors (7, 8, 9). 9.1.1 already covers major 9, so the extra 9.0.x rung (9.0.2) was redundant; dropping it removes 4 matrix jobs. Kept 7.2.1 (the earliest working 7.x floor) and 8.7.0 (latest 8.x). --- .github/workflows/main.yml | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index cb5d5b0..743f591 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -93,14 +93,14 @@ jobs: gcc_version: [13] llvm_version: [20.1.8] bazel_config: [opt] - # Bazel-version compatibility rungs, crossed with the os/compiler combos - # below. 7.2.1 is the earliest 7.x that works: MODULE.bazel uses - # `include()` (added in 7.2.0, so 7.1.x fails with "name 'include' is not - # defined") and the dep `depend_on_what_you_use@0.16.0` declares - # `bazel_compatibility: [>=7.2.1]` (so 7.2.0 is rejected too). Plus the - # latest 8.x (8.7.0), the latest 9.0.x (9.0.2), and the checked-in - # default 9.1.1. - bazel_version: [7.2.1, 8.7.0, 9.0.2, 9.1.1] + # Bazel-version compatibility rungs: one per supported major (we test + # the last three, 7/8/9), crossed with the os/compiler combos below. + # 7.2.1 is the earliest 7.x that works: MODULE.bazel uses `include()` + # (added in 7.2.0, so 7.1.x fails with "name 'include' is not defined") + # and the dep `depend_on_what_you_use@0.16.0` declares + # `bazel_compatibility: [>=7.2.1]` (so 7.2.0 is rejected too). 8.7.0 is + # the latest 8.x; 9.1.1 is the latest 9.x and the checked-in default. + bazel_version: [7.2.1, 8.7.0, 9.1.1] exclude: - os: ubuntu-latest compiler: native From 30e1debfa358f4f51b222ea6b644070e046114c0 Mon Sep 17 00:00:00 2001 From: helly25 <6420169+helly25@users.noreply.github.com> Date: Sun, 12 Jul 2026 18:16:10 +0100 Subject: [PATCH 5/8] ci: shrink the test matrix and make test-clang gating - test-clang now gates (continue-on-error: false) via the done job; the hermetic-clang coverage was informational-only before. - Reduced the clang LLVM ladder to oldest+newest (dropped 21.1.8) and dropped the fastbuild config. - Consolidated asan onto the newest clang on both OSes (dropped the redundant clang-20 asan); gcc-14 asan stays and runs early in test-gcc. - test-bcr runs the 7.x/8.x Bazel rungs on ubuntu only (platform-agnostic build-system compat); macOS keeps the default 9.1.1 for platform coverage. - test-gcc drops gcc-13 (still built at opt later by test-bcr's ubuntu+gcc). Net: 31 -> 21 test jobs, all gating. --- .github/workflows/main.yml | 50 +++++++++++++++++--------------------- 1 file changed, 22 insertions(+), 28 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 743f591..63c113f 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -66,14 +66,11 @@ jobs: matrix: os: [ubuntu-latest] compiler: [gcc] - gcc_version: [13, 14] + # gcc 14 gets the full config set (asan/cpp23/opt) here, early. gcc 13 is + # still built (opt) later by test-bcr's ubuntu+gcc rungs, so it need not + # repeat in this job. + gcc_version: [14] bazel_config: [asan, cpp23, opt] - # Using `include` does not make the options show up in the generated config names... - exclude: - - bazel_config: asan - gcc_version: 13 - - bazel_config: cpp23 - gcc_version: 13 uses: ./.github/workflows/test.yml with: @@ -106,6 +103,13 @@ jobs: compiler: native - os: macos-26 compiler: gcc + # The 7.x/8.x rungs are a build-system-compat check (MODULE.bazel + # loading, rules resolution) - platform-agnostic, so run them on + # ubuntu only. macOS keeps the default 9.1.1 for platform coverage. + - os: macos-26 + bazel_version: 7.2.1 + - os: macos-26 + bazel_version: 8.7.0 uses: ./.github/workflows/test.yml with: @@ -125,33 +129,23 @@ jobs: os: [ubuntu-latest, macos-26] compiler: [clang] # Hermetic toolchains_llvm clang (independent of the runner's Apple clang). - # Ladder: working default (20), macOS native match (21 = Apple clang 21), newest (22). + # Ladder: oldest supported (20.1.8, the default pin) and newest (22.1.8). # TODO(llvm-23): add 23.x here once released and listed in toolchains_llvm. - llvm_version: [20.1.8, 21.1.8, 22.1.8] - bazel_config: [asan, cpp23, fastbuild, opt] + llvm_version: [20.1.8, 22.1.8] + bazel_config: [asan, cpp23, opt] exclude: - # macOS asan works via toolchains_llvm's @loader_path rpath fix for the - # sanitizer runtime dylib (helly25 fork; upstream PR #767), on LLVM - # 22.1.8. 20.1.8 still hangs in compiler-rt FindDynamicShadowStart on - # macOS 26, so exclude only that combo; macOS asan rides the 22.1.8 rung. - - os: macos-26 - llvm_version: 20.1.8 - bazel_config: asan - # 20.1.8 is the default pin -> full config coverage - # 21.1.8 less coverage, just check opt - # 22.1.8 keep the complex configs, just drop fastbuild - - llvm_version: 21.1.8 + # asan runs only on the newest toolchain (both platforms); test-gcc + # already exercises asan early (gcc 14). macOS asan must be 22.1.8 + # regardless - 20.1.8 hangs in compiler-rt FindDynamicShadowStart on + # macOS 26, while 22.1.8's sanitizer dylib uses toolchains_llvm's + # @loader_path rpath fix (helly25 fork; upstream PR #767) - so + # consolidating asan there also drops the redundant clang-20 asan. + - llvm_version: 20.1.8 bazel_config: asan - - llvm_version: 21.1.8 - bazel_config: cpp23 - - llvm_version: 21.1.8 - bazel_config: fastbuild - - llvm_version: 22.1.8 - bazel_config: fastbuild uses: ./.github/workflows/test.yml with: - continue-on-error: true + continue-on-error: false os: ${{ matrix.os }} compiler: ${{ matrix.compiler }} llvm_version: ${{ matrix.llvm_version }} From 61d1e9cd770fefb32374be2d1dd7a7f07df364b7 Mon Sep 17 00:00:00 2001 From: helly25 <6420169+helly25@users.noreply.github.com> Date: Sun, 12 Jul 2026 18:18:07 +0100 Subject: [PATCH 6/8] ci: fix stale LLVM default-pin reference (it is 22.1.8) The clang-ladder comment claimed 20.1.8 was the default pin; the pin is 22.1.8 (bazelmod/llvm.MODULE.bazel). Also align test-bcr's clang combos to that pinned default instead of 20.1.8. --- .github/workflows/main.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 63c113f..5684ebf 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -88,7 +88,8 @@ jobs: os: [ubuntu-latest, macos-26] compiler: [gcc, native, clang] gcc_version: [13] - llvm_version: [20.1.8] + # The pinned default toolchain (see bazelmod/llvm.MODULE.bazel). + llvm_version: [22.1.8] bazel_config: [opt] # Bazel-version compatibility rungs: one per supported major (we test # the last three, 7/8/9), crossed with the os/compiler combos below. @@ -129,7 +130,7 @@ jobs: os: [ubuntu-latest, macos-26] compiler: [clang] # Hermetic toolchains_llvm clang (independent of the runner's Apple clang). - # Ladder: oldest supported (20.1.8, the default pin) and newest (22.1.8). + # Ladder: oldest supported (20.1.8) and the pinned default, also newest (22.1.8). # TODO(llvm-23): add 23.x here once released and listed in toolchains_llvm. llvm_version: [20.1.8, 22.1.8] bazel_config: [asan, cpp23, opt] From 5f118a77c15080148bdec85576ab94cb4bf30c00 Mon Sep 17 00:00:00 2001 From: helly25 <6420169+helly25@users.noreply.github.com> Date: Sun, 12 Jul 2026 18:23:36 +0100 Subject: [PATCH 7/8] bazelmod: correct LLVM distribution comments 22.1.8 is the default pin (not just a forward-looking rung); 21.1.8 matches macOS 26's native Apple clang 21 (a pairing that tracks Xcode). Also replaces the em-dashes with hyphens. --- bazelmod/llvm.MODULE.bazel | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/bazelmod/llvm.MODULE.bazel b/bazelmod/llvm.MODULE.bazel index 0d51fc0..cddbf92 100644 --- a/bazelmod/llvm.MODULE.bazel +++ b/bazelmod/llvm.MODULE.bazel @@ -22,20 +22,18 @@ llvm.toolchain( name = "llvm_toolchain_llvm", llvm_version = "22.1.8", extra_llvm_distributions = { - # 22.1.8 — newest released; a forward-looking rung in the CI clang matrix + # 22.1.8 - the default pin (newest released). "LLVM-22.1.8-Linux-ARM64.tar.xz": "805efad2bb91cb4967fa569e0881d10c0f69c04461cf671cccbae19f547acc34", "LLVM-22.1.8-Linux-X64.tar.xz": "df0e1ecf16caf3489a272a5eea4eec9b0d82878f6477fa309504f918a0006384", "LLVM-22.1.8-macOS-ARM64.tar.xz": "f260f4f7c0d430828a81ae8a3826a1d63fc0963ec2459489308cc23b1f7eab4f", "clang+llvm-22.1.8-aarch64-pc-windows-msvc.tar.xz": "de718c58ebbc5f61d58c17b90457fcf42983bc2c4a4aba3e010d108713bfd7f1", - # 21.1.8 — best native match for macOS (Apple clang 21). + # 21.1.8 - matches macOS 26's native Apple clang 21 (tracks Xcode; shifts with the runner image). "LLVM-21.1.8-Linux-ARM64.tar.xz": "65ce0b329514e5643407db2d02a5bd34bf33d159055dafa82825c8385bd01993", "LLVM-21.1.8-Linux-X64.tar.xz": "b3b7f2801d15d50736acea3c73982994d025b01c2f035b91ae3b49d1b575732b", "LLVM-21.1.8-macOS-ARM64.tar.xz": "b95bdd32a33a81ee4d40363aaeb26728a26783fcef26a4d80f65457433ea4669", "clang+llvm-21.1.8-aarch64-pc-windows-msvc.tar.xz": "f214b1226d8de005b5f691dd29d9dfea2b49e22d0de445429916173dbb626f7f", "clang+llvm-21.1.8-armv7a-linux-gnueabihf.tar.gz": "4c25b04275d7b34f47f6a7f8f05ef5518ab391c31c084e8f19e6a89a24f8fa57", - # 20.1.8 - minimum supported LLVM and the default pin: we develop against - # this baseline so we don't lean on newer-LLVM behaviour (CI also exercises - # 21.1.8 and 22.1.8). Bumping it would drop support for older LLVM. + # 20.1.8 - minimum supported LLVM. "LLVM-20.1.8-Linux-ARM64.tar.xz": "b855cc17d935fdd83da82206b7a7cfc680095efd1e9e8182c4a05e761958bef8", "LLVM-20.1.8-Linux-X64.tar.xz": "1ead36b3dfcb774b57be530df42bec70ab2d239fbce9889447c7a29a4ddc1ae6", "LLVM-20.1.8-macOS-ARM64.tar.xz": "a9a22f450d35f1f73cd61ab6a17c6f27d8f6051d56197395c1eb397f0c9bbec4", From 1741d04518bb946db9a16033b4573b21eaad1f35 Mon Sep 17 00:00:00 2001 From: helly25 <6420169+helly25@users.noreply.github.com> Date: Sun, 12 Jul 2026 19:01:04 +0100 Subject: [PATCH 8/8] ci: exercise clang cpp23 on the newest LLVM only C++23 is used on recent compilers, so testing the cpp23 mode on the oldest clang (20.1.8) is low value; the oldest rung now just proves the minimum supported LLVM still builds (opt). Matches the asan 'newest-only' rule. test-clang 10 -> 8. --- .github/workflows/main.yml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 5684ebf..65ff307 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -143,6 +143,11 @@ jobs: # consolidating asan there also drops the redundant clang-20 asan. - llvm_version: 20.1.8 bazel_config: asan + # C++23 is used on recent compilers, so exercise cpp23 on the newest + # clang only; the oldest rung just proves the minimum supported LLVM + # still builds (opt). + - llvm_version: 20.1.8 + bazel_config: cpp23 uses: ./.github/workflows/test.yml with: