feat(hash): port mumbo to Starlark (hash.mumbo)

CHANGELOG.md

@@ -4,6 +4,7 @@
 - Fixed SMHasher3 result parsing so an invalid hash name or crash reads as `ERROR` instead of a false `PASS`, and the pass/fail score and failing families now parse from the run Summary; legacy/short SMHasher3 names are aliased to their registered form when a measured dataset is loaded.
 - Integrated `mbo/hash/measurements` as a normal dev package (dropped its nested module and `.bazelignore` entry; still stripped from release archives) and added a `quality_sh_test` bazel test gating the generated tables; the test CI job now fetches the measurement bundles (Git LFS) so it reads real data.
 - Added a `no-deps-on-measurements` pre-commit guard so nothing outside the release-stripped `mbo/hash/measurements` may depend on it (which would dangle in releases).
+- Ported the default `mumbo` 64-bit hash to Starlark (`//mbo/hash:hash.bzl` `hash.mumbo`), byte-identical to C++ and verified against it; `hash.bzl` now offers `mumbo`, `dumbo`, and `fnv1a`.
 
 # 0.13.1
 

mbo/hash/README.md

@@ -76,7 +76,7 @@ For the exact score and the failing families see [Quality: SMHasher3](#quality-s
 
 | Algorithm   | Bits | Available via                     | Starlark | NOTICE                  | Seeded | Streaming | SMHasher3 |
 | ----------- | ---: | --------------------------------- | -------- | ----------------------- | ------ | --------- | --------- |
-| `mumbo`     |   64 | `hash.h` (default 64/32)          | no       | none (in-house)         | yes    | yes       | PASS      |
+| `mumbo`     |   64 | `hash.h` (default 64/32)          | yes      | none (in-house)         | yes    | yes       | PASS      |
 | `jumbo`     |  128 | `hash.h` (default 128)            | no       | none (in-house)         | yes    | yes (64)  | PASS      |
 | `murmur3`   |  128 | `hash.h`                          | no       | none (public domain)    | yes    | no        | FAIL      |
 | `siphash`   |   64 | `hash.h`                          | no       | none (CC0)              | keyed  | yes       | PASS      |
@@ -90,9 +90,10 @@ For the exact score and the failing families see [Quality: SMHasher3](#quality-s
 <!-- END algorithm overview -->
 
 Notes: the **Starlark** column marks the hashes also implemented at build time
-in [`hash.bzl`](hash.bzl) (`hash.dumbo` and `hash.fnv1a`), kept byte-for-byte
-identical to the C++ prime and verified against it (`hash_tool`); mumbo and
-jumbo stay C++-only. `fnv1a` is the algorithm family many `std::hash`
+in [`hash.bzl`](hash.bzl) (`hash.mumbo`, `hash.dumbo`, `hash.fnv1a`), kept
+byte-for-byte identical to the C++ prime and verified against it (`hash_tool`);
+only the one-shot 64-bit form is ported, so the native 128-bit `jumbo` and
+streaming stay C++-only. `fnv1a` is the algorithm family many `std::hash`
 implementations use (e.g. MSVC) - included as the familiar baseline. `siphash`
 is a keyed PRF: the DoS-resistant choice when the seed is a secret. `dumbo` is
 the compact single-lane member of the MUM family: the fastest hash here for tiny
@@ -167,19 +168,28 @@ The version-and-seed fold uses the in-house `dumbo` hash (SMHasher3-proven -
 and the shipped library agree on the algorithm. C++ is the prime
 implementation; the Starlark port is kept byte-for-byte identical to it,
 verified by `//mbo/hash:hash_bzl_vs_cpp_dumbo_test` (the bzl output diffed
-against the `hash_tool` C++ binary). `dumbo` is the only in-house hash available
-in Starlark - mumbo and jumbo stay C++-only - and the standard `fnv1a` is
-offered there as well, likewise verified against C++.
+against the `hash_tool` C++ binary). `mumbo` (the default 64-bit hash) and
+`dumbo` are both offered in Starlark alongside the standard `fnv1a`, each
+likewise verified against C++; only the one-shot 64-bit form is ported (the
+native 128-bit `jumbo` and streaming stay C++-only).
 
 Call the ports from your own rules via `@helly25_mbo//mbo/hash:hash.bzl` (at load
-time; input is a printable-ASCII string or a list of byte values `0..255`, plus
-an optional seed; returns the 64-bit hash as an `int`):
+time; input is a printable-ASCII string or a list of byte values `0..255`). The
+seed is an **optional** second argument: omit it and each hash uses its own
+canonical default - the same one the matching C++ `GetHash64` uses, so the
+values agree by default. `mumbo` defaults to the library-wide `kDefaultSeed`
+(`5381`), `dumbo` to `0`, and `fnv1a` to the FNV-1a **offset basis**
+(`0xCBF29CE484222325`, the standard FNV-1a starting constant). Pass a seed
+explicitly to match a specific seeded C++ call. Each returns the 64-bit hash as
+an `int`:
 
 ```starlark
 load("@helly25_mbo//mbo/hash:hash.bzl", "hash")
 
-_key = hash.dumbo("my build-time key")  # dumbo, seed 0
-_fnv = hash.fnv1a([0x61, 0x62, 0x63])   # fnv1a, seed = FNV offset basis
+_a = hash.mumbo("my build-time key")        # default 64-bit hash; default seed kDefaultSeed (5381)
+_b = hash.dumbo("my build-time key")        # dumbo; default seed 0
+_c = hash.fnv1a([0x61, 0x62, 0x63])         # fnv1a over bytes; default seed = FNV offset basis
+_d = hash.mumbo("my build-time key", 1234)  # explicit seed == mbo::hash::mumbo::GetHash64(key, 1234)
 ```
 
 ## Configuration

mbo/hash/hash.bzl

@@ -22,10 +22,11 @@ identical to its C++ counterpart, verified by `//mbo/hash:hash_bzl_vs_cpp_*_test
 algorithm-count agnostic: offering more (or fewer) bzl hashes only widens or
 narrows that comparison.
 
-Of the in-house mumbo/jumbo and dumbo family only `dumbo` is ported (single
-accumulator, no lanes/streaming/128-bit form - see `hash_dumbo.h`); mumbo and
-jumbo stay C++-only. The standard `fnv1a` is also provided (it is what the mangle
-seed generation historically folded).
+Of the in-house mumbo/jumbo and dumbo family, `mumbo` (the default 64-bit hash)
+and `dumbo` (the compact companion) are ported; only their one-shot 64-bit
+`GetHash64` is offered here - the native 128-bit `jumbo` form and streaming stay
+C++-only. The standard `fnv1a` is also provided (it is what the mangle seed
+generation historically folded).
 
 Inputs are either a printable-ASCII string or a list of byte values (0..255);
 strings are converted with the printable-ASCII table below (Starlark has no
@@ -103,6 +104,108 @@ def _dumbo(data, seed = 0):
     hash_value = _dumbo_mum_step(hash_value, _load_le(data, ptr, length - ptr))
     return _dumbo_finalize(hash_value, seed, length)
 
+# mumbo (see `hash_mumbo.h`): the library's default 64-bit hash. Its secret bank
+# is the 64-bit fractional parts of the square roots of the first sixteen primes
+# (the SHA-512 / SHA-384 initial hash values, FIPS 180-4). Unlike dumbo (default
+# seed 0) and fnv1a (offset basis), mumbo's canonical default seed is the
+# library-wide `kDefaultSeed`. Only the one-shot 64-bit `GetHash64` is ported;
+# the native 128-bit `jumbo` form and streaming stay C++-only.
+_KDEFAULT_SEED = 5381  # `mbo::hash::kDefaultSeed` (hash_types.h).
+_MUMBO_SECRET = [
+    0x6A09E667F3BCC908,
+    0xBB67AE8584CAA73B,
+    0x3C6EF372FE94F82B,
+    0xA54FF53A5F1D36F1,
+    0x510E527FADE682D1,
+    0x9B05688C2B3E6C1F,
+    0x1F83D9ABFB41BD6B,
+    0x5BE0CD19137E2179,
+    0xCBBB9D5DC1059ED8,
+    0x629A292A367CD507,
+    0x9159015A3070DD17,
+    0x152FECD8F70E5939,
+    0x67332667FFC00B31,
+    0x8EB44A8768581511,
+    0xDB0C2E0D64F98FA7,
+    0x47B5481DBEFA4FA4,
+]
+_MUMBO_BULK_WINDOW = 128
+
+def _mumbo_load_small(data, ptr, length):
+    """Loads a 0..16 byte key into two words `(a, b)` (see `mumbo_internal::LoadSmall`)."""
+    if length >= 4:
+        if length >= 9:  # 9..16: two 64-bit loads overlapping the end.
+            return _load_le(data, ptr, 8), _load_le(data, ptr + length - 8, 8)
+        if length == 8:
+            value = _load_le(data, ptr, 8)
+            return value, value
+        return _load_le(data, ptr, 4), _load_le(data, ptr + length - 4, 4)  # 4..7 (len 4 -> equal)
+    if length == 3:
+        value = ((data[ptr] << 45) | (data[ptr + 1] << 8) | data[ptr + 2]) & _MASK64
+        return value, value
+    if length == 2:
+        value = ((data[ptr] << 45) | (data[ptr + 1] << 8) | data[ptr]) & _MASK64
+        return value, value
+    if length == 1:
+        value = ((data[ptr] << 45) | data[ptr]) & _MASK64
+        return value, value
+    return 0, 0
+
+def _mumbo_finish(val_a, val_b, seed, length):
+    """The shared two-multiply finalizer (see `mumbo_internal::Finish`)."""
+    low, high = _mult128((val_a ^ _MUMBO_SECRET[2] ^ length) & _MASK64, (val_b ^ seed) & _MASK64)
+    return _mul128_fold64((low ^ _MUMBO_SECRET[3] ^ length) & _MASK64, (high ^ _MUMBO_SECRET[1]) & _MASK64)
+
+def _mumbo(data, seed = _KDEFAULT_SEED):
+    """mumbo 64-bit hash of `data`; identical to C++ `mbo::hash::mumbo::GetHash64`."""
+    data = _to_bytes(data)
+    length = len(data)
+
+    # Absorb + finalize the seed (structured seeds must not correlate with input).
+    seed = _mul128_fold64((seed ^ _MUMBO_SECRET[0]) & _MASK64, _MUMBO_SECRET[1])
+
+    if length <= 16:
+        val_a, val_b = _mumbo_load_small(data, 0, length)
+        return _mumbo_finish(val_a, val_b, seed, length)
+
+    ptr = 0
+    remaining = length
+
+    # >= 128 bytes: eight independent MUM chains over a 128-byte fetch window.
+    if length >= _MUMBO_BULK_WINDOW:
+        chain = [(seed ^ _MUMBO_SECRET[8 + i]) & _MASK64 for i in range(8)]
+        for _ in range(length // _MUMBO_BULK_WINDOW):
+            if remaining < _MUMBO_BULK_WINDOW:
+                break
+            for i in range(8):
+                chain[i] = _mul128_fold64(
+                    (_load_le(data, ptr + 16 * i, 8) ^ _MUMBO_SECRET[4 + i]) & _MASK64,
+                    (_load_le(data, ptr + 16 * i + 8, 8) ^ chain[i]) & _MASK64,
+                )
+            ptr += _MUMBO_BULK_WINDOW
+            remaining -= _MUMBO_BULK_WINDOW
+        merged = 0
+        for value in chain:
+            merged ^= value
+        seed = merged & _MASK64
+
+    # 17..127 bytes (and any bulk remainder): one MUM chain, 16 bytes per step.
+    # C++ loops `while remaining > 16`; Starlark has no `while`, so bound + break.
+    for _ in range(length // 16 + 1):
+        if remaining <= 16:
+            break
+        seed = _mul128_fold64(
+            (_load_le(data, ptr, 8) ^ _MUMBO_SECRET[1]) & _MASK64,
+            (_load_le(data, ptr + 8, 8) ^ seed) & _MASK64,
+        )
+        ptr += 16
+        remaining -= 16
+
+    # Final 1..16 bytes as two loads overlapping the end (len > 16, so in-bounds).
+    val_a = _load_le(data, ptr + remaining - 16, 8)
+    val_b = _load_le(data, ptr + remaining - 8, 8)
+    return _mumbo_finish(val_a, val_b, seed, length)
+
 # fnv1a constants (see `hash_fnv1a.h`): the offset basis is the canonical default
 # seed, so `hash.fnv1a(data)` matches published FNV-1a 64 reference values.
 _FNV_OFFSET_BASIS = 0xCBF29CE484222325
@@ -121,4 +224,5 @@ def _fnv1a(data, seed = _FNV_OFFSET_BASIS):
 hash = struct(
     dumbo = _dumbo,
     fnv1a = _fnv1a,
+    mumbo = _mumbo,
 )

mbo/hash/internal/hash_bzl_verify.bzl

@@ -36,7 +36,7 @@ _LENGTHS = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 15, 16, 17, 23, 24, 25, 31, 32, 33, 63
 # A handful of concrete strings, including mangle fold inputs.
 _EXTRA = ["0.13.0|", "0.13.1|", "hello world, this is dumbo!", "|"]
 
-_FNS = {"dumbo": hash.dumbo, "fnv1a": hash.fnv1a}
+_FNS = {"dumbo": hash.dumbo, "fnv1a": hash.fnv1a, "mumbo": hash.mumbo}
 
 def _pattern(length):
     return "".join([_PRINTABLE[i % len(_PRINTABLE)] for i in range(length)])
@@ -46,7 +46,7 @@ def _hex16(value):
     digits = "%X" % value
     return ("0" * (16 - len(digits))) + digits
 
-def hash_bzl_verify(name, algos = ["dumbo", "fnv1a"], tool = "//mbo/hash:hash_tool"):
+def hash_bzl_verify(name, algos = ["dumbo", "fnv1a", "mumbo"], tool = "//mbo/hash:hash_tool"):
     """Wires up the bzl-vs-C++ diff tests for each `algo` in `algos`.
 
     Args:

mbo/hash/measurements/hash_algorithms.json

@@ -41,7 +41,7 @@
       "notice": "none (in-house)",
       "seeded": "yes",
       "streaming": "yes",
-      "starlark": "no"
+      "starlark": "yes"
     },
     {
       "algo": "rapidhash",