Add SelectF128 sortition implementation

.github/workflows/oracle.yml

@@ -0,0 +1,28 @@
+name: Regenerate numerical oracle
+
+on:
+  schedule:
+    # The full supply-scale Arb corpus takes roughly two minutes. Running it
+    # nightly catches certificate/generator drift without burdening every PR.
+    - cron: "17 5 * * *"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  arb-certificates:
+    name: Verify Arb certificates
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@v7
+      - uses: actions/setup-python@v6
+        with:
+          python-version: "3.11"
+          cache: pip
+          cache-dependency-path: tools/requirements-oracle.txt
+      - name: Install python-flint
+        run: python -m pip install -r tools/requirements-oracle.txt
+      - name: Regenerate and compare certified quantiles
+        run: python tools/generate_arb_oracle.py --check testdata/f128_arb_certificates.json

.github/workflows/test.yml

@@ -6,17 +6,120 @@ on:
       - main
   pull_request:
 
+concurrency:
+  group: test-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
+
 jobs:
   build:
+    name: Tests (${{ matrix.os }})
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        # Both architectures on each OS. Exact-output tests pin both paths, so
+        # a floating-point divergence in the cgo/Boost CDF between these
+        # targets shows up as a failure rather than a silent consensus split.
+        os:
+          - ubuntu-latest    # linux/amd64
+          - ubuntu-24.04-arm # linux/arm64
+          - macos-latest     # darwin/arm64
+          - macos-15-intel   # darwin/amd64
+          - windows-latest   # windows/amd64
+    steps:
+    - uses: actions/checkout@v7
+
+    - name: Set up Go
+      uses: actions/setup-go@v7
+      with:
+        go-version-file: go.mod
+
+    - name: Build
+      run: go build -v ./...
+
+    - name: Test
+      run: go test -v ./...
+
+    # Ordinary `go test` runs the rapid property tests at their default 100
+    # cases; run them again at a depth that reliably samples every generator
+    # band (the divU shallow-quotient bug reproduced within ~30 banded cases,
+    # so 20000 leaves ample margin while staying under a few seconds).
+    - name: Rapid property tests
+      shell: bash
+      run: go test -v -run TestRapid ./... -args -rapid.checks=20000
+
+  # Cross-checks the "bit-identical on every platform" claim on a 32-bit-int
+  # target, where Go's int is 32 bits -- the reason f128.exp is explicitly
+  # int64. x86-64 runs i386 user code natively, so this needs no emulation;
+  # only a 32-bit C/C++ toolchain for the cgo/Boost build, which the Go
+  # toolchain invokes with -m32 automatically under GOARCH=386. armv7 (the
+  # Raspberry Pi target) has the same 32-bit int but would need slow QEMU, so
+  # 386 buys the same integer width for free.
+  test-386:
+    name: Tests (linux/386, native 32-bit)
     runs-on: ubuntu-latest
+    env:
+      GOARCH: "386"
+      CGO_ENABLED: "1"
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v7
 
     - name: Set up Go
-      uses: actions/setup-go@v3
+      uses: actions/setup-go@v7
+      with:
+        go-version-file: go.mod
+
+    - name: Install 32-bit C/C++ toolchain
+      run: sudo apt-get update && sudo apt-get install -y gcc-multilib g++-multilib
 
     - name: Build
       run: go build -v ./...
 
     - name: Test
       run: go test -v ./...
+
+    - name: Rapid property tests
+      shell: bash
+      run: go test -v -run TestRapid ./... -args -rapid.checks=20000
+
+  # Same 32-bit-int check as the 386 leg, but on the real Raspberry Pi target
+  # (armv7, GOARM=7). ARM has no i386-style native path on the amd64 runner, so
+  # it runs under QEMU in a foreign-arch container; the stock golang image
+  # carries gcc/g++ and the repo vendors Boost, so nothing extra is installed.
+  # Emulated, hence the slowest leg -- it runs the default rapid depth only,
+  # leaving the 20000-case sweep to the native legs.
+  test-armv7:
+    name: Tests (linux/arm/v7, emulated)
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v7
+
+    - name: Set up QEMU
+      uses: docker/setup-qemu-action@v3
+
+    - name: Test under QEMU (armv7 -- Raspberry Pi, 32-bit int)
+      run: |
+        docker run --rm --platform linux/arm/v7 \
+          -v "$PWD":/src -w /src -e CGO_ENABLED=1 \
+          golang:1.23 \
+          go test -v ./...
+
+  # Applies each curated single-site mutant of the f128 arithmetic and
+  # verifies the fast suite kills it (survivors marked equivalent carry
+  # written proofs and are expected). This keeps the kill-rate claim true
+  # over time: a weakened test lets a mutant survive, and a semantic change
+  # to the arithmetic makes an equivalent mutant die or a target string
+  # disappear -- both fail the job, deliberately coupling the mutant table to
+  # the consensus-critical lines it guards. Platform-independent, so one leg.
+  mutation:
+    name: Mutation campaign
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v7
+
+    - name: Set up Go
+      uses: actions/setup-go@v7
+      with:
+        go-version-file: go.mod
+
+    - name: Run mutation campaign
+      run: go run mutation_check.go

.gitignore

@@ -0,0 +1 @@
+.venv-oracle/

README.md

@@ -5,8 +5,11 @@ This package provides an implementation of cryptographic sortition used by [go-a
 
 Please visit the [go-algorand README](https://github.com/algorand/go-algorand/blob/master/README.md) for more information on building and using this software.
 
+See [TESTING.md](TESTING.md) for the testing strategy, oracle layers, and
+developer runbook for the deterministic sortition implementation.
+
 ## License
 
 Please see the [COPYING_FAQ](COPYING_FAQ) for details about how to apply our license.
 
-Copyright (C) 2019-2023, Algorand Inc.
+Copyright (C) 2019-2026, Algorand Foundation Ltd.

TESTING.md

@@ -0,0 +1,95 @@
+# Testing sortition
+
+The test suite combines bit-identical differential testing, independent exact
+mathematics, metamorphic properties, certified offline vectors, and mutation
+testing. No single oracle is treated as sufficient for the consensus-critical
+`SelectF128` path.
+
+## Common commands
+
+```sh
+# Complete ordinary suite
+go test -count=1 ./...
+
+# Property tests at the depth used by CI
+go test -run TestRapid -rapid.checks=20000 -count=1 ./...
+
+# Curated production and test-oracle mutation campaign
+go run mutation_check.go
+
+# Longer fuzzing runs
+go test -run xxx -fuzz FuzzF128Ops -fuzztime 10m
+go test -run xxx -fuzz FuzzSelectF128 -fuzztime 10m
+```
+
+## Assurance layers
+
+| Approach | Purpose | Location |
+|---|---|---|
+| Ordinary and compatibility tests | Basic sortition behavior, benchmarks, agreement with the deployed C++/Boost path, defined frozen-tail behavior, and current-scale regression cases | `sortition_test.go`, `f128_test.go` |
+| Bit-identical 128-bit oracle | Mirrors the production CDF walk with 128-bit `math/big.Float`; detects implementation defects in the hand-written integer arithmetic | `selectBigOracle` and related tests in `f128_test.go` |
+| Primitive differential fuzzing | Checks f128 arithmetic against `big.Float`, with constructed tie/carry seeds and magnitude-banded rapid generators | `FuzzF128Ops` in `f128_test.go`, `TestRapidF128Ops` in `f128_rapid_test.go` |
+| Exact-integer RNE oracle | Computes normalization and round-to-nearest-even solely with `big.Int` quotient/remainder arithmetic, avoiding shared reliance on `big.Float` | `f128_integer_oracle_test.go` |
+| Division certificates | Proves sampled `divStep` digits satisfy `U = q*V + R` and `0 <= R < V`, including quotient-estimate corner paths | `TestDivStepExactCertificate` in `f128_integer_oracle_test.go` |
+| Exact binomial formula | Builds the CDF from binomial coefficients and integer cross-products rather than the production PMF recurrence | `f128_exact_test.go`, `cdf_exhaustive_test.go` |
+| Exhaustive boundary grids | Enumerates small parameter domains and checks digest integers below, at, and above every exact CDF boundary, with anti-vacuity counters | `cdf_exhaustive_test.go`, `oracle_hardening_test.go` |
+| Higher-precision convergence | Requires the 512- and 1024-bit CDF walks to converge and compares them with exact small-domain results and certified large-money results | `selectAtPrecision` in `f128_exact_test.go`, tests in `oracle_hardening_test.go` |
+| Internal trajectory and liveness | Checks PMF/CDF error envelopes and monotonicity, audits the maximum-domain exponent path, proves bounded freeze permanence, and pins CDF-evaluation limits | `f128_trajectory_test.go` |
+| Metamorphic properties | Checks digest monotonicity, power-of-two and arbitrary common-factor probability scaling, primitive identities, and arithmetic order properties without a numeric oracle | `f128_rapid_test.go` |
+| Distribution sanity | Checks aggregate selection weight against the expected binomial mean without reusing the CDF formula | `TestSelectF128Distribution` in `f128_exact_test.go` |
+| Arb-certified quantiles | Uses Arb's regularized incomplete beta implementation to certify large-money quantile inequalities with rigorous dyadic endpoints | `tools/generate_arb_oracle.py`, `testdata/f128_arb_certificates.json`, `f128_arb_certificate_test.go` |
+| Mutation testing | Applies curated bugs to production code and in-process test oracles; every non-equivalent mutant must be killed | `mutation_check.go` |
+
+## Arb corpus
+
+The Arb generator is an offline tool and is not a dependency of ordinary
+`go test`. Set it up in an ignored virtual environment:
+
+```sh
+python3 -m venv .venv-oracle
+.venv-oracle/bin/pip install -r tools/requirements-oracle.txt
+```
+
+Regenerate the corpus or verify it without rewriting the checked-in file:
+
+```sh
+.venv-oracle/bin/python tools/generate_arb_oracle.py \
+  > testdata/f128_arb_certificates.json
+
+.venv-oracle/bin/python tools/generate_arb_oracle.py \
+  --check testdata/f128_arb_certificates.json
+```
+
+The Go certificate test exactly compares the stored dyadic endpoints with the
+digest ratio. Trust in the external tool is limited to Arb's assertion that
+those endpoints enclose the true incomplete-beta CDF.
+
+## Important test semantics
+
+- The near-one frozen-tail sliver is defined to return `money`. Exact-math
+  tolerance tests exclude that interval deliberately; dedicated tests pin its
+  behavior and liveness at current-scale values.
+- `money` must remain below `SelectF128MaxMoney`. The maximum-domain exponent
+  audit checks the worst representable `1-p` combination without performing a
+  supply-sized walk.
+- Large-money in-process oracles have a committee-scale step budget. A broken
+  reference must fail loudly instead of hanging while attempting trillions of
+  iterations.
+- Randomized tests use deterministic seeds. Measure-zero events such as exact
+  ties and inclusive boundaries are constructed explicitly rather than left
+  to sampling.
+- Tests with skip or clamp paths need anti-vacuity counts or mandatory spot
+  cases proving that meaningful assertions ran.
+
+## Continuous integration
+
+- `.github/workflows/test.yml` runs builds, ordinary tests, and 20,000-check
+  rapid properties across Linux amd64/arm64, macOS amd64/arm64, and Windows
+  amd64. It also runs the mutation campaign on Linux.
+- `.github/workflows/oracle.yml` regenerates and compares the complete Arb
+  corpus nightly and on manual dispatch.
+
+When arithmetic or recurrence semantics change, update the relevant
+independent oracle and exact/certified vectors deliberately, then rerun the
+full suite and mutation campaign. A changed knife-edge pin is evidence that
+the numerical trajectory changed and should be explained in the commit.