Every quantitative claim that appears in the README or the paper maps to a committed data artifact and the script that produced it, with n, uncertainty, and a verification status. Statuses:
- provisional — inherited from the APA predecessor, not yet re-derived in this repo. Not paper-eligible.
- screening — re-derived here at n=250; valid for the collapse-vs-recover contrast, not for fine per-model orderings.
- confirmed — re-derived here at n≥1000 with CIs; paper-eligible.
- retracted — superseded; kept with the reason.
PPL is not an admissible success metric anywhere in this ledger (spec §8).
| # | Claim | Value (± 95% CI) | n | Artifact | Script | Status |
|---|---|---|---|---|---|---|
| C1 | Naive per-token INT4 collapses at scale (Qwen2-7B) | 0.212 [0.166, 0.267] | 250 | analysis/c20_q7_summary.json |
analysis/c20_reference_repro.py |
screening (reproduces c17 Δ=0.000) |
| C2 | Naive per-token INT4 collapses at scale (Qwen2-1.5B) | 0.248 [0.199, 0.305] | 250 | analysis/c20_q15_summary.json |
analysis/c20_reference_repro.py |
screening (reproduces c17 Δ=0.000) |
| C3 | Per-channel INT4 recovers ~FP16 (7B: 0.604 vs FP16 0.612) | 0.604 [0.542, 0.663] | 250 | analysis/c20_q7_summary.json |
analysis/c20_reference_repro.py |
screening (reproduces c17 Δ=0.000) |
| C4 | KIVI (per-ch K / per-tok V) recovers ~FP16 (7B) | 0.600 [0.538, 0.659] | 250 | analysis/c20_q7_summary.json |
analysis/c20_reference_repro.py |
screening (reproduces c17 Δ=0.000) |
| C5 | Outlier (top-2 FP16 + per-ch, both K&V) best, ~FP16 (7B) | 0.616 [0.554, 0.674] | 250 | analysis/c20_q7_summary.json |
analysis/c20_reference_repro.py |
screening (reproduces c17 Δ=0.000) |
| C6 | INT8 per-token is near-lossless 2× floor (7B) | 0.600 [0.538, 0.659] | 250 | analysis/c20_q7_summary.json |
analysis/c20_reference_repro.py |
screening (reproduces c17 Δ=0.000) |
| C7 | MXFP4/NVFP4 do NOT win (fake-quant); earlier recommendation | 7B: 0.288 / 0.380 | 250 | analysis/c17_q7_summary.json |
analysis/c17_quantizer_sweep.py |
retracted (see note) |
| C8 | Static outlier ROM valid: top-2 channel stability ≥ 0.90 (mean 0.958/0.986/0.984) | mean per scale | 8 batches × 16 | analysis/c19_{q05,q15,q7}_summary.json |
analysis/c19_outlier_stability.py |
provisional |
| C9 | Outlier-channel concentration 5.4–8.0× median channel | per scale | 8 batches × 16 | analysis/c19_{q05,q15,q7}_summary.json |
analysis/c19_outlier_stability.py |
provisional |
| C10 | Combined compression CQ-4 ~3.8×, CQ-4+ ~3.6× (D=64) | accounting | — | REVAMP_SPEC.md §4 |
(analytical) | provisional — confirm vs KVCE packer |
| C11 | Reference reproduces c17 bit-exactly: max|Δ|=0.000 over 6 variants × 3 models | Δ=0.000 | 250 | analysis/c20_{q05,q15,q7}_summary.json |
analysis/c20_reference_repro.py |
screening (GATE PASS, spec §7 step 1) |
| C12 | Shippable tier CQ-4+ (outlier K-only + per-tok V) ~FP16 at small scale; 7B point est. 0.584 sits −0.032 below the c17 both-sides outlier row but inside overlapping CIs | q05 0.432 / q15 0.536 / q7 0.584 [0.522,0.643] | 250 | analysis/c20_{q05,q15,q7}_summary.json |
analysis/c20_reference_repro.py |
screening — not a confirmed per-model ordering; resolve at n≥1000 (Phase 3) |
| C13 | Golden-vector self-consistency: numpy compress→decompress round-trips and torch fakequant == numpy pack→unpack bit-exactly for all 9 vectors (CQ-8/4/4+) | bit-exact | 9 vectors | reference/testvectors/ + manifest.json |
reference/gen_testvectors.py, reference/test_reference.py |
confirmed (deterministic, SEED=20260622) |
| C14 | WHT-rotated INT3 value codec bit-exact (WHT rotation originates in TurboQuant+, themoddedcube/turboquant-plus turboquant/wht_kernel.py): round-trips + matches the LonghornSilicon KVE hardware reference (channelquant_ref.hpp / wht_unit.sv) on real Qwen tensors (348,160/348,160 elems, D=64+128); pack_int3 = 3.000 bits/val |
bit-exact | 44 slices | KVE rtl/tb/testvectors/qwen/*/multi/ |
reference/channelquant_ref.py, reference/test_reference.py |
confirmed (deterministic; cross-checked vs KVE C++/RTL) |
| C15 | WHT-rotated INT3 values near-lossless (WHT rotation reused from TurboQuant+ / themoddedcube/turboquant-plus, by C. Talasila; A. Bandi identified the TurboQuant+ rotation (from its paper) as the fix for the 4-bit outlier problem → flat-INT3 CQ-3-rot, joint A. Bandi + C. Talasila): HellaSwag acc_norm within ~0.005 of FP16 — Qwen2-1.5B −0.004, 0.5B −0.007, Llama-3.2-1B +0.004; naive INT3 collapses like naive INT4 |
q15 −0.004 / q05 −0.007 / llama +0.004 | 1000 | KVE analysis/wht_ref_accuracy_*.json |
KVE analysis/wht_ref_accuracy.py |
screening — measured in the KVE repo at n=1000; re-derive in this repo's harness (Phase 3) |
| C16 | CQ-3-rot combined compression ~4.8× at ~3.3 bits/value (WHT INT3 values + CQ-4+ keys) | accounting | — | README.md tier table |
(analytical) | provisional — confirm vs KVE packer |
| C14 | Group-size knee G=128*: acc_norm statistically flat in G (gaps ≪ n=250 CI) → pick the bits floor that still streams; G=128 cq4 ≈ 4.13 eff-bits, near-lossless | knee | 250 | analysis/c22_{q05,q15,q7}_sweep.json, analysis/fig_group_size_pareto.png |
analysis/c22_group_size_sweep.py |
screening (confirm headline G=128 at n≥1000, Phase 3) |
| C15 | Static calibrated outlier mask reproduces c19 concentration (5.43/7.76/8.0 vs 5.4/7.8/8.0) — the per-(layer,head) ROM is a stable weight property | 5.43/7.76/8.0 | 128 calib | reference/masks/{q05,q15,q7}_k2.npz + .json |
analysis/outlier_calibration.py |
screening |
| C16 | At D=128 the "+" outlier lane shows no significant benefit over plain CQ-4. Settled at n=1000 (paired): CQ-4+ − CQ-4 delta CI includes 0 at both scales (1.5B +0.012 [−0.001,0.025] p=0.088; 7B −0.002 [−0.017,0.013] p=0.897) | paired Δ | 1000 | analysis/c23_{q15,q7}_headline.json |
analysis/c23_headline.py |
confirmed → D=128 default CQ-4, "+" lane optional (fed to KVCE contract v0.2 §4.1) |
| C12′ | Headline (n=1000): CQ-4+ ≥ FP16−0.02 at 0.5B (0.415 vs 0.426, Δ−0.011), 1.5B (0.517 vs 0.522, Δ−0.005), 7B (0.613 vs 0.614, Δ−0.002); ≤4.22 eff-bits; CQ-4 also passes (0.5B −0.005, 1.5B −0.017, 7B +0.001). CQ-4+ vs CQ-4 (paired): only a marginal, non-significant +0.012 at 1.5B (D=128, p=0.09); −0.002 at 7B; −0.006 at 0.5B (D=64, p=0.50) — reverses the n=250 screening's D=64 "helps", so CQ-4 is the default at every head dim. Resolves the Phase-1 7B caveat | Δ ≤ 0.02 | 1000 | analysis/c23_{q05,q15,q7}_headline.json |
analysis/c23_headline.py |
confirmed (acceptance gate, spec §7) |
| C14′ | Headline G=128 confirmed near-lossless at n=1000 (acceptance passes at the pinned knee on 1.5B & 7B); the full G-flatness Pareto remains screening (n=250) | see C12′ | 1000 | analysis/c23_{q15,q7}_headline.json |
analysis/c23_headline.py |
confirmed (G=128 point; Pareto shape still screening) |
| C17 | Recover generalizes: CQ-4+ ≥ FP16−0.02 on a non-Qwen GQA model (Mistral-7B) AND a non-HellaSwag task — HellaSwag Δ−0.001, ARC-Challenge Δ−0.019; CQ-4 lossless on Mistral×HellaSwag (+0.002) | Δ ≤ 0.02 | 1000 | analysis/c24_mistral_{arc,hellaswag}.json |
analysis/c24_generalization.py |
confirmed |
| C18 | HONEST — the collapse is model-dependent, tracked by the outlier TAIL not the median: naive per-token INT4 craters on Qwen2 (Δ≈−0.29/−0.40) but only dips −0.02 on Mistral-7B. Median per-head concentration is ~identical (top-2/median 6.0 vs 6.1; ~9% energy-in-top-2 both) so a median stat cannot distinguish them; the difference is the tail — Qwen has extreme heads (top-2 up to 247× median, 35% of key energy in 2 channels) absent in Mistral (max 16×, 18%) | tail 247× vs 16× | 1000 acc / 128 calib | analysis/c24_mistral_*.json, analysis/c25_key_spectra_*.npz |
analysis/c24_generalization.py, analysis/c25_capture_key_spectra.py |
confirmed — mechanistic; reframes value prop as "safe near-lossless default" |
| C19 | Outlier concentration reproduces on a non-Qwen arch: Mistral-7B median 7.62 [p10 5.94, p90 10.48] — extends C15 (static ROM is a general weight property) | 7.62 | 128 calib | reference/masks/mistral_k2.{npz,json} |
analysis/outlier_calibration.py |
confirmed (deterministic) |
C7 note (retraction): the docs/research_kv_quant_landscape.md note
(2026-06-18) recommended microscaling FP4 (MXFP4/NVFP4) as the 4-bit target.
c17 (fake-quant) showed it is erratic and does not win; per-channel INT4 is the
simpler clear winner. Retained as a documented correction, not a live claim.
Caveat: real Blackwell NVFP4 with hardware rounding may differ — not our bet.
Phase 3 executed 2026-07-01 (H100 host — see NOTES). Headline + generalization landed: C12′/C14′/C16/C17/C18/C19 above. Nothing Phase-3 is pending. Remaining:
- Phase 4 (method paper) — prose pass over
docs/paper/draft.md§4/§4.5 using the confirmed claims; optional generalization figure. - Silicon (KVCE companion) — area/Fmax vs TurboQuant+; no number here.
- 3-way Python↔C++↔SV bit-exact parity — run in the KVCE block against this repo's golden vectors, not here.
- Area / Fmax vs TurboQuant+ baseline — produced by the KVCE revamp on its RTL; no number exists yet and none will be fabricated here. The method paper notes it as forthcoming / companion.