GRIT — Grouped Reduced-precision Interchange Type

The 64 bytes your checkpoint is missing. Website · Live workbench · Paper · Spec · Reproduce everything

Here is an experiment you can run today. Take a GPTQ checkpoint and an AWQ checkpoint of the same model. Their zero-point planes can be byte-identical — same shapes, same dtypes, same tensor names — while decoding to different numbers, because one convention stores zero points as-is and the other stores them minus one, and nothing in either file records which. The decode semantics of every quantized checkpoint you have ever downloaded live in tool source code, not in the file that carries the bytes.

GRIT is the smallest complete fix we could specify: a descriptor language and boundary-check protocol for block-scaled, reduced-precision arrays (the MXFP4 / Q4_K / NVFP4 / sparse-INT8 family). A producer writes a 64-byte POD descriptor once; a consumer verifies descriptor-vs-bytes agreement in O(1) — measured at 296 ns in C — at every load boundary. One normative spec, five zero-dependency implementations (C11, C++20, Rust, Python-stdlib, strict TypeScript) that agree bit-for-bit, 68 SHA-256-pinned conformance vectors, and a cross-language fingerprint check that CI re-proves on every push, 96/96.

Two things in this repo are useful before anyone adopts anything:

1. grit scan — a diagnostic that requires zero adoption

Point it at safetensors / GGUF files or a directory tree. No framework change, no vendor cooperation, no registry:

pip install grit-datatype
python3 -m grit.scan [--deep] [--json] [--exit-policy graded|simple] PATH...

It recovers descriptors from a __grit__ safetensors header key or grit.desc.<tensor> GGUF KV entries — and where no descriptor exists (today: everywhere), it infers one from well-known ggml block structs, llama.cpp file types, and the GPTQ / AWQ / compressed-tensors safetensors families, then labels every inferred contract as inferred, never as declared. It reports:

• descriptor-vs-bytes violations — the SPEC §7.3 size equations run against the actual on-disk plane lengths, catching wrong group size, forgotten padding, dense/sparse confusion, truncated shards;
• declared-vs-actual disagreement — a GGUF whose general.file_type summary contradicts what the tensor bytes are (file_type_mismatch);
• convention ambiguity — 4-bit zero-point planes that are byte-identical under zpc=asis and zpc=minus1 (zp_convention_ambiguous);
• cross-file gid disagreement, with a field-by-field grit_explain diff;
• grade-drift clusters — one logical role carrying two grade_ids inside a single checkpoint, the shape of a half-converted export;
• honest inexpressibility — formats the v1 grammar cannot carry are reported as inferred_inexpressible, not silently approximated.

Exit codes: 0 clean, 1 structural violation, 2 disagreement, 3 warnings only, 4 unparseable. Machine-readable with --json; CI-gateable by design.

Field results, reported with the two-run structure the paper uses. We downloaded four real, popular checkpoints (GGUF Q4_K_M, GPTQ-Int4, AWQ-Int4, compressed-tensors W4A16) and read them byte by byte. Run 1 — hand analysis plus the scanner as first shipped — confirmed real mismatches in three of four files but auto-caught only one, indirectly. We closed exactly three inference gaps and re-ran on bit-identical bytes: 12 findings became 349, every mismatch class fires automatically, and a false-positive control (two known-clean files) stayed at zero findings, exit 0. The complete experiment log, both runs, is audit/scan-experiment.md.

The failure class this addresses is documented, not hypothetical: 2026 saw signed scales read as unsigned (roughly four orders of magnitude of dequant error), a swizzled producer feeding a linear consumer, dropped scale tensors and silent E8M0 truncation, across vLLM and SGLang; and the GPTQ "zero point minus one" convention turned tens of thousands of zero points into NaN perplexity as late as January 2026. Every one of those is a descriptor-vs-bytes or convention-mismatch question that something at the boundary could have answered.

2. The contract vocabulary

Even without the type, the field names are worth stealing. GRIT's Grade names things that current formats leave implicit:

• esr — the shared-scale derivation rule, with no default, always emitted. exp2floormaxabs vs exp2ceilmaxabs vs exp2rnemaxabs vs ratiomaxabs is a real fork; a PyTorch maintainer records in pytorch#146414 that there is no consensus and that the OCP floor rule differs from a vendor's modified ceiling rule. It is also the field the nearest neighbour drops: torchao has a ScaleCalculationMode and does not serialize it. GRIT's rne rule is specified to the bit — e_floor + (f·f > 2 ? 1 : 0) with f = amax·2^−e_floor exact in binary64 — and unit-tested in all five implementations (SPEC §9.2).
• Explicit zero-point conventions — zpc (asis vs minus1) and zps (shared vs l1-scaled) are declared flags, not folklore about which tool wrote the file.
• Exact format identity — e4m3 and e4m3fnuz are different formats, and spelling one as the other is a parse error rather than a silent alias.
• Sparsity metadata order — SPORDER_ASC_LSB is a declared, checkable precondition instead of the undefined behaviour PTX leaves it as.
• Layout, separately — a placement value, so that "which layout" and "which numeric contract" stop being one fused enum name like marlin_24. This is also the capability vllm#8913 has been asking for since 2024: "a more sophisticated way of describing packed layouts".

The type, for those who want it

A GRIT array is a quadruple (Grade, Placement, Planes, Shape):

• Grade — the complete numeric contract (element format, group sizes, scale formats, zero-point convention, sparsity, packing, rounding/saturation/ accumulation promises), with a canonical string form and a 64-bit id.
• Placement — the physical layout, carried as a 64-bit value rather than a type parameter, so a new layout is a new constant. Layout-as-value is standard practice (cuBLASLt orders, XLA LayoutProto, NVIDIA's CUtensorMap); what GRIT adds is that the value is hashed, registry-free, and attached to the bytes next to the numeric contract.
• Planes — up to four byte buffers: data, scale0, scale1, aux.
• Shape — rank and extents, carried in the view.

The whole thing serializes to a fixed-layout 64-byte descriptor — every field at a fixed offset, little-endian, no variable-length anything:

Offset	Field	Carries
[0, 8)	magic · hdr · ver · levels · flags	"GRIT", header size, version, level count, flag bits (zp presence & convention, sparsity, bit order, interleave)
[8, 16)	elem · scale0 · scale1 · zp	formats from a closed ladder (e2m1, e8m0, e4m3, binary16/32, u2/u4/u6/u8, s4/s8)
[16, 22)	axis · k0 · k1	group axis and group sizes for one- or two-level block scaling
[22, 28)	sparse · container	n:m sparsity, metadata format/order, container packing
[28, 32)	note	assumption provenance — the one field excluded from every fingerprint, so annotating a file never changes its identity
[32, 56)	placement × 3	64-bit placement ids for the data, scale, and metadata planes
[56, 64)	grade_id	FNV-1a-64 of the canonical grade string

The grade + hashed frame + shape fingerprint to a 128-bit gid (FNV-1a-128): two tools holding the same gid hold the same contract. You can build, corrupt, and check descriptors interactively in the workbench — it runs the byte-identical npm build in your browser.

decode() is the normative semantics: two GRIT arrays are semantically equal iff their decodes are equal, and everything else exists to make that statement checkable at an O(1) boundary crossing. The two testable guarantees:

1. equal gid and equal plane bytes ⇒ bit-identical decode() on every conformant implementation;
2. grit_check never has undefined behaviour, even on adversarial 64-byte descriptors.

What holds this together is the law set around decode(): NaN poison, sparse-wins-over-poison, slice honesty, group-axis transpose being requantization rather than a view, fixed outermost-first evaluation, no FMA substitution. StableHLO's 0 < scales constraint affirmatively excludes the hardest of these cases, so the two semantics are disjoint exactly on the edges where implementations diverge in practice.

What the check costs

Measured on an i9-13900HK (one machine, indicative not guaranteed — harnesses in bench/, methodology in the paper §8.4):

	level-1 check (structure)	level-2 digest (contents)
C	296 ns / call (887 cycles)	0.46 GB/s
Rust	687 ns	1.24 GB/s
Python	9.8 µs	0.017 GB/s

Checking every tensor of a synthetic 8-shard, 1.07 GB MXFP4 checkpoint at load time: +33.4 KB of headers, +2.7 ms, +108 KiB RSS for 192 checks.

Verification

Every number below is CI-enforced on each push and reproducible from a clean checkout via REPRODUCING.md (prereqs, exact commands, expected outputs, a few minutes end to end):

Suite	Result
Shared conformance vectors (SHA-256-pinned)	68 entries — 34 positive · 15 negative · 16 fingerprint · 3 supersedes
C (gcc, C11)	910 checks, 0 failures
C++ (g++, C++20; plain + strict + ASan)	801 checks, 0 failures
Rust (stable, no unsafe in the hot path)	37 tests + conformance suite
Python (stdlib only; numpy never imported)	124 tests
TypeScript (strict, Node ≥ 18)	91 tests
Cross-language gid (spec/crosslang/run.sh)	96/96 descriptors identical across all five + the reference, 96 distinct gids

python3 spec/reference.py verify        # 68 entries, 0 failures
cd c && make clean && make && ./test_grit
cd cpp && make clean && make test       # also: make asan, make strict
cd rust && cargo test --release
cd python && python3 -m unittest discover
cd typescript && npm ci && npm test
bash spec/crosslang/run.sh              # 96/96 on every implementation

What GRIT does not claim

spec/SPEC.md §0.2 carries a component-by-component claims table with the closest prior art for each, and §0.3 states the scope honestly. The short version:

• Parameterised quantized types, nested two-level scales, and scale + sparsity in one format are not GRIT's — see MLIR's UniformQuantizedSubChannelType, compressed-tensors, and Qualcomm's LPBQ struct.
• "The arithmetic contract as a first-class value" is deployed art (StableHLO / JAX DotAlgorithm), and the refinement structure is textbook graded modal typing under the same word Grade (Granule, ICFP 2019).
• Canonicalise-then-fingerprint for registry-free cross-language identity is Apache Avro's, step for step; GRIT changes the object being hashed.
• The portable POD descriptor pattern is the Khronos Data Format Specification's and DLPack's, including the identical LSB-first sub-byte packing rule.
• FNV-1a fingerprints defend against drift and mislabeling, not against an adversary — there is no collision resistance claim, and consumers MUST treat a gid match as a spurious-match risk bounded only under non-adversarial assumptions (SPEC §11).

And the real v1 coverage holes (SPEC §0.3): AMD FNUZ FP8 is inexpressible (bias_ovr MUST be 0 in v1), there is no codebook / LUT element class — so NF4 and the llama.cpp IQ* families are out of scope, GPTQ act-order g_idx grouping is unsupported — contiguous grouping is structural, and act-order cannot be hidden in Placement without making the placement-invariance law false — and dense-nibble zero-point planes are inexpressible, so GPTQ / AWQ qzeros, which pack two values per byte, are not byte-representable under §3.4's CT_PAD8-always rule. The scanner names all of these inferred_inexpressible instead of pretending. The field study is n=4: existence of the mismatch classes, not prevalence — a prevalence claim needs a stratified sweep of hundreds of checkpoints, which is future work.

Repository layout

spec/            SPEC.md (normative, 2,164 lines), reference.py (executable
                 reference), conformance/vectors.json (68 shared vectors),
                 crosslang.py + crosslang/ (cross-language gid harness)
c/               C11 implementation (grit.h/grit.c) + 910-check harness
cpp/             C++20 header-only implementation (include/grit/grit.hpp)
rust/            Rust crate (no unsafe in the hot path) + conformance tests
python/          Pure-Python package `grit`, incl. the `grit.scan` CLI
typescript/      Strict TypeScript implementation + node:test suite
bench/           Cost-measurement harnesses behind the paper's §8.4 tables
paper/           The 26-page paper (grit.tex + grit.md mirror + refs.bib)
docs/            Website + workbench, RELEASING.md, launch notes
audit/           Field-study experiment log & design-review records
REPRODUCING.md   Every claim above, as copy-pasteable commands
CONFORMANCE_REPORT.md   Latest integration-gate report across all five

The four reference grades (SPEC §10): GRADE_MXFP4 (e2m1 / e8m0 scales, k=32), GRADE_Q4KS (u4 two-level u6/binary16 with l1-scaled zero points), GRADE_NVF4 (e2m1 / e4m3 + binary32, k=16), GRADE_I8S24 (s8, 2:4 sparse, binary16 scales) — plus the _RNE / _RTZ esr variants that exist precisely because the rounding rule is part of the contract.

How this was built

GRIT started as a dare, not a product idea: find something genuinely missing, and only build it if the gap survives an adversarial attempt to prove it already exists. Before any code, the idea was attacked with a sweep of the closest work — MLIR sub-channel quantized types, StableHLO DotAlgorithm, compressed-tensors, Avro's canonicalize-then-fingerprint discipline, Khronos DFS, DLPack, torchao, TOSA block-scaled types — and every novelty claim that did not survive was retracted before publication. What survived was not a format but a missing contract, which decided everything else: a checkable type, not another container.

The repository is built in the open by a human–AI team: singhpratech (direction, constraints, the standard of evidence) with Claude (Anthropic) as repository co-author (orchestrated implementation across five languages, exact-arithmetic oracle sweeps against every encoder, and a hostile three-reviewer panel run against the paper before release). The working rules: every number must regenerate from a clean checkout, and every reviewer finding is either fixed or documented as a limitation — never softened. Nothing here asks to be trusted; everything here asks to be checked.

Versioning

The repo tag and all three registries move in lockstep — v1.1.0 == npm grit-datatype@1.1.0 == crates.io grit-datatype@1.1.0 == PyPI grit-datatype 1.1.0. Release mechanics are docs/RELEASING.md.

Apache-2.0 · built in the open by singhpratech, with Claude (Anthropic) as repository co-author.