AI analysis

[mcelrath]

ROCmFP4 — empirical evaluation on gfx1100

TL;DR

Tested three ROCmFP4 variants on Qwen3.5-0.8B (wikitext-2, 1126 windows). The rocmfp4_bf16scale variant (g=32, bf16 scale, 5.0 bpw) is competitive at PPL 20.04, beating braidinfer's PcG32Q4 (4.25 bpw, PPL ~21.0) despite higher bpw. The E4M3-scale variants are broken: rocmfp4_fast hits PPL 47.44 (2.5× baseline) and rocmfp4_full hits 89.09 (4.8× baseline). The codebook — the coarse {0, ±1..±4, ±6, ±8, ±10} ladder — is the dominant error source, adding 5-10× MSE over uniform int4 across FFN and attention weight types. E4M3 scale is acceptable when paired with a finer codebook (uniform int4 + E4M3 adds only 10-100% extra MSE depending on type) but catastrophic when combined with the already-coarse ladder. The dequant kernel is clean and memory-bandwidth-bound at 1.24 ms for 78.6M elements on gfx1100.

---

Setup

• Hardware: RX 7900 XTX (gfx1100, RDNA3) — NOT Strix Halo (gfx11.5.1). Results are valid for RDNA3 but may differ on Strix Halo.
• Model: Qwen3.5-0.8B
• Dataset: wikitext-2-raw-v1 test split, 1126 sliding windows, attn_implementation="eager"
• E4M3 note: emulated via torch.float8_e4m3fn. Rounding mode and denormal handling may differ from the author's native implementation.
• Embeddings / lm_head: skipped — group_size (16 or 32) does not divide vocab dimension (248320), left in bf16.

---

What we tested

Variant	group_size	scale_dtype	bpw
rocmfp4_full	16	e4m3fn	4.50
rocmfp4_fast	32	e4m3fn	4.25
rocmfp4_bf16scale	32	bf16	5.00

rocmfp4_bf16scale replaces E4M3 scale with bf16 as a diagnostic control to isolate codebook error from scale-quantization error.

---

Headline result: PPL on Qwen3.5-0.8B

Source: exp20_rocmfp4_ppl.json (1126 windows, 95% CI)

Method	bpw	PPL	95% CI	notes
baseline (bf16)	16.0	18.67	[18.20, 19.15]
rocmfp4_bf16scale	5.00	20.04	[19.53, 20.56]	codebook only; bf16 scale
braidinfer PcG32Q4	4.25	21.00	(from exp20 ref)	uniform int4 + bf16 scale, g=32
turboquant	4.25	21.46	(from exp20 ref)
had_uniform	4.25	23.01	(from exp20 ref)
rocmfp4_fast	4.25	47.44	[46.13, 48.79]	codebook + E4M3 scale
rocmfp4_full	4.50	89.09	[86.48, 91.77]	codebook + E4M3 + g=16

rocmfp4_bf16scale wins against all non-baseline competitors at the cost of more bpw (5.00 vs 4.25). The E4M3 variants are catastrophic — rocmfp4_fast is 2.54× PPL over baseline despite using a finer group, rocmfp4_full is 4.76× despite using a smaller group.

---

The 4-way decomposition: codebook × scale

Source: exp_e4m3_ablation.json

2×2: {uniform int4, ROCmFP4 codebook} × {bf16 scale, E4M3 scale}.

Codebook effect (rocmfp4_fast vs pc_g32_bf16, same g=32, bf16 scale held)

Weight type	pc_g32_bf16 MSE	rocmfp4_fast MSE	codebook ratio
mlp_down	4.51e-07	4.37e-06	9.7×
mlp_gate	1.38e-06	7.52e-06	5.5×
mlp_up	4.90e-07	4.12e-06	8.4×
attn_k	8.46e-07	6.50e-06	7.7×
attn_o	1.23e-06	7.21e-06	5.9×
attn_q	1.69e-06	8.02e-06	4.7×
attn_v	1.87e-06	7.84e-06	4.2×

Scale effect (pc_g32_e4m3 vs pc_g32_bf16, uniform int4 held)

Weight type	pc_g32_bf16 MSE	pc_g32_e4m3 MSE	scale ratio
mlp_down	4.51e-07	1.74e-06	3.9× (outlier-sensitive)
mlp_gate	1.38e-06	1.47e-06	1.1×
mlp_up	4.90e-07	5.28e-07	1.1×
attn_k	8.46e-07	1.68e-06	2.0×
attn_o	1.23e-06	2.15e-06	1.8×
attn_q	1.69e-06	1.83e-06	1.1×
attn_v	1.87e-06	2.03e-06	1.1×

Conclusion: codebook is the dominant error source (5-10× MSE). E4M3 scale alone adds 10-100% MSE — survivable in isolation. Combined with the coarse 15-value ladder, errors multiply, not add. The {0, ±1..±4, ±6, ±8, ±10} codebook has integer spacing in the body of the Gaussian weight distribution, wasting bits on the tails.

For reference, Lloyd-Max-optimal 4-bit centroids for Gaussian weights (from exp20_turboquant_weights.json):

±{0.128, 0.388, 0.657, 0.943, 1.257, 1.618, 2.069, 2.733}

Far more finely spaced in the Gaussian body than the ROCmFP4 ladder.

---

KV-cache: where a heavy-tail ladder might have helped

Source: exp1_rocmfp4_kvcache.json (4-bit, layer 4)

Method	K MSE	K cosine sim	attn_output MSE
kivi_per_channel	1.43e-02	0.9969	4.72e-04
rocmfp4_fast	2.51e-02	0.9947	1.57e-03
kivi_per_token	4.98e-02	0.9895	3.85e-03
polarquant	2.12	0.9939	7.69e-02
residual_pc (2+2 bit)	6.34e-05	0.99999	3.24e-06

rocmfp4_fast falls between kivi_per_channel (better) and kivi_per_token (worse). K-cache vectors are RoPE-rotated and have approximately uniform magnitude — the heavy-tail codebook confers no advantage here.

---

Per-tensor MSE breakdown

Source: exp6_rocmfp4_mse.json — mean MSE per weight type; embeddings excluded.

Weight type	n	pc_4bit (ref)	rocmfp4_fast	ratio	rocmfp4_bf16s	ratio
attn_k	6	2.57e-06	6.50e-06	2.5×	1.17e-06	0.5×
attn_o	6	3.74e-06	7.21e-06	1.9×	1.67e-06	0.4×
attn_q	6	1.21e-05	8.02e-06	0.7×	2.38e-06	0.2×
attn_v	6	5.96e-06	7.84e-06	1.3×	2.50e-06	0.4×
mlp_down	24	1.45e-06	4.37e-06	3.0×	6.39e-07	0.4×
mlp_gate	24	8.99e-06	7.52e-06	0.8×	1.95e-06	0.2×
mlp_up	24	2.01e-06	4.12e-06	2.1×	6.95e-07	0.3×
gdn_a	18	2.18e-06	1.23e-05	5.6×	6.48e-06	3.0×
gdn_b	18	8.60e-07	8.16e-06	9.5×	2.65e-06	3.1×
gdn_out	18	3.06e-06	7.04e-06	2.3×	1.23e-06	0.4×
gdn_qkv	18	2.22e-05	9.37e-06	0.4×	3.29e-06	0.1×
gdn_z	18	9.56e-06	8.53e-06	0.9×	2.47e-06	0.3×
gdn_conv	18	8.96e-04	4.58e-05	0.1×	3.19e-05	0.0×

Notes:

• rocmfp4_bf16scale beats per-channel int4 (ratio < 1) on attn and mlp types — denser g=32 grouping with exact bf16 scale outperforms g=∞ per-channel int4.
• GDN a/b weights are most sensitive: small row count (16 × 1024) means few elements per group, and the coarse codebook has disproportionate impact.
• gdn_conv (4-element convolutions) degrades badly under any group quantization.

---

Dequant wall-clock

Source: exp2_rocmfp4_dequant.json

Kernel	n_elements	mean (ms)	std (ms)	CV
rocmfp4_fast	78,643,200	1.237	0.003	0.2%

At ~1.6 TB/s theoretical peak: ~2 GB total traffic = ~1.2 ms theoretical. Measured 1.24 ms ≈ 97% of peak bandwidth. Kernel is clean (~250 LOC HIP, LDS-cached codebook), compiles cleanly with hipcc --offload-arch=gfx1100.

Python-based baselines from exp2_dequant_cost.json (kivi 1431 ms, residual_pc 3080 ms, polarquant 3741 ms) are dominated by Python dispatch overhead and are not directly comparable to the kernel.

---

Verdict / recommendation

Variant	bpw	PPL	Δ baseline	recommendation
rocmfp4_bf16scale	5.00	20.04	+1.37	USE — competitive
braidinfer PcG32Q4	4.25	21.00	+2.33	reference baseline
rocmfp4_fast	4.25	47.44	+28.8	DO NOT USE
rocmfp4_full	4.50	89.09	+70.4	DO NOT USE

ROCmFP4 with bf16 scale is production-ready for gfx1100. Beats braidinfer's PcG32Q4 by ~1 PPL point at higher bpw.

ROCmFP4 with E4M3 scale is not usable for any inference application on this model.

The codebook is the core weakness. The {0, ±1..±4, ±6, ±8, ±10} ladder misrepresents Gaussian-distributed weights by uniform-spacing in the body. NF4 or Lloyd-Max centroids would substantially reduce this error at the same bpw, as a drop-in codebook replacement.

Suggested path forward

1. Ship rocmfp4_bf16scale as the production variant today.
2. Switch to NF4 or Lloyd-Max codebook (same kernel structure, better MSE).
3. Only then re-evaluate E4M3 scale; with a denser codebook it may become acceptable.
4. KV-cache application is not a win — stick with kivi_per_channel for K-cache.

---

Caveats

• Only tested on Qwen3.5-0.8B. Larger models may have different sensitivity profiles.
• Only tested on gfx1100 (RX 7900 XTX), not Strix Halo (gfx11.5.1).
• torch.float8_e4m3fn may differ from the author's E4M3 in rounding mode or denormal handling.
• Embeddings and lm_head excluded from quantization throughout.
• wikitext-2 (1126 windows) can detect ~10% PPL differences; all effects reported are far larger.

---

Reproduce

# Phase 1 — per-tensor weight MSE (commit 01bec9c)
launch-gpu -g 1 -- python scripts/exp6_rocmfp4_mse.py \
  --output results/rocmfp4_eval/exp6_rocmfp4_mse.json

# Phase 2 — end-to-end PPL (commit 683330f)
launch-gpu -g 1 -- python scripts/exp21_rocmfp4_ppl.py \
  --output results/rocmfp4_eval/exp20_rocmfp4_ppl.json

# Phase 3 — E4M3 codebook × scale ablation (commit 49bbef8)
launch-gpu -g 1 -- python scripts/exp_e4m3_ablation.py \
  --output results/rocmfp4_eval/exp_e4m3_ablation.json

# Phase 4 — KV-cache MSE at 4-bit (commit 83eb449)
launch-gpu -g 1 -- python scripts/exp1_rocmfp4_kvcache.py \
  --output results/rocmfp4_eval/exp1_rocmfp4_kvcache.json

# Phase 5 — dequant wall-clock on gfx1100 (commit 94fae23)
hipcc --offload-arch=gfx1100 -O3 scripts/rocmfp4_dequant_bench.hip \
  -o scripts/rocmfp4_dequant_bench
launch-gpu -g 1 -- ./scripts/rocmfp4_dequant_bench \
  --output results/rocmfp4_eval/exp2_rocmfp4_dequant.json