MegaKernels in Iris 🔥 - #541

Draft

neoblizz wants to merge 63 commits into

mainfrom

neoblizz/megakernel-perf

Commits on Jun 22, 2026

Add GPT-OSS-120B single-GPU persistent megakernel
neoblizz
committed
18cee47
Add FP4xFP8 scaled-MFMA quantized path to GPT-OSS megakernel
neoblizz
committed
3d863fe
Optimize megakernel: tiled GEMVs and fewer barriers
neoblizz
committed
aa1d18a
Fold FP8 activation quant into SwiGLU to drop a barrier per expert
neoblizz
committed
35ab0b9
Update megakernel README with optimization results
neoblizz
committed
c72704d
Clean up megakernel example docs and comments
neoblizz
committed
e5a57f3
Batch experts per phase to cut grid barriers
neoblizz
committed
1a481a8
Tune expert GEMV tiling and coalesce FP4 weight loads
neoblizz
committed
7309141
Widen BF16 GEMV row tile for better lm-head bandwidth
neoblizz
committed
1c3f5d3
Fold post-attention residual into router and expert norm
neoblizz
committed
b0ea166
Fuse argmax into the LM-head GEMV
neoblizz
committed
6631f20
Widen expert GEMV K block to 1024
neoblizz
committed
efa50b4
Use a separate row-tile size for the down-projection GEMV
neoblizz
committed
3fe44e6
Vectorize attention KV loop with a blocked flash-decode
neoblizz
committed
3fc5534
Update README TPOT figure to 9.4 ms/token
neoblizz
committed
6075adc

Commits on Jun 23, 2026

Use 192 persistent programs to lower barrier cost
neoblizz
committed
89f2421
Fuse KV-cache append into the attention phase
neoblizz
committed
fdffe81
Merge router, expert-input quant and top-k to drop a barrier
neoblizz
committed
17ae0b7
Fold final residual into the last expert down GEMV
neoblizz
committed
383c7be
Use BLOCK_M=8 for the BF16 GEMVs
neoblizz
committed
b7ed1e0
Fold final RMSNorm into the LM-head GEMV
neoblizz
committed
ef38981
Retune to 184 persistent programs
neoblizz
committed
4b592d0
Widen BF16 GEMV K block to 1024
neoblizz
committed
4c5212d
Give the LM-head GEMV its own row-tile size
neoblizz
committed
b8063ab
Retune to 180 persistent programs (TPOT 5.78 ms)
neoblizz
committed
a3f68a6
Add input/output length benchmark and results table
neoblizz
committed
c1a3668
Pipeline GEMV K-loops with tl.range(num_stages)
neoblizz
committed
f688b79
Refresh ISL/OSL benchmark table with pipelined TPOT
neoblizz
committed
a1ae9ff
Relax barrier poll to avoid per-iteration L1 invalidate
neoblizz
committed
dc00b72
Refresh benchmark numbers for relaxed barrier
neoblizz
committed
34e1b11
Add optional FP8 attention/router weights (W8 scalar-FMA)
neoblizz
committed
cc89d4d
Document FP8 attention mode and refresh benchmarks
neoblizz
committed
89fbddd
Add accuracy eval harness with optional logit dump
neoblizz
committed
b30297f
Add per-component FP8 control and MXFP8 block-scale weight option
neoblizz
committed
6457727

Commits on Jun 24, 2026

Commits on Aug 8, 2026

Fix megakernel output corruption from the grid barrier's cache invalidate

mawad-amd
and
claude
committed
69bba6b
Warn against replacing the barrier's atomic poll with a plain load

mawad-amd
and
claude
committed
ace48fd
Say plainly that the barrier's atomic poll is not understood

mawad-amd
and
claude
committed
1a1a621
State the barrier's precondition as an access pattern, not a batch size

mawad-amd
and
claude
committed
2ea245f
Record the conditions the barrier's 0/300 was measured under

mawad-amd
and
claude
committed
e8289d2
Replace the contention caveat with the measurement that refuted it

mawad-amd
and
claude
committed
0cdc9c1
Mark the pooled failure rate provisional and state what pooling assumes

mawad-amd
and
claude
committed
c68fbaf
Rule out "correct by being slow" for the barrier's atomic poll

mawad-amd
and
claude
committed
a33d83f
Retire the improvement ratio between the two barriers

mawad-amd
and
claude
committed
4eccf1c
Split the barrier's failure rate by machine instead of pooling it

mawad-amd
and
claude
committed
1b9305d
Correct the old barrier's failure rate: 5% is venue-conditioned

mawad-amd
and
claude
committed
b670c25
Retire the 15/300 figure: it has not reproduced, and the fix does not need it

mawad-amd
and
claude
committed
eb4ca66
Sort the old barrier's failures by host, and say the clean venues were one family

mawad-amd
and
claude
committed
c3f5ade
Justify the change on one machine instead of averaging across hosts

mawad-amd
and
claude
committed
df4c9a8
Finalise the one-machine comparison now both arms are complete

mawad-amd
and
claude
committed
14872b3
Record that the failing host is one machine, not a node family

mawad-amd
and
claude
committed
549994b
State the thor-3 result's strength under both pooling choices

mawad-amd
and
claude
committed
03faf25
Fix the multi-GPU KeyError, guard the grid size, add requirements

mawad-amd
and
claude
committed
0b71dc0
Correct the fallback advice: _barrier does not fix the reuse case either

mawad-amd
and
claude
committed
420005f
Make test_barrier deterministic and say why it inlines its barrier

mawad-amd
and
claude
committed
905f7db
Document the unguarded grid-residency hazard in the multi-GPU path

mawad-amd
and
claude
committed
ebfffb0