Skip to content

perf(ggml/hip): add batched ROCmFPX MMQ#513

Draft
weicj wants to merge 1 commit into
Luce-Org:codex/ds4-rocmfpx-serverfrom
weicj:draft/perf-ggml-hip-rocmfpx-batched-mmq
Draft

perf(ggml/hip): add batched ROCmFPX MMQ#513
weicj wants to merge 1 commit into
Luce-Org:codex/ds4-rocmfpx-serverfrom
weicj:draft/perf-ggml-hip-rocmfpx-batched-mmq

Conversation

@weicj

@weicj weicj commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add gfx1151 batch MMQ support for ROCmFP2, ROCmFP3, and ROCmFP4_FAST weights. This keeps DeepSeek V4 Flash batch prefill on the GPU for both dense projections and routed MoE matrices instead of falling through to per-expert dequantization and hipBLAS calls.

Stack: #502 -> #503 -> this PR.

Changes

  • Dispatch ROCmFP2, ROCmFP3, and ROCmFP4_FAST through MMQ on gfx1151.
  • Add MMQ tile loaders for the packed ROCmFPX formats and preserve their per-half-block scales during the integer WMMA reduction.
  • Generate the three ROCmFPX MMQ template instances.

Validation

Review in cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant