Skip to content

fix: update support matrix [main]#1366

Merged
tianhaox merged 3 commits into
ai-dynamo:mainfrom
tianhaox:fix/update-support-matrix-main-local
Jul 17, 2026
Merged

fix: update support matrix [main]#1366
tianhaox merged 3 commits into
ai-dynamo:mainfrom
tianhaox:fix/update-support-matrix-main-local

Conversation

@tianhaox

@tianhaox tianhaox commented Jul 15, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Contributor

Generated by a full local run of tools/support_matrix/generate_support_matrix.py on main @ 2feebac, using the CI docker image (docker/Dockerfile --target test), 16 worker processes, ~22.5h wall clock. 10502 configurations tested across 5251 model/system/backend/version combinations (106 database groups), including the recently added sglang 0.5.14 / vllm 0.24.0 data that the in-repo matrix predates.

Context: the scheduled Support Matrix Daily Run has been hitting the 8h workflow timeout daily (every run since at least 2026-07-09 was cancelled), so main's checked-in matrix is stale. This PR is the offline equivalent of the support-matrix-main.yml flow.

⚠️ Validation note for reviewers: compare_support_matrix.py exits with validation errors (exit 2) on this diff: 588 rows transitioned FRAMEWORK_INCOMPATIBLE → FAIL, all for MiniMaxAI/MiniMax-M2.5 and zai-org/GLM-5-FP8. These are label-level changes, not functional regressions — the combos failed before as well. The old labels were produced by the autofix flow ("Final best-effort FRAMEWORK_INCOMPATIBLE" rows), while the current _is_known_framework_incompatible_gap allowlist does not recognize the failure signatures on the newer databases (e.g. Unsupported gemm quant mode 'fp8_block' outside the pinned versions, and the new Unsupported dsa_context_module quant mode 'fp8' error introduced by the data-driven FMHA quant resolution in #1337). The allowlist likely needs a follow-up (or an autofix pass) to cover the new backend versions/error strings.

This PR updates the split support matrix CSV files with the following changes:

Summary

Category Count
Silicon regressions (PASS -> non-PASS) 101
Hybrid regressions (HYBRID_PASS -> non-pass) 0
Silicon fixes (non-PASS -> PASS) 437
New hybrid coverage (non-pass -> HYBRID_PASS) 674
Reclassified as hardware-incompatible (FAIL -> HW_INCOMPATIBLE) 4
Reclassified as framework-incompatible (FAIL -> FRAMEWORK_INCOMPATIBLE) 196
Removed rows 116
Added rows 3098
Command/Source metadata changes 6048
Header changed yes

1. Silicon regressions (PASS -> non-PASS): 101 rows

HuggingFaceID Architecture System Backend Version Mode Previous Status New Status
Qwen/Qwen3-0.6B Qwen3ForCausalLM b60 vllm 0.12.0 disagg PASS FAIL
Qwen/Qwen3-1.7B Qwen3ForCausalLM b60 vllm 0.12.0 disagg PASS FAIL
Qwen/Qwen3-8B Qwen3ForCausalLM b60 vllm 0.12.0 disagg PASS FAIL
Qwen/Qwen3.5-35B-A3B Qwen3_5MoeForConditionalGeneration b200_sxm sglang 0.5.10 agg PASS HYBRID_PASS
Qwen/Qwen3.5-35B-A3B Qwen3_5MoeForConditionalGeneration b200_sxm sglang 0.5.10 disagg PASS HYBRID_PASS
Qwen/Qwen3.5-35B-A3B Qwen3_5MoeForConditionalGeneration b200_sxm sglang 0.5.12 agg PASS HYBRID_PASS
Qwen/Qwen3.5-35B-A3B Qwen3_5MoeForConditionalGeneration b200_sxm sglang 0.5.12 disagg PASS HYBRID_PASS
XiaomiMiMo/MiMo-7B-Base MiMoForCausalLM b60 vllm 0.12.0 disagg PASS FAIL
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM b200_sxm vllm 0.19.0 agg PASS HYBRID_PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM b200_sxm vllm 0.19.0 disagg PASS HYBRID_PASS
deepseek-ai/DeepSeek-V4-Flash DeepseekV4ForCausalLM b200_sxm sglang 0.5.10 agg PASS FAIL
deepseek-ai/DeepSeek-V4-Flash DeepseekV4ForCausalLM b200_sxm sglang 0.5.10 disagg PASS FAIL
deepseek-ai/DeepSeek-V4-Pro DeepseekV4ForCausalLM b200_sxm sglang 0.5.10 agg PASS FAIL
deepseek-ai/DeepSeek-V4-Pro DeepseekV4ForCausalLM b200_sxm sglang 0.5.10 disagg PASS FAIL
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration b200_sxm vllm 0.19.0 agg PASS HYBRID_PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration b200_sxm vllm 0.19.0 disagg PASS HYBRID_PASS
meta-llama/Meta-Llama-3.1-8B LlamaForCausalLM b60 vllm 0.12.0 disagg PASS FAIL
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM b200_sxm sglang 0.5.9 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM b200_sxm sglang 0.5.9 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM b200_sxm vllm 0.19.0 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM b200_sxm vllm 0.19.0 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM b300_sxm sglang 0.5.10 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM b300_sxm sglang 0.5.10 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM b300_sxm sglang 0.5.9 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM b300_sxm sglang 0.5.9 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM b300_sxm vllm 0.19.0 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM b300_sxm vllm 0.19.0 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM b60 vllm 0.12.0 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM b60 vllm 0.12.0 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM gb200 sglang 0.5.10 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM gb200 sglang 0.5.10 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM gb200 sglang 0.5.9 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM gb200 sglang 0.5.9 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM gb200 vllm 0.14.0 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM gb200 vllm 0.14.0 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM gb200 vllm 0.19.0 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM gb200 vllm 0.19.0 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM gb300 sglang 0.5.10 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM gb300 sglang 0.5.10 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM gb300 sglang 0.5.9 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM gb300 sglang 0.5.9 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM gb300 vllm 0.19.0 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM gb300 vllm 0.19.0 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h100_sxm sglang 0.5.10 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h100_sxm sglang 0.5.10 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h100_sxm sglang 0.5.6.post2 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h100_sxm sglang 0.5.6.post2 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h100_sxm sglang 0.5.9 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h100_sxm sglang 0.5.9 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h100_sxm vllm 0.14.0 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h100_sxm vllm 0.14.0 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h100_sxm vllm 0.19.0 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h100_sxm vllm 0.19.0 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h200_sxm sglang 0.5.10 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h200_sxm sglang 0.5.10 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h200_sxm sglang 0.5.6.post2 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h200_sxm sglang 0.5.6.post2 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h200_sxm sglang 0.5.9 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h200_sxm sglang 0.5.9 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h200_sxm vllm 0.14.0 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h200_sxm vllm 0.14.0 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h200_sxm vllm 0.19.0 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM h200_sxm vllm 0.19.0 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM l40s vllm 0.14.0 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM l40s vllm 0.14.0 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM rtx_pro_6000_server sglang 0.5.10 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM rtx_pro_6000_server sglang 0.5.10 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM rtx_pro_6000_server vllm 0.19.0 agg PASS FRAMEWORK_INCOMPATIBLE
nvidia/Llama-3.1-70B-Instruct-FP8 LlamaForCausalLM rtx_pro_6000_server vllm 0.19.0 disagg PASS FRAMEWORK_INCOMPATIBLE
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM b200_sxm sglang 0.5.10 agg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM b200_sxm sglang 0.5.10 disagg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM b300_sxm sglang 0.5.10 agg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM b300_sxm sglang 0.5.10 disagg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM b300_sxm sglang 0.5.9 agg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM b300_sxm sglang 0.5.9 disagg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM b300_sxm trtllm 1.3.0rc10 agg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM b300_sxm trtllm 1.3.0rc10 disagg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM gb200 sglang 0.5.10 agg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM gb200 sglang 0.5.10 disagg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM gb200 sglang 0.5.9 agg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM gb200 sglang 0.5.9 disagg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM gb200 trtllm 1.3.0rc10 agg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM gb200 trtllm 1.3.0rc10 disagg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM gb300 sglang 0.5.10 agg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM gb300 sglang 0.5.10 disagg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM gb300 sglang 0.5.9 agg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM gb300 sglang 0.5.9 disagg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM gb300 trtllm 1.3.0rc10 agg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM gb300 trtllm 1.3.0rc10 disagg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM rtx_pro_6000_server trtllm 1.3.0rc10 agg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NemotronHForCausalLM rtx_pro_6000_server trtllm 1.3.0rc10 disagg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 NemotronHForCausalLM b200_sxm sglang 0.5.10 agg PASS HYBRID_PASS
nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 NemotronHForCausalLM b200_sxm sglang 0.5.10 disagg PASS HYBRID_PASS
openai/gpt-oss-120b GptOssForCausalLM b200_sxm sglang 0.5.10 agg PASS HYBRID_PASS
openai/gpt-oss-120b GptOssForCausalLM b200_sxm sglang 0.5.10 disagg PASS HYBRID_PASS
openai/gpt-oss-120b GptOssForCausalLM b200_sxm vllm 0.19.0 agg PASS HYBRID_PASS
openai/gpt-oss-120b GptOssForCausalLM b200_sxm vllm 0.19.0 disagg PASS HYBRID_PASS
openai/gpt-oss-20b GptOssForCausalLM b200_sxm sglang 0.5.10 agg PASS HYBRID_PASS
openai/gpt-oss-20b GptOssForCausalLM b200_sxm sglang 0.5.10 disagg PASS HYBRID_PASS
openai/gpt-oss-20b GptOssForCausalLM b200_sxm vllm 0.19.0 agg PASS HYBRID_PASS
openai/gpt-oss-20b GptOssForCausalLM b200_sxm vllm 0.19.0 disagg PASS HYBRID_PASS

2. Hybrid regressions (HYBRID_PASS -> non-pass): 0 rows

No hybrid regressions

3. Silicon fixes (non-PASS -> PASS): 437 rows

HuggingFaceID Architecture System Backend Version Mode Previous Status New Status
MiniMaxAI/MiniMax-M2.5 MiniMaxM2ForCausalLM h100_sxm sglang 0.5.6.post2 agg FAIL PASS
MiniMaxAI/MiniMax-M2.5 MiniMaxM2ForCausalLM h100_sxm sglang 0.5.6.post2 disagg FAIL PASS
MiniMaxAI/MiniMax-M2.5 MiniMaxM2ForCausalLM h200_sxm sglang 0.5.6.post2 agg FAIL PASS
MiniMaxAI/MiniMax-M2.5 MiniMaxM2ForCausalLM h200_sxm sglang 0.5.6.post2 disagg FAIL PASS
MiniMaxAI/MiniMax-M2.5 MiniMaxM2ForCausalLM h200_sxm trtllm 1.2.0rc5 agg FAIL PASS
MiniMaxAI/MiniMax-M2.5 MiniMaxM2ForCausalLM h200_sxm trtllm 1.2.0rc5 disagg FAIL PASS
MiniMaxAI/MiniMax-M2.5 MiniMaxM2ForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
MiniMaxAI/MiniMax-M2.5 MiniMaxM2ForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
MiniMaxAI/MiniMax-M2.7 MiniMaxM2ForCausalLM h100_sxm sglang 0.5.6.post2 agg FAIL PASS
MiniMaxAI/MiniMax-M2.7 MiniMaxM2ForCausalLM h100_sxm sglang 0.5.6.post2 disagg FAIL PASS
MiniMaxAI/MiniMax-M2.7 MiniMaxM2ForCausalLM h200_sxm sglang 0.5.6.post2 agg FAIL PASS
MiniMaxAI/MiniMax-M2.7 MiniMaxM2ForCausalLM h200_sxm sglang 0.5.6.post2 disagg FAIL PASS
MiniMaxAI/MiniMax-M2.7 MiniMaxM2ForCausalLM h200_sxm trtllm 1.2.0rc5 agg FAIL PASS
MiniMaxAI/MiniMax-M2.7 MiniMaxM2ForCausalLM h200_sxm trtllm 1.2.0rc5 disagg FAIL PASS
MiniMaxAI/MiniMax-M2.7 MiniMaxM2ForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
MiniMaxAI/MiniMax-M2.7 MiniMaxM2ForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-0.6B Qwen3ForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-0.6B Qwen3ForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-1.7B Qwen3ForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-1.7B Qwen3ForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-235B-A22B Qwen3MoeForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-235B-A22B Qwen3MoeForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-235B-A22B-FP8 Qwen3MoeForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-235B-A22B-FP8 Qwen3MoeForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-30B-A3B Qwen3MoeForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-30B-A3B Qwen3MoeForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-30B-A3B Qwen3MoeForCausalLM l40s trtllm 1.0.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-30B-A3B Qwen3MoeForCausalLM l40s trtllm 1.0.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-30B-A3B-FP8 Qwen3MoeForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-30B-A3B-FP8 Qwen3MoeForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-32B Qwen3ForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-32B Qwen3ForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-32B-FP8 Qwen3ForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-32B-FP8 Qwen3ForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-32B-FP8-Static-PerTensor Qwen3ForCausalLM b200_sxm trtllm 1.2.0rc5 agg HW_INCOMPATIBLE PASS
Qwen/Qwen3-32B-FP8-Static-PerTensor Qwen3ForCausalLM b200_sxm trtllm 1.2.0rc5 disagg HW_INCOMPATIBLE PASS
Qwen/Qwen3-8B Qwen3ForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-8B Qwen3ForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-Coder-480B-A35B-Instruct Qwen3MoeForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-Coder-480B-A35B-Instruct Qwen3MoeForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-VL-235B-A22B-Instruct Qwen3VLMoeForConditionalGeneration h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-VL-235B-A22B-Instruct Qwen3VLMoeForConditionalGeneration h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-VL-2B-Instruct Qwen3VLForConditionalGeneration h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-VL-2B-Instruct Qwen3VLForConditionalGeneration h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-VL-30B-A3B-Instruct Qwen3VLMoeForConditionalGeneration h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-VL-30B-A3B-Instruct Qwen3VLMoeForConditionalGeneration h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-VL-32B-Instruct Qwen3VLForConditionalGeneration h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-VL-32B-Instruct Qwen3VLForConditionalGeneration h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-VL-32B-Thinking Qwen3VLForConditionalGeneration h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-VL-32B-Thinking Qwen3VLForConditionalGeneration h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-VL-4B-Instruct Qwen3VLForConditionalGeneration h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-VL-4B-Instruct Qwen3VLForConditionalGeneration h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-VL-8B-Instruct Qwen3VLForConditionalGeneration h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3-VL-8B-Instruct Qwen3VLForConditionalGeneration h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3.5-27B Qwen3_5ForConditionalGeneration b200_sxm trtllm 1.2.0rc5 agg HW_INCOMPATIBLE PASS
Qwen/Qwen3.5-27B Qwen3_5ForConditionalGeneration b200_sxm trtllm 1.2.0rc5 disagg HW_INCOMPATIBLE PASS
Qwen/Qwen3.5-27B Qwen3_5ForConditionalGeneration b300_sxm sglang 0.5.10 agg FAIL PASS
Qwen/Qwen3.5-27B Qwen3_5ForConditionalGeneration b300_sxm sglang 0.5.10 disagg FAIL PASS
Qwen/Qwen3.5-27B Qwen3_5ForConditionalGeneration gb200 sglang 0.5.10 agg FAIL PASS
Qwen/Qwen3.5-27B Qwen3_5ForConditionalGeneration gb200 sglang 0.5.10 disagg FAIL PASS
Qwen/Qwen3.5-27B Qwen3_5ForConditionalGeneration gb200 vllm 0.14.0 agg FAIL PASS
Qwen/Qwen3.5-27B Qwen3_5ForConditionalGeneration gb200 vllm 0.14.0 disagg FAIL PASS
Qwen/Qwen3.5-27B Qwen3_5ForConditionalGeneration gb300 sglang 0.5.10 agg FAIL PASS
Qwen/Qwen3.5-27B Qwen3_5ForConditionalGeneration gb300 sglang 0.5.10 disagg FAIL PASS
Qwen/Qwen3.5-27B Qwen3_5ForConditionalGeneration l40s trtllm 1.0.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3.5-27B Qwen3_5ForConditionalGeneration l40s trtllm 1.0.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3.5-27B Qwen3_5ForConditionalGeneration l40s vllm 0.14.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3.5-27B Qwen3_5ForConditionalGeneration l40s vllm 0.14.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3.5-27B Qwen3_5ForConditionalGeneration l40s vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3.5-35B-A3B Qwen3_5MoeForConditionalGeneration b200_sxm trtllm 1.2.0rc5 agg HW_INCOMPATIBLE PASS
Qwen/Qwen3.5-35B-A3B Qwen3_5MoeForConditionalGeneration b200_sxm trtllm 1.2.0rc5 disagg HW_INCOMPATIBLE PASS
Qwen/Qwen3.5-35B-A3B Qwen3_5MoeForConditionalGeneration l40s trtllm 1.0.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3.5-35B-A3B Qwen3_5MoeForConditionalGeneration l40s trtllm 1.0.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3.5-35B-A3B Qwen3_5MoeForConditionalGeneration l40s vllm 0.14.0 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3.5-35B-A3B Qwen3_5MoeForConditionalGeneration l40s vllm 0.14.0 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3.5-35B-A3B Qwen3_5MoeForConditionalGeneration rtx_pro_6000_server sglang 0.5.10 agg FAIL PASS
Qwen/Qwen3.5-35B-A3B Qwen3_5MoeForConditionalGeneration rtx_pro_6000_server sglang 0.5.10 disagg FAIL PASS
Qwen/Qwen3.5-35B-A3B Qwen3_5MoeForConditionalGeneration rtx_pro_6000_server sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3.5-35B-A3B Qwen3_5MoeForConditionalGeneration rtx_pro_6000_server sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
Qwen/Qwen3.5-397B-A17B Qwen3_5MoeForConditionalGeneration b200_sxm sglang 0.5.9 agg HW_INCOMPATIBLE PASS
Qwen/Qwen3.5-397B-A17B Qwen3_5MoeForConditionalGeneration b200_sxm sglang 0.5.9 disagg HW_INCOMPATIBLE PASS
Qwen/Qwen3.5-397B-A17B Qwen3_5MoeForConditionalGeneration b200_sxm trtllm 1.2.0rc5 agg HW_INCOMPATIBLE PASS
Qwen/Qwen3.5-397B-A17B Qwen3_5MoeForConditionalGeneration b200_sxm trtllm 1.2.0rc5 disagg HW_INCOMPATIBLE PASS
Qwen/Qwen3.5-397B-A17B Qwen3_5MoeForConditionalGeneration b300_sxm sglang 0.5.10 agg FAIL PASS
Qwen/Qwen3.5-397B-A17B Qwen3_5MoeForConditionalGeneration b300_sxm sglang 0.5.10 disagg FAIL PASS
Qwen/Qwen3.5-397B-A17B Qwen3_5MoeForConditionalGeneration gb200 sglang 0.5.10 agg FAIL PASS
Qwen/Qwen3.5-397B-A17B Qwen3_5MoeForConditionalGeneration gb200 sglang 0.5.10 disagg FAIL PASS
Qwen/Qwen3.5-397B-A17B Qwen3_5MoeForConditionalGeneration gb200 sglang 0.5.9 agg FAIL PASS
Qwen/Qwen3.5-397B-A17B Qwen3_5MoeForConditionalGeneration gb200 sglang 0.5.9 disagg FAIL PASS
Qwen/Qwen3.5-397B-A17B Qwen3_5MoeForConditionalGeneration gb200 vllm 0.14.0 agg FAIL PASS
Qwen/Qwen3.5-397B-A17B Qwen3_5MoeForConditionalGeneration gb200 vllm 0.14.0 disagg FAIL PASS
Qwen/Qwen3.5-397B-A17B Qwen3_5MoeForConditionalGeneration gb300 sglang 0.5.10 agg FAIL PASS
Qwen/Qwen3.5-397B-A17B Qwen3_5MoeForConditionalGeneration gb300 sglang 0.5.10 disagg FAIL PASS
Qwen/Qwen3.5-397B-A17B Qwen3_5MoeForConditionalGeneration gb300 sglang 0.5.9 agg FAIL PASS
Qwen/Qwen3.5-397B-A17B Qwen3_5MoeForConditionalGeneration gb300 sglang 0.5.9 disagg FAIL PASS
XiaomiMiMo/MiMo-7B-Base MiMoForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
XiaomiMiMo/MiMo-7B-Base MiMoForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM b200_sxm sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM b200_sxm sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM b200_sxm sglang 0.5.9 agg HW_INCOMPATIBLE PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM b200_sxm sglang 0.5.9 disagg HW_INCOMPATIBLE PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM b300_sxm sglang 0.5.10 agg FAIL PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM b300_sxm sglang 0.5.10 disagg FAIL PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM b300_sxm sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM b300_sxm sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM b300_sxm sglang 0.5.9 agg FAIL PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM b300_sxm sglang 0.5.9 disagg FAIL PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM gb200 sglang 0.5.10 agg FAIL PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM gb200 sglang 0.5.10 disagg FAIL PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM gb200 sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM gb200 sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM gb200 sglang 0.5.9 agg FAIL PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM gb200 sglang 0.5.9 disagg FAIL PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM gb300 sglang 0.5.10 agg FAIL PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM gb300 sglang 0.5.10 disagg FAIL PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM gb300 sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM gb300 sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM gb300 sglang 0.5.9 agg FAIL PASS
XiaomiMiMo/MiMo-V2-Flash MiMoV2FlashForCausalLM gb300 sglang 0.5.9 disagg FAIL PASS
deepseek-ai/DeepSeek-R1 DeepseekV3ForCausalLM b200_sxm trtllm 1.2.0rc5 agg HW_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-R1 DeepseekV3ForCausalLM b200_sxm trtllm 1.2.0rc5 disagg HW_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-R1 DeepseekV3ForCausalLM b200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-R1 DeepseekV3ForCausalLM b200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-R1 DeepseekV3ForCausalLM b300_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-R1 DeepseekV3ForCausalLM b300_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-R1 DeepseekV3ForCausalLM gb200 vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-R1 DeepseekV3ForCausalLM gb200 vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-R1 DeepseekV3ForCausalLM gb300 vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-R1 DeepseekV3ForCausalLM gb300 vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-R1 DeepseekV3ForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-R1 DeepseekV3ForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-R1 DeepseekV3ForCausalLM rtx_pro_6000_server vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-R1 DeepseekV3ForCausalLM rtx_pro_6000_server vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3 DeepseekV3ForCausalLM b200_sxm trtllm 1.2.0rc5 agg HW_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3 DeepseekV3ForCausalLM b200_sxm trtllm 1.2.0rc5 disagg HW_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3 DeepseekV3ForCausalLM b200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3 DeepseekV3ForCausalLM b200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3 DeepseekV3ForCausalLM b300_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3 DeepseekV3ForCausalLM b300_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3 DeepseekV3ForCausalLM gb200 vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3 DeepseekV3ForCausalLM gb200 vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3 DeepseekV3ForCausalLM gb300 vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3 DeepseekV3ForCausalLM gb300 vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3 DeepseekV3ForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3 DeepseekV3ForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3 DeepseekV3ForCausalLM rtx_pro_6000_server vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3 DeepseekV3ForCausalLM rtx_pro_6000_server vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM b200_sxm sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM b200_sxm sglang 0.5.9 agg HW_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM b200_sxm sglang 0.5.9 disagg HW_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM b200_sxm trtllm 1.2.0rc5 agg HW_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM b200_sxm trtllm 1.2.0rc5 disagg HW_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM b200_sxm trtllm 1.3.0rc10 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM b200_sxm trtllm 1.3.0rc10 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM b300_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM b300_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM gb200 vllm 0.14.0 agg FAIL PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM gb200 vllm 0.14.0 disagg FAIL PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM gb200 vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM gb200 vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM gb300 vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM gb300 vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM h200_sxm sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM h200_sxm sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM h200_sxm sglang 0.5.6.post2 agg FAIL PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM h200_sxm sglang 0.5.6.post2 disagg FAIL PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM h200_sxm sglang 0.5.9 agg FAIL PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM h200_sxm sglang 0.5.9 disagg FAIL PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM h200_sxm trtllm 1.2.0rc5 agg FAIL PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM h200_sxm trtllm 1.2.0rc5 disagg FAIL PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM h200_sxm trtllm 1.3.0rc10 agg FAIL PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM h200_sxm trtllm 1.3.0rc10 disagg FAIL PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM h200_sxm trtllm 1.3.0rc15 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM h200_sxm trtllm 1.3.0rc15 disagg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM h200_sxm vllm 0.14.0 agg FAIL PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM h200_sxm vllm 0.14.0 disagg FAIL PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
deepseek-ai/DeepSeek-V3.2 DeepseekV32ForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration b200_sxm sglang 0.5.10 agg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration b200_sxm sglang 0.5.10 disagg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration b200_sxm sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration b200_sxm sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration b200_sxm sglang 0.5.9 agg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration b200_sxm sglang 0.5.9 disagg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration b300_sxm sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration b300_sxm sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration gb200 sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration gb200 sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration gb300 sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration gb300 sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration h100_sxm sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration h100_sxm sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration h200_sxm sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration h200_sxm sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration rtx_pro_6000_server sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
google/gemma-4-26B-A4B Gemma4ForConditionalGeneration rtx_pro_6000_server sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration b200_sxm sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration b200_sxm sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration b200_sxm sglang 0.5.9 agg HW_INCOMPATIBLE PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration b200_sxm sglang 0.5.9 disagg HW_INCOMPATIBLE PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration b200_sxm trtllm 1.2.0rc5 agg HW_INCOMPATIBLE PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration b200_sxm trtllm 1.2.0rc5 disagg HW_INCOMPATIBLE PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration b300_sxm sglang 0.5.10 agg FAIL PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration b300_sxm sglang 0.5.10 disagg FAIL PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration b300_sxm sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration b300_sxm sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration b300_sxm sglang 0.5.9 agg FAIL PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration b300_sxm sglang 0.5.9 disagg FAIL PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration gb200 sglang 0.5.10 agg FAIL PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration gb200 sglang 0.5.10 disagg FAIL PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration gb200 sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration gb200 sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration gb200 sglang 0.5.9 agg FAIL PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration gb200 sglang 0.5.9 disagg FAIL PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration gb300 sglang 0.5.10 agg FAIL PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration gb300 sglang 0.5.10 disagg FAIL PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration gb300 sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration gb300 sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration gb300 sglang 0.5.9 agg FAIL PASS
meta-llama/Llama-4-Maverick-17B-128E-Instruct Llama4ForConditionalGeneration gb300 sglang 0.5.9 disagg FAIL PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration b200_sxm sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration b200_sxm sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration b200_sxm sglang 0.5.9 agg HW_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration b200_sxm sglang 0.5.9 disagg HW_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration b200_sxm trtllm 1.2.0rc5 agg HW_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration b200_sxm trtllm 1.2.0rc5 disagg HW_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration b300_sxm sglang 0.5.10 agg FAIL PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration b300_sxm sglang 0.5.10 disagg FAIL PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration b300_sxm sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration b300_sxm sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration b300_sxm sglang 0.5.9 agg FAIL PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration b300_sxm sglang 0.5.9 disagg FAIL PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration gb200 sglang 0.5.10 agg FAIL PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration gb200 sglang 0.5.10 disagg FAIL PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration gb200 sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration gb200 sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration gb200 sglang 0.5.9 agg FAIL PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration gb200 sglang 0.5.9 disagg FAIL PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration gb300 sglang 0.5.10 agg FAIL PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration gb300 sglang 0.5.10 disagg FAIL PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration gb300 sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration gb300 sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration gb300 sglang 0.5.9 agg FAIL PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration gb300 sglang 0.5.9 disagg FAIL PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration l40s sglang 0.5.10 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration l40s sglang 0.5.10 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration l40s sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration l40s sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration l40s sglang 0.5.9 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration l40s sglang 0.5.9 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration l40s trtllm 1.0.0 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration l40s trtllm 1.0.0 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration l40s trtllm 1.3.0rc15 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration l40s trtllm 1.3.0rc15 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration l40s vllm 0.14.0 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration l40s vllm 0.14.0 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration l40s vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration l40s vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration rtx_pro_6000_server sglang 0.5.10 agg FAIL PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration rtx_pro_6000_server sglang 0.5.10 disagg FAIL PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration rtx_pro_6000_server sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Llama-4-Scout-17B-16E-Instruct Llama4ForConditionalGeneration rtx_pro_6000_server sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Meta-Llama-3.1-405B LlamaForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Meta-Llama-3.1-405B LlamaForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Meta-Llama-3.1-70B LlamaForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Meta-Llama-3.1-70B LlamaForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Meta-Llama-3.1-8B LlamaForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
meta-llama/Meta-Llama-3.1-8B LlamaForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
moonshotai/Kimi-K2.5 KimiK25ForConditionalGeneration b300_sxm sglang 0.5.9 agg FAIL PASS
moonshotai/Kimi-K2.5 KimiK25ForConditionalGeneration b300_sxm sglang 0.5.9 disagg FAIL PASS
moonshotai/Kimi-K2.5 KimiK25ForConditionalGeneration h100_sxm sglang 0.5.6.post2 agg FAIL PASS
moonshotai/Kimi-K2.5 KimiK25ForConditionalGeneration h100_sxm sglang 0.5.6.post2 disagg FAIL PASS
moonshotai/Kimi-K2.5 KimiK25ForConditionalGeneration h200_sxm sglang 0.5.6.post2 agg FAIL PASS
moonshotai/Kimi-K2.5 KimiK25ForConditionalGeneration h200_sxm sglang 0.5.6.post2 disagg FAIL PASS
moonshotai/Kimi-K2.5 KimiK25ForConditionalGeneration h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
moonshotai/Kimi-K2.5 KimiK25ForConditionalGeneration h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
nvidia/DeepSeek-V3.1-NVFP4 DeepseekV3ForCausalLM b200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/DeepSeek-V3.1-NVFP4 DeepseekV3ForCausalLM b200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
nvidia/DeepSeek-V3.1-NVFP4 DeepseekV3ForCausalLM b300_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/DeepSeek-V3.1-NVFP4 DeepseekV3ForCausalLM b300_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
nvidia/DeepSeek-V3.1-NVFP4 DeepseekV3ForCausalLM gb200 vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/DeepSeek-V3.1-NVFP4 DeepseekV3ForCausalLM gb200 vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
nvidia/DeepSeek-V3.1-NVFP4 DeepseekV3ForCausalLM gb300 vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/DeepSeek-V3.1-NVFP4 DeepseekV3ForCausalLM gb300 vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM b200_sxm sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM b200_sxm sglang 0.5.9 agg HW_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM b200_sxm sglang 0.5.9 disagg HW_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM b200_sxm trtllm 1.2.0rc5 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM b200_sxm trtllm 1.2.0rc5 disagg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM b200_sxm trtllm 1.3.0rc10 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM b200_sxm trtllm 1.3.0rc10 disagg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM b300_sxm sglang 0.5.10 agg FAIL PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM b300_sxm sglang 0.5.10 disagg FAIL PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM b300_sxm sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM b300_sxm sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM b300_sxm trtllm 1.3.0rc15 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM b300_sxm trtllm 1.3.0rc15 disagg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM b300_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM b300_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM gb200 trtllm 1.3.0rc15 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM gb200 trtllm 1.3.0rc15 disagg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM gb200 vllm 0.14.0 agg FAIL PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM gb200 vllm 0.14.0 disagg FAIL PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM gb200 vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM gb200 vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM gb300 trtllm 1.3.0rc15 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM gb300 trtllm 1.3.0rc15 disagg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM gb300 vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/GLM-5-NVFP4 GlmMoeDsaForCausalLM gb300 vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
nvidia/Llama-3_3-Nemotron-Super-49B-v1 DeciLMForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/Llama-3_3-Nemotron-Super-49B-v1 DeciLMForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 NemotronHForCausalLM h100_sxm sglang 0.5.6.post2 agg FAIL PASS
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 NemotronHForCausalLM h100_sxm sglang 0.5.6.post2 disagg FAIL PASS
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 NemotronHForCausalLM h200_sxm sglang 0.5.6.post2 agg FAIL PASS
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 NemotronHForCausalLM h200_sxm sglang 0.5.6.post2 disagg FAIL PASS
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 NemotronHForCausalLM h200_sxm trtllm 1.2.0rc5 agg FAIL PASS
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 NemotronHForCausalLM h200_sxm trtllm 1.2.0rc5 disagg FAIL PASS
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 NemotronHForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 NemotronHForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 NemotronHForCausalLM l40s trtllm 1.0.0 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 NemotronHForCausalLM l40s trtllm 1.0.0 disagg FRAMEWORK_INCOMPATIBLE PASS
nvidia/Nemotron-H-56B-Base-8K NemotronHForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
nvidia/Nemotron-H-56B-Base-8K NemotronHForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
openai/gpt-oss-120b GptOssForCausalLM b200_sxm trtllm 1.2.0rc5 agg HW_INCOMPATIBLE PASS
openai/gpt-oss-120b GptOssForCausalLM b200_sxm trtllm 1.2.0rc5 disagg HW_INCOMPATIBLE PASS
openai/gpt-oss-120b GptOssForCausalLM h100_sxm trtllm 1.3.0rc10 agg FAIL PASS
openai/gpt-oss-120b GptOssForCausalLM h100_sxm trtllm 1.3.0rc10 disagg FAIL PASS
openai/gpt-oss-120b GptOssForCausalLM h200_sxm vllm 0.14.0 agg FAIL PASS
openai/gpt-oss-120b GptOssForCausalLM h200_sxm vllm 0.14.0 disagg FAIL PASS
openai/gpt-oss-120b GptOssForCausalLM h200_sxm vllm 0.22.0 agg FRAMEWORK_INCOMPATIBLE PASS
openai/gpt-oss-120b GptOssForCausalLM h200_sxm vllm 0.22.0 disagg FRAMEWORK_INCOMPATIBLE PASS
openai/gpt-oss-120b GptOssForCausalLM l40s sglang 0.5.10 agg FRAMEWORK_INCOMPATIBLE PASS
openai/gpt-oss-120b GptOssForCausalLM l40s sglang 0.5.10 disagg FRAMEWORK_INCOMPATIBLE PASS
openai/gpt-oss-120b GptOssForCausalLM l40s sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
openai/gpt-oss-120b GptOssForCausalLM l40s sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
openai/gpt-oss-120b GptOssForCausalLM l40s sglang 0.5.9 agg FRAMEWORK_INCOMPATIBLE PASS
openai/gpt-oss-120b GptOssForCausalLM l40s sglang 0.5.9 disagg FRAMEWORK_INCOMPATIBLE PASS
openai/gpt-oss-120b GptOssForCausalLM rtx_pro_6000_server sglang 0.5.10 agg FAIL PASS
openai/gpt-oss-120b GptOssForCausalLM rtx_pro_6000_server sglang 0.5.10 disagg FAIL PASS
openai/gpt-oss-120b GptOssForCausalLM rtx_pro_6000_server sglang 0.5.12 agg FRAMEWORK_INCOMPATIBLE PASS
openai/gpt-oss-120b GptOssForCausalLM rtx_pro_6000_server sglang 0.5.12 disagg FRAMEWORK_INCOMPATIBLE PASS
openai/gpt-oss-20b GptOssForCausalLM b200_sxm trtllm 1.2.0rc5 agg HW_INCOMPATIBLE PASS
openai/gpt-oss-20b GptOssForCausalLM b200_sxm trtllm 1.2.0rc5 disagg HW_INCOMPATIBLE PASS
openai/gpt-oss-20b GptOssForCausalLM h200_sxm vllm 0.14.0 agg FAIL PASS
openai/gpt-oss-20b GptOssForCausalLM h200_sxm vllm 0.14.0 disagg

(diff tables truncated to fit the PR body limit; full diff.json available on request)

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Added collection coverage for the MiniMax M3 language model.
    • Enabled base-operation analysis for MiniMax M3 using the MiniMaxAI/MiniMax-M3 model.
  • Known Limitations
    • Coverage currently excludes operations specific to the MSA attention module.
  • Bug Fixes
    • Updated compatibility/support detection expectations for zai-org/GLM-5-FP8 for specific sglang and TensorRT-LLM versions.

@tianhaox
tianhaox requested review from a team as code owners July 15, 2026 07:21
@copy-pr-bot

copy-pr-bot Bot commented Jul 15, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the fix label Jul 15, 2026
@coderabbitai

coderabbitai Bot commented Jul 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2095f3a4-7357-48f6-a3c9-7a7a296e4535

📥 Commits

Reviewing files that changed from the base of the PR and between 4ffe034 and d50bb17.

📒 Files selected for processing (1)
  • tests/unit/sdk/test_common.py
📜 Recent review details
⏰ Context from checks skipped due to timeout. (7)
  • GitHub Check: Collect snapshot (new)
  • GitHub Check: Collect snapshot (old)
  • GitHub Check: Build and Test (unit)
  • GitHub Check: Build and Test (e2e)
  • GitHub Check: Rust/Python engine-step parity
  • GitHub Check: create-charts
  • GitHub Check: Cargo Deny
🧰 Additional context used
📓 Path-based instructions (5)
tests/**/*.{py,yaml,txt,sh}

📄 CodeRabbit inference engine (.claude/rules/generator/testing.md)

Use integration tests for the full input-to-artifacts pipeline, comparing output against golden snapshots without external dependencies.

Files:

  • tests/unit/sdk/test_common.py
**/*

📄 CodeRabbit inference engine (.claude/rules/repo-guide.md)

**/*: Treat this file as the only always-injected rule file; do not add new always-on rules here without human approval. New rule files must include paths: frontmatter.
When reviewing changes in a governed area, read that area's rule files first, even if path-based auto-loading does not occur during a read-only review. Rule violations are review findings even when the code works.
A task must remain within its module: collector tasks may change only collector/ and its tests, generator tasks only src/aiconfigurator/generator/ and its tests, and SDK tasks must not modify either. Cross-module contract changes require explicit human approval.

Files:

  • tests/unit/sdk/test_common.py

⚙️ CodeRabbit configuration file

**/*: - Prefer applicable inline comments. When the correct fix is clear, small, and limited to the commented diff hunk, include it as a GitHub Suggested Change so the author can apply it with one click.

  • Do not use a suggested change when the fix requires broader design choices, multiple files, generated artifacts, unavailable context, or validation that cannot be inferred from the diff.
  • If a comment is not directly applicable, state the smallest concrete next step and why a one-click suggestion is not safe.

Files:

  • tests/unit/sdk/test_common.py
**/*.{py,pyi}

📄 CodeRabbit inference engine (AGENTS.md)

Run ruff check . and ruff format --check . to validate Python linting and formatting.

Files:

  • tests/unit/sdk/test_common.py
tests/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

Run unit tests with pytest -m unit; use pytest -m "unit or build" for the build-test subset when required data is available.

Files:

  • tests/unit/sdk/test_common.py
tests/**

⚙️ CodeRabbit configuration file

tests/**: - Check that tests cover the changed behavior rather than only the happy path.

  • Watch for fixtures or golden outputs that mask backend drift, support-matrix ordering changes, or CLI output regressions.

Files:

  • tests/unit/sdk/test_common.py
🔇 Additional comments (1)
tests/unit/sdk/test_common.py (1)

286-287: 📐 Maintainability & Code Quality

Run pytest -m unit in an environment with pytest installed.

Walkthrough

Adds a YAML collection plan for MiniMaxM3ForCausalLM and updates GLM-5-FP8 support expectations for two runtime versions.

Changes

Model collection and support expectations

Layer / File(s) Summary
Model collection configuration
collector/cases/models/MiniMaxM3ForCausalLM_cases.yaml		 Defines the schema version, architecture, model path, base-operation inclusion, and current MSA attention coverage description.
GLM-5-FP8 support expectations
tests/unit/sdk/test_common.py		 Changes aggregate and disaggregate support expectations to True for sglang 0.5.9 and trtllm 1.3.0rc10.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Poem

MiniMax joins the model parade,
Base operations are displayed.
GLM support turns green and bright,
Two runtimes pass the test tonight.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: refreshing the support matrix.
Description check ✅ Passed The description is detailed and covers the regeneration, validation notes, and key results, though it doesn't use the exact template headings.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@tianhaox @claude
fix: update support matrix
dfadb26 
Full local regeneration of the support matrix on main (2feebac) using
the CI docker image (docker/Dockerfile --target test), 16 workers.
10502 configurations across 5251 model/system/backend/version combos.

Rebased onto main after ai-dynamo#1322 relocated the matrix to
aic-core/src/aiconfigurator_core/systems/support_matrix/.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: tianhaox <tianhaox@nvidia.com>
@tianhaox
tianhaox force-pushed the fix/update-support-matrix-main-local branch from f868838 to dfadb26 Compare July 16, 2026 09:22
@tianhaox @claude
fix: register MiniMax-M3 collector model-case alias
4ffe034 
The support matrix now contains MiniMaxAI/MiniMax-M3 rows (SDK support
landed in ai-dynamo#1238), but no collector case file existed for the
MiniMaxM3ForCausalLM architecture, failing
test_support_matrix_models_have_model_case_aliases. Register the model
with base-op coverage only; the MSA attention op is not authored yet.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: tianhaox <tianhaox@nvidia.com>
@tianhaox

tianhaox commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

Unit-test fix: the regenerated matrix now contains MiniMaxAI/MiniMax-M3 (SDK support from #1238), which exposed a missing collector model-case registration — test_support_matrix_models_have_model_case_aliases requires every matrix model to be declared in collector/cases/models/. Added a minimal MiniMaxM3ForCausalLM_cases.yaml (base-op coverage only, per the DeciLM pattern); authoring the MSA attention op and model shapes remains a follow-up collector task — until then M3 rows stay HYBRID_PASS/estimated. Full tests/unit/collector suite passes locally (493 passed, 5 skipped).

@tianhaox @claude
test: refresh GLM-5-FP8 b200 support expectations to regenerated matrix
d50bb17 
sglang 0.5.9 and trtllm 1.3.0rc10 rows changed to PASS (Source=silicon)
in the regenerated matrix via shared-layer reuse of newly collected
sibling-version data, so check_support now returns True for them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: tianhaox <tianhaox@nvidia.com>
@tianhaox
tianhaox requested review from a team as code owners July 16, 2026 16:43
@tianhaox
tianhaox merged commit 92b4ce8 into ai-dynamo:main Jul 17, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Arsene12358 Arsene12358 Arsene12358 approved these changes

Assignees

No one assigned

Labels

automated fix support-matrix

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@tianhaox @Arsene12358