What coverage are you requesting?
The DGDR/Sweeper comparison suite exposes four reproducible AIConfigurator performance-data gaps. Legal, KV-feasible configurations are found, but Replay cannot evaluate any sampled candidate for the affected path.
| Model |
Hardware |
Backend / version |
Mode |
Missing data |
deepseek-ai/DeepSeek-R1 |
h200_sxm |
SGLang 0.5.14 |
disaggregated |
Context MLA for fmha_quant=fp8, kv_quant=fp8 |
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 |
h200_sxm |
SGLang 0.5.14 |
aggregated and disaggregated |
FP8 MoE data for the model's 512-expert shape |
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 |
h200_sxm |
vLLM 0.24.0 |
aggregated |
FP8 MoE data for the model's 512-expert shape |
Qwen/Qwen3-32B-FP8 |
a100_sxm |
vLLM 0.14.0 |
disaggregated |
fp8_block GEMM data; only BF16 is available |
These correspond to six cases because DeepSeek is tested with two GPU budgets and Nemotron/SGLang in both deployment modes.
Reproduction and actual errors
The failures reproduce with AIConfigurator and AIConfigurator Core 0.11.0, using the native RustForwardPassPerfModel through Dynamo Replay:
DeepSeek-R1 / H200 / SGLang:
context MLA data missing for ContextKey { fmha_quant: "fp8", kv_quant: "fp8" }
Nemotron-3 Super / H200 / SGLang:
MoE data missing for MoeKey { quant: "fp8", distribution: "uniform",
topk: 22, num_experts: 512, hidden_size: 1024, inter_size: 2688,
moe_tp_size: 1, moe_ep_size: 2 }
Nemotron-3 Super / H200 / vLLM:
MoE data missing for MoeKey { quant: "fp8", distribution: "power_law_1.01",
topk: 22, num_experts: 512, hidden_size: 1024, inter_size: 2688,
moe_tp_size: 1, moe_ep_size: 2 }
Qwen3-32B-FP8 / A100 / vLLM:
GEMM perf data missing for quant 'fp8_block'; available: ["bfloat16"]
Expected behavior
Either:
- package performance data that covers these configurations; or
- reject them during a readiness/preflight check with a typed unsupported-coverage result, before a Sweeper search repeatedly evaluates candidates that cannot be predicted.
Additional context
- #850 registered DeepSeek-R1 but did not add the missing H200/SGLang FP8 Context-MLA data.
- #1522 fixed the analogous Nemotron FP8 MoE gap for H100/vLLM; H200 vLLM and SGLang remain affected.
- #1414 covers fp8-block data quality, but not the absence of A100/vLLM rows reported here.
- The originating cases are in ai-dynamo/dynamo#13765, based on ai-dynamo/dynamo#8469.
What coverage are you requesting?
The DGDR/Sweeper comparison suite exposes four reproducible AIConfigurator performance-data gaps. Legal, KV-feasible configurations are found, but Replay cannot evaluate any sampled candidate for the affected path.
deepseek-ai/DeepSeek-R1h200_sxm0.5.14fmha_quant=fp8,kv_quant=fp8nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8h200_sxm0.5.14nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8h200_sxm0.24.0Qwen/Qwen3-32B-FP8a100_sxm0.14.0fp8_blockGEMM data; only BF16 is availableThese correspond to six cases because DeepSeek is tested with two GPU budgets and Nemotron/SGLang in both deployment modes.
Reproduction and actual errors
The failures reproduce with AIConfigurator and AIConfigurator Core
0.11.0, using the nativeRustForwardPassPerfModelthrough Dynamo Replay:Expected behavior
Either:
Additional context