Add GB300 vLLM mHC perf data#1349
Conversation
|
Caution Review failedAn error occurred during the review process. Please try again later. WalkthroughAdds a GB300 vLLM 0.20.1 declaration documenting shared-layer reuse support, measured silicon tables, and compatible-shape inheritance through ChangesGB300 Shared Layer Reuse
Estimated code review effort: 1 (Trivial) | ~2 minutes Poem
Comment |
|
could you check the latest main with this PR merged recently #1344 is this working for your case |
Sanity Check Chart Generation ReportNew perf data files were detected in this PR. Please use the link above to Below is a report of whether the chart generation was successful for each op. Chart Generation Report for system: gb300, backend: vllm, backend_version: 0.20.1
|
Summary
Adds measured GB300 silicon data for vLLM 0.20.1 mHC module lookup:
src/aiconfigurator/systems/data/gb300/vllm/0.20.1/mhc_module_perf.parquetsrc/aiconfigurator/systems/data/gb300/vllm/0.20.1/SHARED_LAYER_REUSE.txtThe data was collected on a GB300 node with:
nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.2.0-deepseek-v4-cuda13-dev.30.20.1python3 collector/collect.py --backend vllm --ops mhc_module --gpu gb300 --keep-csv20260713-152245Validation
NVIDIA GB300.VLLM/0.20.1.pre,post.4096,7168.1through524288.0.0084to21.7754ms.PerfDatabase("gb300", "vllm", "0.20.1")resolves the new mHC rows inSILICONmode.num_tokens=8192,hidden_size=4096,hc_mult=4,sinkhorn_iters=20:0.2412ms, sourcesilicon0.1824ms, sourcesilicon0.4236ms, sourcesiliconNotes
This unblocks GB300/vLLM 0.20.1 mHC lookup, but full DeepSeek-V4 vLLM SILICON runs still need additional GB300/vLLM 0.20.1 DeepSeek-V4 context attention module data. The default DeepSeek-V4 Flash model path also currently hits an unsupported MoE quant mode for
w4a8_mxfp4_mxfp8.Summary by CodeRabbit