Skip to content

Add GB300 vLLM mHC perf data#1349

Draft
vvagias wants to merge 1 commit into
mainfrom
vvagias/gb300-vllm-mhc-data
Draft

Add GB300 vLLM mHC perf data#1349
vvagias wants to merge 1 commit into
mainfrom
vvagias/gb300-vllm-mhc-data

Conversation

@vvagias

@vvagias vvagias commented Jul 13, 2026

Copy link
Copy Markdown

Summary

Adds measured GB300 silicon data for vLLM 0.20.1 mHC module lookup:

  • src/aiconfigurator/systems/data/gb300/vllm/0.20.1/mhc_module_perf.parquet
  • src/aiconfigurator/systems/data/gb300/vllm/0.20.1/SHARED_LAYER_REUSE.txt

The data was collected on a GB300 node with:

  • image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.2.0-deepseek-v4-cuda13-dev.3
  • vLLM: 0.20.1
  • collector: python3 collector/collect.py --backend vllm --ops mhc_module --gpu gb300 --keep-csv
  • run id: 20260713-152245

Validation

  • 140 parquet rows loaded successfully.
  • Device: NVIDIA GB300.
  • Framework/version: VLLM / 0.20.1.
  • Ops: pre, post.
  • Hidden sizes: 4096, 7168.
  • Token sweep: 35 token counts from 1 through 524288.
  • Latency range: 0.0084 to 21.7754 ms.
  • PerfDatabase("gb300", "vllm", "0.20.1") resolves the new mHC rows in SILICON mode.
  • Sample query for num_tokens=8192, hidden_size=4096, hc_mult=4, sinkhorn_iters=20:
    • pre: 0.2412 ms, source silicon
    • post: 0.1824 ms, source silicon
    • both: 0.4236 ms, source silicon

Notes

This unblocks GB300/vLLM 0.20.1 mHC lookup, but full DeepSeek-V4 vLLM SILICON runs still need additional GB300/vLLM 0.20.1 DeepSeek-V4 context attention module data. The default DeepSeek-V4 Flash model path also currently hits an unsupported MoE quant mode for w4a8_mxfp4_mxfp8.

Summary by CodeRabbit

  • Documentation
    • Added support documentation for GB300 with vLLM 0.20.1 shared-layer reuse.
    • Documented how compatible silicon data may be inherited when matching shapes are unavailable.

@copy-pr-bot

copy-pr-bot Bot commented Jul 13, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

An error occurred during the review process. Please try again later.

Walkthrough

Adds a GB300 vLLM 0.20.1 declaration documenting shared-layer reuse support, measured silicon tables, and compatible-shape inheritance through op_kernel_source_manifest.yaml.

Changes

GB300 Shared Layer Reuse

Layer / File(s) Summary
Shared-layer reuse documentation
src/aiconfigurator/systems/data/gb300/vllm/0.20.1/SHARED_LAYER_REUSE.txt
Declares GB300 shared-layer reuse support for vLLM 0.20.1 and documents measured tables plus sibling-row shape inheritance.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Poem

A shared layer joins the stream,
GB300 tables mark the dream.
Shapes may borrow, neatly aligned,
From sibling rows already mined.
One small note, the path is clear.


Comment @coderabbitai help to get the list of available commands.

@tianhaox

Copy link
Copy Markdown
Contributor

could you check the latest main with this PR merged recently #1344 is this working for your case

@github-actions

Copy link
Copy Markdown
Contributor

Sanity Check Chart Generation Report

New perf data files were detected in this PR. Please use the link above to
download sanity check charts for the new perf data to compare the collected
perf data vs SOL (theoretical max performance).

Below is a report of whether the chart generation was successful for each op.
If doesn't validate whether the perf data itself is sane.

Chart Generation Report for system: gb300, backend: vllm, backend_version: 0.20.1

  • CLI smoke test Skipped ⚠️: required default-model perf files are not present for this backend version: gemm_perf.parquet, context_attention_perf.parquet, generation_attention_perf.parquet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants