This repository was archived by the owner on Apr 20, 2026. It is now read-only.
recipes: consolidate B200 recipes and fix config correctness#203
Open
weireweire wants to merge 2 commits into
Open
recipes: consolidate B200 recipes and fix config correctness#203weireweire wants to merge 2 commits into
weireweire wants to merge 2 commits into
Conversation
Contributor
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. Run ID: 📒 Files selected for processing (9)
📝 WalkthroughWalkthroughThis PR adds eight new recipe YAML configuration files for B200 GPU deployments across FP4 and FP8 precisions with 1k1k and 8k1k scales in MTP/STP variants, and updates the config override logic to preserve explicit "name" fields in override configurations rather than auto-generating them. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@recipes/b200-fp4/8k1k-mtp.yaml`:
- Around line 16-229: The CI failure is due to validating this file as a direct
root-schema recipe while it contains nested recipe fragments (keys like base,
override_tp4, zip_override_lowlat, override_maxtpt_4p1d, override_maxtpt_7p2d)
instead of a fully expanded root-schema document; fix by either (A) changing the
CI validator to run on the generated/expanded configs (update the validation
step that reads/validates recipe fragments to call the expansion logic before
schema validation), or (B) committing the expanded/flattened recipe(s) that
conform to the root schema (replace the base/override_* fragments with the final
merged root fields) so keys such as base and the override_* sections are removed
and the file matches the expected root fields.
In `@recipes/b200-fp4/8k1k-stp.yaml`:
- Around line 14-189: The recipe uses a top-level "base" plus override blocks
(base, override_tp4, zip_override_lowlat, override_maxtpt_7p2d) which breaks the
validator that requires root keys name, model, and resources; fix by committing
the fully expanded/merged recipe(s) so each committed YAML has the required
top-level keys (name, model, resources, backend, etc.) instead of nested
"base"/"override_*" blocks, or update CI to run the override expansion step
before validation (i.e., ensure the validator sees the generated/expanded output
rather than the templated file).
ℹ️ Review info
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 6816e256-f642-486e-b790-4647558a1de9
📒 Files selected for processing (10)
recipes/b200-fp4/1k1k-mtp.yamlrecipes/b200-fp4/1k1k-stp.yamlrecipes/b200-fp4/8k1k-mtp.yamlrecipes/b200-fp4/8k1k-stp.yamlrecipes/b200-fp8/1k1k-mtp.yamlrecipes/b200-fp8/1k1k-stp.yamlrecipes/b200-fp8/8k1k-mtp.yamlrecipes/b200-fp8/8k1k-stp.yamlsrc/srtctl/core/config.pytests/test_override.py
Comment on lines
+16
to
+229
| base: | ||
| name: "b200-fp4-mtp-8k1k" | ||
|
|
||
| model: | ||
| path: "dsr1" | ||
| container: "dynamo-sglang" | ||
| precision: "fp4" | ||
|
|
||
| frontend: | ||
| num_additional_frontends: 4 | ||
|
|
||
| resources: | ||
| gpu_type: "b200" | ||
| prefill_nodes: 1 | ||
| prefill_workers: 1 | ||
| gpus_per_prefill: 4 | ||
| decode_nodes: 1 | ||
| decode_workers: 1 | ||
| gpus_per_node: 8 | ||
|
|
||
| backend: | ||
| prefill_environment: | ||
| TORCH_DISTRIBUTED_DEFAULT_TIMEOUT: "1800" | ||
| PYTHONUNBUFFERED: "1" | ||
| DYN_SKIP_SGLANG_LOG_FORMATTING: "1" | ||
| SGLANG_ENABLE_JIT_DEEPGEMM: "false" | ||
| SGLANG_DISAGGREGATION_HEARTBEAT_MAX_FAILURE: "100000" | ||
| SGLANG_DISAGGREGATION_BOOTSTRAP_TIMEOUT: "100000" | ||
| SGLANG_DISAGGREGATION_WAITING_TIMEOUT: "100000" | ||
| SGLANG_MOONCAKE_CUSTOM_MEM_POOL: "True" | ||
| SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: "0" | ||
| SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: "1" | ||
| MC_FORCE_MNNVL: "1" | ||
| NCCL_MNNVL_ENABLE: "1" | ||
| NCCL_CUMEM_ENABLE: "1" | ||
| SGLANG_PER_TOKEN_GROUP_QUANT_8BIT_V2: "1" | ||
| DYN_REQUEST_PLANE: nats | ||
| SGLANG_ENABLE_SPEC_V2: "1" | ||
| decode_environment: | ||
| TORCH_DISTRIBUTED_DEFAULT_TIMEOUT: "1800" | ||
| PYTHONUNBUFFERED: "1" | ||
| DYN_SKIP_SGLANG_LOG_FORMATTING: "1" | ||
| SGLANG_ENABLE_JIT_DEEPGEMM: "false" | ||
| SGLANG_DISAGGREGATION_HEARTBEAT_MAX_FAILURE: "100000" | ||
| SGLANG_DISAGGREGATION_BOOTSTRAP_TIMEOUT: "100000" | ||
| SGLANG_DISAGGREGATION_WAITING_TIMEOUT: "100000" | ||
| SGLANG_DECODE_BOOTSTRAP_TIMEOUT: "1000" | ||
| SGLANG_MOONCAKE_CUSTOM_MEM_POOL: "True" | ||
| SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: "0" | ||
| SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: "1" | ||
| MC_FORCE_MNNVL: "1" | ||
| NCCL_MNNVL_ENABLE: "1" | ||
| NCCL_CUMEM_ENABLE: "1" | ||
| SGLANG_PER_TOKEN_GROUP_QUANT_8BIT_V2: "1" | ||
| DYN_REQUEST_PLANE: nats | ||
| SGLANG_ENABLE_SPEC_V2: "1" | ||
| sglang_config: | ||
| prefill: | ||
| served-model-name: "deepseek-ai/DeepSeek-R1" | ||
| trust-remote-code: true | ||
| quantization: "modelopt_fp4" | ||
| disaggregation-mode: "prefill" | ||
| disaggregation-transfer-backend: nixl | ||
| mem-fraction-static: 0.85 | ||
| max-prefill-tokens: 32768 | ||
| chunked-prefill-size: 32768 | ||
| context-length: 9600 | ||
| max-running-requests: 512 | ||
| disable-cuda-graph: true | ||
| tensor-parallel-size: 4 | ||
| data-parallel-size: 4 | ||
| expert-parallel-size: 4 | ||
| enable-dp-attention: true | ||
| enable-dp-lm-head: true | ||
| attention-backend: "trtllm_mla" | ||
| kv-cache-dtype: "fp8_e4m3" | ||
| moe-runner-backend: "flashinfer_trtllm" | ||
| moe-dense-tp-size: 1 | ||
| fp4-gemm-backend: "flashinfer_trtllm" | ||
| stream-interval: 30 | ||
| watchdog-timeout: 1000000 | ||
| enable-flashinfer-allreduce-fusion: true | ||
| disable-radix-cache: true | ||
| decode: | ||
| served-model-name: "deepseek-ai/DeepSeek-R1" | ||
| trust-remote-code: true | ||
| quantization: "modelopt_fp4" | ||
| disaggregation-mode: "decode" | ||
| disaggregation-transfer-backend: nixl | ||
| mem-fraction-static: 0.85 | ||
| max-prefill-tokens: 32768 | ||
| chunked-prefill-size: 32768 | ||
| context-length: 9600 | ||
| max-running-requests: 512 | ||
| cuda-graph-max-bs: 512 | ||
| tensor-parallel-size: 8 | ||
| data-parallel-size: 1 | ||
| expert-parallel-size: 8 | ||
| attention-backend: "trtllm_mla" | ||
| kv-cache-dtype: "fp8_e4m3" | ||
| moe-runner-backend: "flashinfer_trtllm" | ||
| fp4-gemm-backend: "flashinfer_trtllm" | ||
| stream-interval: 30 | ||
| watchdog-timeout: 1000000 | ||
| enable-flashinfer-allreduce-fusion: true | ||
| disable-radix-cache: true | ||
| speculative-algorithm: "EAGLE" | ||
| speculative-num-steps: 2 | ||
| speculative-eagle-topk: 1 | ||
| speculative-num-draft-tokens: 3 | ||
|
|
||
| health_check: | ||
| max_attempts: 360 | ||
| interval_seconds: 10 | ||
|
|
||
| benchmark: | ||
| type: "sa-bench" | ||
| isl: 8192 | ||
| osl: 1024 | ||
| req_rate: "inf" | ||
|
|
||
|
|
||
| # TP4 prefill mode: tensor-parallel instead of data-parallel for prefill | ||
| override_tp4: | ||
| name: "b200-fp4-low-latency-tp4-1p-tp8-1d" | ||
| frontend: | ||
| num_additional_frontends: 2 | ||
| backend: | ||
| sglang_config: | ||
| prefill: | ||
| data-parallel-size: 1 | ||
| expert-parallel-size: 1 | ||
| enable-dp-attention: null | ||
| enable-dp-lm-head: null | ||
| decode: | ||
| expert-parallel-size: 1 | ||
| benchmark: | ||
| concurrencies: "4x8x16x64" | ||
|
|
||
|
|
||
| # dep4 low-latency scale sweep: 1p1d, 1p5d, 2p5d | ||
| zip_override_lowlat: | ||
| name: | ||
| - "b200-fp4-low-latency-dep4-1p-tep8-1d" | ||
| - "b200-fp4-low-latency-dep4-1p-tep8-5d" | ||
| - "b200-fp4-low-latency-dep4-2p-tep8-5d" | ||
| resources: | ||
| prefill_nodes: [1, 1, 2] | ||
| prefill_workers: [1, 1, 2] | ||
| decode_nodes: [1, 5, 5] | ||
| decode_workers: [1, 5, 5] | ||
| benchmark: | ||
| concurrencies: ["64x128", "8", "4x128"] | ||
|
|
||
|
|
||
| # Max-throughput 4p1d: dep8 decode, no frontends config, SGLANG_FLASHINFER_FP4_GEMM_BACKEND, | ||
| # mem-fraction=0.75, no fp4-gemm-backend in sglang_config | ||
| override_maxtpt_4p1d: | ||
| name: "b200-fp4-max-tpt-dep4-4p-dep8-1d" | ||
| frontend: null | ||
| resources: | ||
| prefill_nodes: 4 | ||
| prefill_workers: 4 | ||
| decode_nodes: 1 | ||
| decode_workers: 1 | ||
| backend: | ||
| decode_environment: | ||
| SGLANG_MOE_NVFP4_DISPATCH: "1" | ||
| SGLANG_FLASHINFER_FP4_GEMM_BACKEND: "cutlass" | ||
| sglang_config: | ||
| prefill: | ||
| max-running-requests: 1024 | ||
| fp4-gemm-backend: null | ||
| decode: | ||
| mem-fraction-static: 0.75 | ||
| data-parallel-size: 8 | ||
| enable-dp-attention: true | ||
| enable-dp-lm-head: true | ||
| moe-dense-tp-size: 1 | ||
| max-running-requests: 1024 | ||
| cuda-graph-max-bs: 1024 | ||
| fp4-gemm-backend: null | ||
| benchmark: | ||
| concurrencies: "1024" | ||
|
|
||
|
|
||
| # Max-throughput 7p2d: dep8 decode, with frontends, SGLANG_MOE_NVFP4_DISPATCH only, | ||
| # flashinfer_cutlass backend, max-running-requests=2048 for decode | ||
| override_maxtpt_7p2d: | ||
| name: "b200-fp4-max-tpt-dep4-7p-dep8-2d" | ||
| resources: | ||
| prefill_nodes: 7 | ||
| prefill_workers: 7 | ||
| decode_nodes: 2 | ||
| decode_workers: 2 | ||
| backend: | ||
| decode_environment: | ||
| SGLANG_MOE_NVFP4_DISPATCH: "1" | ||
| sglang_config: | ||
| prefill: | ||
| max-prefill-tokens: 65536 | ||
| chunked-prefill-size: 65536 | ||
| max-running-requests: 1024 | ||
| fp4-gemm-backend: "flashinfer_cutlass" | ||
| decode: | ||
| data-parallel-size: 8 | ||
| enable-dp-attention: true | ||
| enable-dp-lm-head: true | ||
| moe-dense-tp-size: 1 | ||
| max-running-requests: 2048 | ||
| cuda-graph-max-bs: 1024 | ||
| fp4-gemm-backend: "flashinfer_cutlass" | ||
| benchmark: | ||
| concurrencies: "1024x2048" |
Contributor
There was a problem hiding this comment.
This file is also blocked by root-schema incompatibility in CI.
The current check path is validating this as a direct recipe and failing on missing root fields plus unknown base/override_* keys (Line 16 onward, including Line 139, Line 157, Line 173, Line 204). This needs the same resolution: validate generated/expanded configs in CI (or commit expanded schema-valid recipes at this location).
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@recipes/b200-fp4/8k1k-mtp.yaml` around lines 16 - 229, The CI failure is due
to validating this file as a direct root-schema recipe while it contains nested
recipe fragments (keys like base, override_tp4, zip_override_lowlat,
override_maxtpt_4p1d, override_maxtpt_7p2d) instead of a fully expanded
root-schema document; fix by either (A) changing the CI validator to run on the
generated/expanded configs (update the validation step that reads/validates
recipe fragments to call the expansion logic before schema validation), or (B)
committing the expanded/flattened recipe(s) that conform to the root schema
(replace the base/override_* fragments with the final merged root fields) so
keys such as base and the override_* sections are removed and the file matches
the expected root fields.
Comment on lines
+14
to
+189
| base: | ||
| name: "b200-fp4-stp-8k1k" | ||
|
|
||
| model: | ||
| path: "dsr1" | ||
| container: "dynamo-sglang" | ||
| precision: "fp4" | ||
|
|
||
| frontend: | ||
| num_additional_frontends: 4 | ||
|
|
||
| resources: | ||
| gpu_type: "b200" | ||
| prefill_nodes: 1 | ||
| prefill_workers: 1 | ||
| gpus_per_prefill: 4 | ||
| decode_nodes: 1 | ||
| decode_workers: 1 | ||
| gpus_per_node: 8 | ||
|
|
||
| backend: | ||
| prefill_environment: | ||
| TORCH_DISTRIBUTED_DEFAULT_TIMEOUT: "1800" | ||
| PYTHONUNBUFFERED: "1" | ||
| DYN_SKIP_SGLANG_LOG_FORMATTING: "1" | ||
| SGLANG_ENABLE_JIT_DEEPGEMM: "false" | ||
| SGLANG_DISAGGREGATION_HEARTBEAT_MAX_FAILURE: "100000" | ||
| SGLANG_DISAGGREGATION_BOOTSTRAP_TIMEOUT: "100000" | ||
| SGLANG_DISAGGREGATION_WAITING_TIMEOUT: "100000" | ||
| SGLANG_MOONCAKE_CUSTOM_MEM_POOL: "True" | ||
| SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: "0" | ||
| SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: "1" | ||
| MC_FORCE_MNNVL: "1" | ||
| NCCL_MNNVL_ENABLE: "1" | ||
| NCCL_CUMEM_ENABLE: "1" | ||
| SGLANG_PER_TOKEN_GROUP_QUANT_8BIT_V2: "1" | ||
| DYN_REQUEST_PLANE: nats | ||
| decode_environment: | ||
| TORCH_DISTRIBUTED_DEFAULT_TIMEOUT: "1800" | ||
| PYTHONUNBUFFERED: "1" | ||
| DYN_SKIP_SGLANG_LOG_FORMATTING: "1" | ||
| SGLANG_ENABLE_JIT_DEEPGEMM: "false" | ||
| SGLANG_DISAGGREGATION_HEARTBEAT_MAX_FAILURE: "100000" | ||
| SGLANG_DISAGGREGATION_BOOTSTRAP_TIMEOUT: "100000" | ||
| SGLANG_DISAGGREGATION_WAITING_TIMEOUT: "100000" | ||
| SGLANG_DECODE_BOOTSTRAP_TIMEOUT: "1000" | ||
| SGLANG_MOONCAKE_CUSTOM_MEM_POOL: "True" | ||
| SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: "0" | ||
| SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: "1" | ||
| MC_FORCE_MNNVL: "1" | ||
| NCCL_MNNVL_ENABLE: "1" | ||
| NCCL_CUMEM_ENABLE: "1" | ||
| SGLANG_PER_TOKEN_GROUP_QUANT_8BIT_V2: "1" | ||
| DYN_REQUEST_PLANE: nats | ||
| sglang_config: | ||
| prefill: | ||
| served-model-name: "deepseek-ai/DeepSeek-R1" | ||
| trust-remote-code: true | ||
| quantization: "modelopt_fp4" | ||
| disaggregation-mode: "prefill" | ||
| disaggregation-transfer-backend: nixl | ||
| mem-fraction-static: 0.85 | ||
| max-prefill-tokens: 32768 | ||
| chunked-prefill-size: 32768 | ||
| context-length: 9600 | ||
| max-running-requests: 512 | ||
| disable-cuda-graph: true | ||
| tensor-parallel-size: 4 | ||
| data-parallel-size: 4 | ||
| expert-parallel-size: 4 | ||
| enable-dp-attention: true | ||
| enable-dp-lm-head: true | ||
| attention-backend: "trtllm_mla" | ||
| kv-cache-dtype: "fp8_e4m3" | ||
| moe-runner-backend: "flashinfer_trtllm" | ||
| moe-dense-tp-size: 1 | ||
| fp4-gemm-backend: "flashinfer_trtllm" | ||
| stream-interval: 30 | ||
| watchdog-timeout: 1000000 | ||
| enable-flashinfer-allreduce-fusion: true | ||
| disable-radix-cache: true | ||
| decode: | ||
| served-model-name: "deepseek-ai/DeepSeek-R1" | ||
| trust-remote-code: true | ||
| quantization: "modelopt_fp4" | ||
| disaggregation-mode: "decode" | ||
| disaggregation-transfer-backend: nixl | ||
| mem-fraction-static: 0.85 | ||
| max-prefill-tokens: 32768 | ||
| chunked-prefill-size: 32768 | ||
| context-length: 9600 | ||
| max-running-requests: 512 | ||
| cuda-graph-max-bs: 512 | ||
| tensor-parallel-size: 8 | ||
| data-parallel-size: 1 | ||
| expert-parallel-size: 8 | ||
| attention-backend: "trtllm_mla" | ||
| kv-cache-dtype: "fp8_e4m3" | ||
| moe-runner-backend: "flashinfer_trtllm" | ||
| fp4-gemm-backend: "flashinfer_trtllm" | ||
| stream-interval: 30 | ||
| watchdog-timeout: 1000000 | ||
| enable-flashinfer-allreduce-fusion: true | ||
| disable-radix-cache: true | ||
|
|
||
| health_check: | ||
| max_attempts: 360 | ||
| interval_seconds: 10 | ||
|
|
||
| benchmark: | ||
| type: "sa-bench" | ||
| isl: 8192 | ||
| osl: 1024 | ||
| req_rate: "inf" | ||
|
|
||
|
|
||
| # TP4 prefill mode: tensor-parallel instead of data-parallel for prefill | ||
| override_tp4: | ||
| name: "b200-fp4-low-latency-tp4-1p-tp8-1d" | ||
| frontend: | ||
| num_additional_frontends: 2 | ||
| backend: | ||
| sglang_config: | ||
| prefill: | ||
| data-parallel-size: 1 | ||
| expert-parallel-size: 1 | ||
| enable-dp-attention: null | ||
| enable-dp-lm-head: null | ||
| decode: | ||
| expert-parallel-size: 1 | ||
| benchmark: | ||
| concurrencies: "4x8x16x64" | ||
|
|
||
|
|
||
| # dep4 low-latency scale sweep: 1p1d, 1p5d, 2p5d | ||
| zip_override_lowlat: | ||
| name: | ||
| - "b200-fp4-low-latency-dep4-1p-tep8-1d" | ||
| - "b200-fp4-low-latency-dep4-1p-tep8-5d" | ||
| - "b200-fp4-low-latency-dep4-2p-tep8-5d" | ||
| resources: | ||
| prefill_nodes: [1, 1, 2] | ||
| prefill_workers: [1, 1, 2] | ||
| decode_nodes: [1, 5, 5] | ||
| decode_workers: [1, 5, 5] | ||
| benchmark: | ||
| concurrencies: ["64x128", "8", "4x128"] | ||
|
|
||
|
|
||
| # Max-throughput 7p2d: dep8 decode, large-scale, flashinfer_cutlass backend | ||
| override_maxtpt_7p2d: | ||
| name: "b200-fp4-max-tpt-dep4-7p-dep8-2d" | ||
| resources: | ||
| prefill_nodes: 7 | ||
| prefill_workers: 7 | ||
| decode_nodes: 2 | ||
| decode_workers: 2 | ||
| backend: | ||
| decode_environment: | ||
| SGLANG_MOE_NVFP4_DISPATCH: "1" | ||
| sglang_config: | ||
| prefill: | ||
| max-prefill-tokens: 65536 | ||
| chunked-prefill-size: 65536 | ||
| max-running-requests: 1024 | ||
| fp4-gemm-backend: "flashinfer_cutlass" | ||
| decode: | ||
| data-parallel-size: 8 | ||
| enable-dp-attention: true | ||
| enable-dp-lm-head: true | ||
| moe-dense-tp-size: 1 | ||
| max-running-requests: 2048 | ||
| cuda-graph-max-bs: 1024 | ||
| fp4-gemm-backend: "flashinfer_cutlass" | ||
| benchmark: | ||
| concurrencies: "1024x2048" |
Contributor
There was a problem hiding this comment.
Top-level template structure is currently invalid for the recipe validator.
CI is failing because the validator expects name, model, and resources at the root, but this file uses base plus override_* keys (e.g., Line 14, Line 131, Line 149, Line 164), which are being treated as unknown fields. Please either validate expanded outputs (post-override generation) in CI or commit schema-valid expanded recipe files for this path.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@recipes/b200-fp4/8k1k-stp.yaml` around lines 14 - 189, The recipe uses a
top-level "base" plus override blocks (base, override_tp4, zip_override_lowlat,
override_maxtpt_7p2d) which breaks the validator that requires root keys name,
model, and resources; fix by committing the fully expanded/merged recipe(s) so
each committed YAML has the required top-level keys (name, model, resources,
backend, etc.) instead of nested "base"/"override_*" blocks, or update CI to run
the override expansion step before validation (i.e., ensure the validator sees
the generated/expanded output rather than the templated file).
added 2 commits
March 4, 2026 02:38
Reduce 40 individual recipe files to 8 override files (one per precision × isl × stp/mtp combination). Each file uses zip_override_scale to sweep all node-count variants, eliminating per-variant YAML duplication. FP4 8k1k files additionally use override_tp4 to cover the TP4 prefill mode alongside the default dep4 variants. Before: b200-fp8 (21 files) + b200-fp4 (19 files) = 40 files After: 8 override files covering all same variants recipes/b200-fp8/1k1k-stp.yaml (4 variants: 1p1d/1p3d low-lat, 1p5d/2p5d max-tpt) recipes/b200-fp8/1k1k-mtp.yaml (6 variants) recipes/b200-fp8/8k1k-stp.yaml (5 variants: 1p1d/1p4d/1p6d low-lat, 1p1d/2p1d max-tpt) recipes/b200-fp8/8k1k-mtp.yaml (6 variants) recipes/b200-fp4/1k1k-stp.yaml (4 variants: 1p5d/1p6d low-lat, 1p1d/1p2d max-tpt) recipes/b200-fp4/1k1k-mtp.yaml (4 variants) recipes/b200-fp4/8k1k-stp.yaml (5 dep4 variants + override_tp4) recipes/b200-fp4/8k1k-mtp.yaml (5 dep4 variants + override_tp4)
…ions
Recipe fixes:
- Move num_additional_frontends from resources: to frontend: in FP4 8k1k
files (was causing schema validation Unknown field error)
- Fix override_maxtpt_4p1d: use frontend: null to drop frontend config
(original file has no frontend section)
- Fix override_tp4: remove erroneous fp4-gemm-backend: null (original
tp4 file keeps flashinfer_trtllm backend), add decode expert-parallel-size: 1
- Separate low-lat and max-tpt into distinct zip_override_ groups so each
carries appropriate sglang_config overrides (DP=8, moe-dense-tp-size, etc.)
- FP4 1k1k MTP max-tpt: add per-variant mem-fraction-static list [0.75, 0.85]
- FP8 MTP max-tpt: keep max-running-requests=512 (STP raises to 1024, MTP does not)
- FP8 1k1k MTP: add override_maxtpt_1p2d special case with spec-steps=1, draft-tokens=2
Core fix:
- generate_override_configs: respect explicit name: field in override_* dicts
instead of always auto-generating {base_name}_{suffix}; add test coverage
weireweire
force-pushed
the
simplify/b200-recipes
branch
from
March 4, 2026 02:39
4bdf85a to
bdd3ca9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
num_additional_frontendsplacement: was incorrectly underresources:, now correctly underfrontend:(resolves schema validation errors)override_tp4: remove erroneousfp4-gemm-backend: nulland add missingdecode: expert-parallel-size: 1zip_override_lowlat/zip_override_maxtptgroups, each carrying correct sglang_config overrides (DP=8, moe-dense-tp-size, env vars, etc.)max-running-requests=512in max-tpt (STP raises to 1024, MTP does not — preserves original behavior)override_maxtpt_1p2dspecial case (speculative-num-steps=1,speculative-num-draft-tokens=2)mem-fraction-static: [0.75, 0.85]generate_override_configs: honor explicitname:field inoverride_*dicts instead of always auto-generating{base_name}_{suffix}; add test coverageFile summary
recipes/b200-fp8/1k1k-stp.yamlrecipes/b200-fp8/1k1k-mtp.yamlrecipes/b200-fp8/8k1k-stp.yamlrecipes/b200-fp8/8k1k-mtp.yamlrecipes/b200-fp4/1k1k-stp.yamlrecipes/b200-fp4/1k1k-mtp.yamlrecipes/b200-fp4/8k1k-stp.yamlrecipes/b200-fp4/8k1k-mtp.yamlTest plan
make checkpasses (329 tests)srtctl dry-runon all 8 files shows correct variant counts and names with no errorsSummary by CodeRabbit
New Features
Tests