Skip to content

Add Kimi K2.5 B200 workload#43

Open
khluu wants to merge 3 commits into
mainfrom
codex/add-kimi-k2-5-b200
Open

Add Kimi K2.5 B200 workload#43
khluu wants to merge 3 commits into
mainfrom
codex/add-kimi-k2-5-b200

Conversation

@khluu

@khluu khluu commented Jul 22, 2026

Copy link
Copy Markdown
Member

This PR was authored with assistance from Codex.

Summary

  • add a Kimi K2.5 NVFP4 workload for four B200 GPUs
  • follow the official vLLM Blackwell recipe with FlashInfer FP4, FP8 KV cache, and Eagle3 speculative decoding
  • retain the existing Kimi GSM8K, BFCL, and random serving benchmark coverage

Recipe: https://recipes.vllm.ai/moonshotai/Kimi-K2.5

Local validation

  • parser smoke test with a stubbed lm_eval registry
  • bash -n lib/run.sh lib/server.sh lib/run_lm_eval.sh lib/run_vllm_bench.sh
  • python3 .buildkite/test_generate_pipeline.py (6/6 passed)
  • explicit WORKLOADS=workloads/kimi_k2_5_b200.yaml pipeline generation
  • git diff --check

GPU validation

  • Buildkite #311: exposed an unavailable optional attention-config field in the pinned nightly
  • Buildkite #315: infrastructure-only failure; the pod remained Pending for 15 minutes and the workload never started
  • Buildkite #316: passed in 46m56s on 4x B200; server startup, the 512-request 8K/1K serving benchmark, GSM8K, and all five BFCL categories completed successfully

Follow the vLLM Blackwell NVFP4 recipe and retain the existing Kimi accuracy, BFCL, and serving benchmark coverage.

Co-Authored-By: Codex <noreply@anthropic.com>
Signed-off-by: khluu <khluu000@gmail.com>
The current vLLM nightly rejects use_trtllm_ragged_deepseek_prefill as an unknown AttentionConfig field during startup.

Co-Authored-By: Codex <noreply@anthropic.com>
Signed-off-by: khluu <khluu000@gmail.com>

khluu commented Jul 22, 2026

Copy link
Copy Markdown
Member Author

GPU validation update:

  • Buildkite #311 reached vllm serve but failed immediately because the current nightly rejects use_trtllm_ragged_deepseek_prefill as an unknown AttentionConfig field.
  • Removed that optional performance override in dddacbf while retaining the NVFP4/FP8 Kimi recipe.
  • Retry: Buildkite #315.

Co-Authored-By: Codex <noreply@anthropic.com>
Signed-off-by: khluu <khluu000@gmail.com>

khluu commented Jul 22, 2026

Copy link
Copy Markdown
Member Author

Buildkite #315 never reached the workload: agent-stack failed the job after its Kubernetes pod remained Pending for 15 minutes. I updated the workload to the official recipe's supported 4x B200 / TP4 topology in da44766, reran the parser/generator/shell/tests locally, and will retrigger GPU validation when capacity is available.

khluu commented Jul 22, 2026

Copy link
Copy Markdown
Member Author

Buildkite #316 passed in 46m56s on da44766. The four-B200 server became healthy, the 512-request 8K/1K benchmark completed, GSM8K completed, and all five BFCL categories completed. Marking ready for review.

@khluu
khluu marked this pull request as ready for review July 22, 2026 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant