docs(qwen3): Qwen3-14B P/D campaign — long-input A/B, hugepage pool, elastic 2P+2D#701
docs(qwen3): Qwen3-14B P/D campaign — long-input A/B, hugepage pool, elastic 2P+2D#701xiaguan wants to merge 6 commits into
Conversation
…elastic 2P+2D Benchmark snapshot for the 2026-07-17 Qwen3-14B P/D re-verification on 8x H200: correctness/failure gates, standard-load A/B vs mixed and vLLM 0.25.1 (equal-card P/D wins throughput +17% and ITL p99 by 4.5x with doubled input lengths), heavy-load hugepage host-pool contrast (pool sizing decides decode purity), and the elastic 2P+2D ladder showing prefix-affinity routing (pegaflow #405) is required for multi-P/multi-D. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: abbe8f6f89
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -0,0 +1,103 @@ | |||
| # Qwen3-14B P/D 分离 vs mixed vs vLLM（8×H200，多轮长输入负载） | |||
|
|
|||
| > **TL;DR**: Qwen3-14B bf16 多轮长输入（首轮 8k + 每轮 2k）A/B（2026-07-17）：等卡数下 P/D 全面胜出——2 卡 1P+1D 比 mixed×2 吞吐 +17%（354 vs 303 tok/s）、TPOT p99 -42%（22.8 vs 39.4ms）、**ITL p99 23.5 vs 105ms（-78%）**；对 vLLM 0.25.1×2，vLLM 吞吐/decode 内核更快（453 tok/s、TPOT p50 16.7ms），但 ITL p99 141ms 是 P/D 的 6 倍。重负载（60 会话，KV 1.9× 超 D 的 HBM）下 200GiB hugepage host 池 vs 16GiB 小池：小池丢 decode 纯净性（ITL p99 23.6→84.6ms）+吞吐 -9%。弹性 2P+2D 重负载 ladder：router round-robin ITL p99 85ms → P 亲和 49ms → **P+D 亲和 23.0ms + 495 tok/s**（pegaflow [#405](https://github.com/novitalabs/pegaflow/pull/405)），达到 1P+1D 的隔离度和 2.5× 其吞吐。 | |||
There was a problem hiding this comment.
The TL;DR describes the workload as 1.9× the decode node's HBM capacity, but the documented figures below are 1.02M tokens versus 626k tokens, or about 1.63×; the body accordingly calls it 1.6×. This inconsistency changes the reported memory-pressure level and can mislead readers sizing the host tier, so the TL;DR should use the measured ratio.
Useful? React with 👍 / 👎.
| ## 5. 正确性与故障门（14B 复验） | ||
|
|
||
| - **逐字节一致**：3 档 prompt（<1 page / 跨页 / ~600 tok），temp=0，router P/D vs 直连 D baseline 输出 IDENTICAL ×3。 | ||
| - **P2P 实证**：~600 tok prompt D 侧 `RDMA fetch summary: blocks=33/33 bytes_mib=82.5`，连接复用后 rdma_wait 1.80ms（30.4 GiB/s）。 |
There was a problem hiding this comment.
The stated transfer figures do not yield 30.4 GiB/s: transferring 82.5 MiB in 1.80 ms is approximately 44.8 GiB/s, while 30.4 GiB/s corresponds to about 2.65 ms. Since this is the P2P performance evidence, clarify whether 1.80 ms is RDMA-only and 30.4 GiB/s uses a larger end-to-end duration, or correct whichever value is inaccurate.
Useful? React with 👍 / 👎.
The co-resident decode hiccup during bulk KV restores is not bandwidth or SM contention: the GPU stays ~95% busy with no kernel gaps. The executor step thread's per-step pageable token-readback DtoH degrades from sub-ms to a flat 23.6 ms behind the restore's fragmented H2D flood (~14k copies averaging 63 KiB), pausing token delivery ~110 ms for every active stream. Filed as #704. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…l remeasurement The original router ladder (437/465/495 tok/s) was measured sequentially on one live stack with vllm-bench's fixed seed - later configs got full prefix hits from earlier runs. Cold restarts per config measure 259 (RR) and 268 (prefix affinity) on the #705 engine; the affinity win is decode purity (ITL p99 86 -> 22.8ms), not raw throughput. Section 4.1 now records the measured #704 mechanism (scheduler-thread commit bookkeeping, not DtoH bandwidth) and its fix, and the warm-rerun hazard joins the tooling-pitfalls section. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…oundary Affinity heavy remeasured on 1812d6b: 338 tok/s / ITL p99 22.9 (the global commit budget + scheduler-owned pacing is worth +26% over the pre-review revision). Round-robin: 241 / 76.2. Also records the byte-identical gate's real boundary: restore+suffix-prefill vs local full prefill differ in chunk grid, so near-tie greedy tokens can flip legally under the Tuned policy - reproduced on the pre-fix binary, not a P/D defect. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…bers The #704 fix consolidated into one PR (#705): registry PRT inner map root cause + pinned readback, no pacing machinery. Cold-protocol numbers remeasured on the final branch (RR 248 / affinity ~270); the earlier 338 affinity run and its +26% pacing attribution were run-to-run variance - TPOT/ITL p99 are the stable decision metrics (22.5/22.8 across all three runs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 participant
Benchmark snapshot + M2 doc refresh for the Qwen3-14B P/D re-verification on a single 8x H200 node (per-GPU 400G IB), openinfer main
c116077.What was verified
Files
docs/benchmarks/qwen3-14b-pd-vs-mix-h200.md— full matrices, per-turn tables, tooling pitfallsdocs/models/qwen3/pd-disaggregation-m2.md— TL;DR + boundary notes updated (multi-P/D now measured, host-pool sizing note)docs/index.md— new benchmark row🤖 Generated with Claude Code