feat(deepep-efa): vLLM DeepEP-V2 MoE all-to-all over EFA via NCCL-GIN GDAKI (GPU-initiated)

[dmvevents]

What

Adds a vLLM + DeepEP-V2 MoE all-to-all test case that routes the expert-parallel dispatch/combine over AWS EFA using the NCCL-GIN GDAKI transport (NCCL_GIN_TYPE=3, OFI_NCCL_GIN_GDAKI=1) — the GPU-initiated path where the GPU kernel posts the RDMA work requests itself, instead of handing them to a CPU proxy thread.

Placement: 3.test_cases/pytorch/vllm/deepep-v2-GDAKI-efa, parallel to the CPU-proxy sample 3.test_cases/pytorch/vllm/deepep-v2-efa (PR #1230). It mirrors that sibling's layout, README section order, and voice; the only functional deltas are the GDAKI transport stack. (The proxy sibling is not on main yet — #1230 is open — so this PR branches from main and adds the GDAKI sample as a peer; both touch only 3.test_cases/pytorch/vllm/README.md's index table.)

Why GDAKI needs a different substrate

DeepEP's default transport is NVSHMEM/IBGDA, which EFA does not provide. The proxy sibling runs V2's ElasticBuffer over the GIN CPU-proxy. This sample uses GDAKI (kernel-posted WQEs), which on EFA needs a newer transport substrate, all built from public source and pinned by immutable SHA:

• rdma-core @ master post-PR#1701 (merged 2026-08-04) — EFA completion-counter verbs. No release tag carries it yet → SHA pin.
• libfabric @ main post-PR#12591 (merged 2026-07-28) — prov/efa consumes the comp-cntr caps; built against the rdma-core above.
• aws-ofi-nccl @ a3d2680 --enable-gdaki + a PR#1351 cherry-pick (forced-PCIe gdrcopy override). a3d2680 carries PR#1311 (the per-platform EFA hw-counter tristate OFI_NCCL_GDAKI_EFA_HW_COUNTER) and the GIN seq-space aliasing fix.
• DeepEP b306af06 + PR#612 (same base+PR as the proxy sample) + a dlsym-guarded V13 host-UC shim that is inert (logs a fallback and no-ops) unless the plugin exports a hook upstream does not — carried as evidence lineage, not a live code path.

Everything else — EP_REUSE_NCCL_COMM=0 (or serve init segfaults under vLLM's lazy NCCL comm), the vLLM wheel-pin, and the eager/non-eager fix stack — is identical to the proxy sample.

Measured evidence (2026-08-14, 2×/4× p5en.48xlarge H200, Qwen/Qwen3-30B-A3B-FP8)

Full tables + provenance + honest caveats in benchmarks/README.md.

• Eager + non-eager serve, DP16/EP16 and DP32/EP32 — every sweep 121/121 HTTP 200, zero GDAKI-CQE … status 9 (the known upstream failure signature) through every phase.
• Same-node-set, same-image transport A/B (env-flip only): GDAKI ≥ CPU-proxy in all 10 paired comparisons across the EP16+EP32 2×2, +0.2–3.2% aggregate tok/s.
• Honest scope: single sweep per arm (the 2×2 consistency shows a direction, not a tight interval); the validation nodes ran efa.ko 3.0.x/3.1.x, so cross-node proof is functional (efa-direct boot banner ×8 + coherent EP output), not a byte-level /sys hw-counter tally — that needs efa.ko ≥ 3.3.0 (documented in the node preconditions).

Build / reproducibility

• NGC-from-scratch: FROM nvcr.io/nvidia/cuda:13.0.0-devel-ubuntu22.04; every layer from a public source (EFA installer, rdma-core, libfabric, gdrcopy, torch cu130, aws-ofi-nccl, DeepEP, vLLM wheel). A tester with docker + NGC + GitHub/PyPI reproduces the image with zero private access.
• No registry is hardcoded — setup/env_vars.example uses the standard placeholder; the manifest uses REPLACE_WITH_YOUR_REGISTRY.
• setup_deepep_v2_gdaki_efa.sh is COPY'd (in-tree, reviewable), not curled, and is first-party for the GDAKI path — deliberately outside .github/workflows/deepep-vendor-sync.yml (that CI gates the NVSHMEM setup_deepep_efa.sh vendored copy, a different script). The V13 shim patch was dry-run-verified to apply cleanly (patch -p0, exit 0) against upstream DeepEP b306af06.

Layout

deepep-v2-GDAKI-efa/
  Dockerfile                       NGC-from-scratch; GDAKI stack + rdma-core/libfabric build layers
  setup_deepep_v2_gdaki_efa.sh     COPY'd: builds aws-ofi-nccl --enable-gdaki + stages DeepEP + V13 shim
  v13-host-uc-shim.patch           dlsym-guarded, inert on upstream
  setup/                           build-push.sh + env_vars.example (bring your own ECR)
  recipe/                          serve.sh, run-kernel-test.sh, build_deepep.sh (in-pod _C.so),
                                   verify-image.sh, benchmark.sh, benchmark_probe.py, apply-noneager-fix-stack.sh
  kubernetes/                      2-node StatefulSet + headless service (GDAKI env contract + hw-counter tristate)
  benchmarks/README.md             measured eager/non-eager/A-B tables (EP16 + EP32) + caveats

Opened as draft — pending maintainer review of placement/scope alongside #1230.

DCO signed off.

[KeitaW]

Let's incorporate necessary changes into upstream instead of patches.

[dmvevents]

Re-scoped per the repo's no-patches rule — thank you for stating it plainly rather than letting it surface in review.

What changed (this push):

• recipe/apply-noneager-fix-stack.sh (cherry-pick + build-time source rewrite) is removed. The case now ships zero patches.
• Eager (--enforce-eager) is the shipped + supported path — it is also our production recommendation today.
• Default compilation (CUDA graphs) is documented as pending one upstream fix: the empty-ExpertTokensMetadata guard, now filed as vllm-project/vllm#52632. Once it merges, a vLLM pin bump enables non-eager here with no recipe change — exactly the pin-bump-survivable shape the rule requires.
• The non-eager numbers in benchmarks/ are retained but relabeled as historical measurements taken with the then-unmerged guard.

This keeps the case alive across pin bumps by construction: nothing in the build rewrites upstream source.

[dmvevents]

One follow-up commit (6fe5b02b): the NCCL linker-precedence assert in the Dockerfile used a pattern ldconfig -p can never print (name => path — the real format is name (libc6,x86-64) => path), so the step failed 100% of the time. Found by live-building the sibling vLLM DeepEP-V2 case (#1230) from its branch, where the same assert shipped; the eager path there is now proven end-to-end on 2× H200 (details on #1230). Same one-line fix here; still zero patches.

[dmvevents]

@KeitaW the changes-requested items are addressed on this branch (patch machinery deleted, zero-patch eager path is the shipped recipe — details in the two comments above), and the sibling #1230 eager path is live-verified E2E from its branch. Ready for another look whenever convenient. (Noting it here since fork authors can't use the re-request-review button on this repo.)

[dmvevents]

Thanks @KeitaW — agreed, upstream-first is the right call, and it's the direction this stack is already built on. Here's exactly where each modification lives today (every status re-verified against the source repos):

Already merged upstream — the Dockerfile pins the post-merge SHA, so these are incorporated upstream, not carried as patches:

• rdma-core #1701 (EFA completion-counter verbs) — merged 2026-08-04
• libfabric #12591 (prov/efa comp-cntr caps) — merged 2026-07-28
• aws-ofi-nccl #1311 (per-platform EFA hw-counter tristate) — merged 2026-08-04, carried by the a3d2680 pin

Filed upstream, open: DeepEP deepseek-ai/DeepEP#612 (cap auto-QP at 2 on EFA). I'll pin the merge SHA once it lands and drop the local reference.

The two genuine local artifacts — and how I'll retire them:

• The aws-ofi-nccl forced-PCIe cherry-pick was #1351; I filed it upstream and the maintainer closed it with an explicit decision to require GDRCopy 2.5+ rather than support the 2.4.x series. I'll follow that guidance: make GDRCopy 2.5+ a node precondition and drop the cherry-pick.
• v13-host-uc-shim.patch is dlsym-guarded and inert on the pinned plugin (a3d2680 doesn't export the host-UC register hook), so it's dead code at this pin. I'll remove it from this build — behavior is unchanged.

Net: this folder reduces to post-merge upstream SHA pins + one open upstream DeepEP PR, with zero local source patches. I'll push that reduction, re-run the verify-image.sh gate, and re-request review.

[dmvevents]

Reduction pushed + re-review requested — head is now 003072084b, zero local source patches

Following up on the 08-26 plan above (the "I'll push that reduction … and re-request review" note) — that reduction is now on the branch, and I want to put the ball back in your court, @KeitaW.

Your ask ("incorporate necessary changes into upstream instead of patches") is satisfied — the folder now carries zero local source patches. Verified against the current head 003072084b:

• No .patch files, no patch-applying RUN step in examples/inference/vllm/deepep-v2-gdaki-efa/ — the only two lines matching patch/cherry-pick in the Dockerfile are comments explaining why there are none.
• DeepEP source is pinned to the amazon-contributing/DeepEP fork (the AWS EPv2/NCCL-GIN fork that carries the EFA delta in-code, superseding the draft deepseek-ai/DeepEP#612) — an immutable SHA, no local source patches (commit 003072084b).
• The forced-PCIe cherry-pick is gone. aws-ofi-nccl#1351 was closed-unmerged with the maintainer's explicit guidance to require GDRCopy 2.5+ on the node rather than support the 2.4.x series. I followed that: the folder now states a GDRCopy ≥ 2.5 node precondition (Dockerfile Layer 3) instead of carrying the cherry-pick.
• v13-host-uc-shim.patch removed — it was dlsym-guarded and inert on the pinned plugin, so removing it is behavior-preserving.

Net vs the 08-17 review: post-merge upstream SHA pins + the amazon-contributing DeepEP fork pin, with zero local source patches. (The three enabling upstream PRs — rdma-core#1701, libfabric#12591, aws-ofi-nccl#1311 — all merged and are carried by SHA pins, not patches.)

Also since your review the folder migrated to examples/inference/vllm/deepep-v2-gdaki-efa/ (reorg #1119) — that's why the older paths moved; file contents are unchanged by the move. npx markdownlint-cli2 is clean on the moved tree.

Ready for another look whenever you have a cycle — thank you.