Add DeepEP V2 benchmark (NCCL GIN)

[Xuan-1998]

DeepEP V2 replaces the NVSHMEM transport with NCCL GIN (GPU-Initiated Networking).

Two ways to consume it, mirroring the existing deepep-benchmark (v1):

• setup_deepep_gin.sh: run inside your own container. Uninstalls any existing deep_ep, validates the NCCL is GIN-capable (nccl_device.h), and builds + installs the EFA/GIN DeepEP against your torch.
• deepep.Dockerfile: reference image for those without a stack. Builds EFA userspace, a GIN-capable NCCL, aws-ofi-nccl with --enable-gdaki, and torch, then calls the same script.

Slurm launchers run tests/elastic/test_ep.py on 1 node (NVLink) and 2 nodes (EFA-GDA). The sbatch files pin NCCL_GIN_TYPE=5: with it unset, NCCL selects the CPU-proxy GIN backend.

[KeitaW]

Review Batch 1/4: Runnability on the Stated Platforms (hardware-verified)

I ran this benchmark end-to-end on 2× p6-b300.48xlarge (Blackwell, EFA-GDA-capable stack: EFA kernel driver 3.3.0g, libfabric 2.6, aws-ofi-nccl with GDAKI, NCCL v2.31.2-1, torch 2.11.0+cu130). setup_deepep_gin.sh executed unmodified inside that container, and the internode/intranode tests ran with the sbatch environment ported verbatim. The good news is real and worth stating up front: the script's validation gates work, DeepEP builds and installs cleanly, and on EFA the GIN device-communicator creation succeeds on both nodes (Experts: 8/256, #QPs: 11/11, no init rejection, no counter exhaustion). The EFA enablement this PR exists to demonstrate is sound. The blockers below are in the compute path and the artifact chain.

[KeitaW]

Review Batch 2/4: Launcher Environment & Host Prerequisites

The internode launcher's environment block mixes real, load-bearing variables with two that don't exist in any aws-ofi-nccl source, and the README documents one of the nonexistent ones while omitting several of the real ones.

[KeitaW]

Review Batch 3/4: Pinning & Reproducibility

[KeitaW]

Review Batch 4/4: Script & Docs Polish, Things That Look Great, Sources

Things That Look Great

• The setup script works as designed: I ran setup_deepep_gin.sh unmodified inside a GIN-capable container: the nccl_device.h validation gate fired correctly during development of the run (it refused a non-GIN NCCL), DeepEP cloned/built/installed cleanly, and the completion notes are accurate (verified live on B300, 2026-08-21).
• The EFA-GDA enablement at the heart of this benchmark is real: on 2× p6-b300 the GIN device-communicator creation succeeded on all ranks with the fork's QP/signal sizing (Experts: 8/256, #QPs: 11/11), with no init rejection and no counter exhaustion (verified live, 2026-08-21). The blockers I found are compute-path and artifact-chain issues, not EFA ones.
• The GIN-type table matches observed runtime behavior: with NCCL_GIN_TYPE=5 the Libfabric_GDAKI backend loads, and without it NCCL selects the proxy backend, exactly as the README describes (verified live, 2026-08-21).
• BuildKit usage is properly cued: the Dockerfile uses RUN --mount and the README build commands all carry DOCKER_BUILDKIT=1.
• Secret hygiene is right: the GitHub token comes in via a BuildKit secret mount, the credential helper is unset afterward, and nothing lands in image layers.
• NCCL_SOCKET_IFNAME=^docker,lo,veth follows the exclusion-pattern convention this repo standardizes on.
• The RANK/WORLD_SIZE-as-node-rank/node-count convention matches DeepEP's init_dist (deep_ep/utils/envs.py reads them exactly this way; verified in source).
• All the resolvable pins resolve: NCCL v2.31.2-1, gdrcopy v2.5.2, nvidia/cuda:13.0.2-devel-ubuntu22.04, and torch==2.11.0 cu130 wheels all exist (verified live, 2026-08-21).

Sources

Hardware verification (2026-08-21): 2× p6-b300.48xlarge, EFA kernel driver 3.3.0g, libfabric 2.6, aws-ofi-nccl with GDAKI, NCCL v2.31.2-1 + CUDA 13.0, torch 2.11.0+cu130; setup_deepep_gin.sh run unmodified; internode/intranode test_ep.py runs with the sbatch environment ported verbatim; JIT failure reproduced offline with the same nvcc flags; st_bulk fix validated in-container.

Public-components build verification (2026-08-21): a four-delta variant of this PR's Dockerfile (EFA installer 1.49.0, rdma-core linux-rdma/rdma-core @ 9715b7b, libfabric aws/libfabric v2.6.0amzn1.0 built against it with the HAVE_EFADV_CREATE_COMP_CNTR 1 gate, aws-ofi-nccl v1.21.1 with no --enable-gdaki and the ncclGinPlugin_v14 gate) built start-to-finish from public artifacts only, and on 2× p6-b300 initialized NCCL + the GIN device communicator over EFA (Experts: 8/256, #QPs: 11/11, zero GIN/counter errors) before hitting the identical DeepEP st.bulk ptxas failure (same PTX line signature) as the primary run.

Artifact availability (verified live, 2026-08-21)

• aws-efa-installer-1.50.0.tar.gz → HTTP 403; 1.45.0–1.49.0 → 200; latest → 200, bundles libfabric-aws 2.4.0amzn5.0.

aws-ofi-nccl master (verified live, 2026-08-21)

• configure.ac: no AC_ARG_ENABLE([gdaki]); GDAKI auto-detect at :355-362.
• src/rdma/gin/nccl_ofi_gin_api.cpp:566,595 (unconditional ncclGinPlugin_v11/v13), :620-621 (v14 is GDAKI-only), src/rdma/gin/nccl_ofi_gin_gdaki.cpp:1225 (v14 export), :1177-1178 (supportsStrongSignals/supportsVASignals = false).
• src/rdma/gin/nccl_ofi_gin_resources.cpp:349 (nccl_ofi_gin_gdaki_capable referenced unconditionally).
• Repo-wide grep + GitHub code search: OFI_NCCL_GIN_TYPE, OFI_NCCL_GIN_STRONG_SIGNAL → 0 hits.

DeepEP (verified in source, 2026-08-21)

• amazon-contributing/DeepEP deep_ep/include/deep_ep/common/ptx.cuh (st_bulk, "r"(kNumBytes) under __CUDA_ARCH__ >= 1000); same in deepseek-ai/DeepEP.
• deep_ep/__init__.py:51 (EP_SUPPRESS_NCCL_CHECK gate), deep_ep/utils/envs.py (init_dist).

Docs

• PTX ISA: st.bulk (size operand is .u64)
• Blackwell Compatibility Guide (sm_100 family / sm_103)
• amzn-drivers EFA RELEASENOTES (completion counters at r3.3.0)

[KeitaW]

Batch 1/4 — Round-1 resolution, and the revised PR on hardware

I rebuilt this PR from scratch with no build-arg overrides and reran both benchmarks.

The good news is substantial. docker build now completes end to end from public artifacts, and the internode benchmark passes: RUN_EXIT=0 on both nodes, dispatch 115 GB/s (SO) / 377 GB/s (SU), combine 80 GB/s (SO) / 263 GB/s (SU), with no DeepEP patch of any kind. Every blocker I raised in round 1 is gone. By inspection of the diff, 20 of 22 round-1 findings are fixed in code, 1 (the unpublished installer) was overtaken by the 1.50.0 release, and 1 (DeepEP pinned to main) is settled by a decision you stated in-thread, which I am not re-litigating.

I also want to retract a round-1 claim of my own: I reported aws-efa-installer-1.50.0.tar.gz returning HTTP 403 as of 2026-08-21. It returns 200 as of 2026-08-22 and the default build resolves it cleanly, so that finding no longer stands.

The one thing I would not merge without a look is the intranode benchmark, which does not get through initialization with the environment its launcher sets, and then returns a wrong result once it does. Batch 2 is about that; the evidence localises it to a zero-QP divide path in the pinned DeepEP dependency, and it is addressable from the launcher without patching DeepEP. Batch 3 covers what changed now that 1.50.0 shipped. Batch 4 is documentation accuracy plus what stands out.

Process note: this review went through an independent adversarial pass on a different model, which returned a no-go on my first draft. Its objections are the reason several claims below carry explicit boundaries, and two of the findings (the README pinning contradiction, and the --nccl-root help text) are its catches rather than mine.

[KeitaW]

Batch 2/4 — The intranode benchmark does not complete on p6-B300

Both benchmarks in the README's table are presented as things a user runs. The internode one works. The intranode one fails in two separate places, and I isolated each with single-variable arms on the same image and node. Neither is a defect in this PR's own code, and both are addressable in its launcher.

[KeitaW]

Batch 3/4 — What changed now that installer 1.50.0 shipped

The 1.50.0 release landed between round 1 and this build, and it changes one of the PR's premises. The installer now bundles aws-ofi-nccl 1.21.1, the same upstream version the Dockerfile pins for its source build, and that bundled plugin is GDAKI-enabled.

[KeitaW]

Batch 4/4 — Documentation accuracy, a leftover, and what looks great

Three wording corrections, one leftover line, and the parts of this revision that stand out.

Whitespace left by the earlier deletions is still there: a three-blank-line gap at slurm/test-internode.sbatch:45-47, and a two-blank-line gap at setup_deepep_gin.sh:139-140.

[erezzarum]

Can you please add an argument to support producing a wheel file instead of direct install as we have in the DeepEP v1 script? https://github.com/awslabs/awsome-distributed-ai/blob/main/micro-benchmarks/expert-parallelism/deepep-benchmark/setup_deepep_efa.sh#L1239

[KeitaW]

LGTM!

[Xuan-1998]

Latest comments addressed in #1239