test: added device filtering for complex server config

[mpashkovskii]

This PR is only for testing/verification on a specific cloud provider and server. It introduces NIXL_LIBFABRIC_DEVICE_FILTER environment variable for explicit NIC selection, device deduplication, NUMA rail selection after filtering, and ensures GPU detection works for non-EFA providers.

Problem

On the AMD MI300X server I used with many network interfaces (8× Broadcom bnxt_re 400G + 2× Mellanox mlx5 100G), the libfabric plugin had several issues preventing data transfers:

1. No way to select a specific NIC

On machines with 10+ RDMA-capable devices, there was no mechanism to select which NIC(s) to use. Our test environment has 8× Broadcom bnxt_re NICs (400G each) that have RDMA capability but no IP addresses assigned — we do not have the access to configure IPs on those interfaces. Only the 2× Mellanox mlx5 NICs have routable IPv4 addresses. Without filtering, the plugin would attempt to use all 10 devices, failing on the 8 that lack IP connectivity for RDMA CM address resolution.

6. Device deduplication was missing

fi_getinfo returns multiple fi_info entries per device (one per capability combination). Without deduplication, the same device name appeared multiple times in the device list, leading to redundant rail creation attempts.

7. NUMA rail selection assumed all topology devices had rails

buildNumaDataRails() iterated over topology->getAllDevices() using indices as rail IDs. After device filtering, the topology still contained all devices but only filtered devices had rails, causing index mismatches and "no PCIe switches assigned" errors.

Setup

Dockerfile:

FROM vllm/vllm-openai-rocm:v0.15.1

# Install build dependencies
ENV DEBIAN_FRONTEND=noninteractive
RUN --mount=target=/var/lib/apt/lists,type=cache,sharing=locked \
    --mount=target=/var/cache/apt,type=cache,sharing=locked \
    apt-get update && apt-get install -y \
    # Build essentials
    build-essential \
    cmake \
    pkg-config \
    autoconf \
    automake \
    libtool \
    git \
    clang-format \
    libgtest-dev \
    # UCX dependencies
    libaio-dev \
    liburing-dev \
    # Meson and Ninja for RIXL
    ninja-build \
    # RDMA packages
    libibverbs-dev \
    ibverbs-utils \
    librdmacm-dev \
    rdma-core \
    unzip \
    wget

# Broadcom RDMA drivers
RUN wget https://docs.broadcom.com/docs-and-downloads/ethernet-network-adapters/NXE/Thor2/GCA1/bcm5760x_230.2.52.0a.zip && \
    unzip bcm5760x_230.2.52.0a.zip && \
    cd bcm5760x_230.2.52.0a/drivers_linux/bnxt_rocelib/ && \
    results=$(find -name "libbnxt*.tar.gz") && tar -xf $results && \
    untar_dir=$(find . -maxdepth 1 -type d -name "libbnxt*" \! -name "*.tar.gz" | head -n 1) && cd $untar_dir && sh autogen.sh && ./configure && make && \
    find /usr/lib64/ /usr/lib -name "libbnxt_re-rdmav*.so" -exec mv {} {}.inbox \; && \
    make install all && sudo sh -c "echo /usr/local/lib >> /etc/ld.so.conf" && \
    sudo ldconfig && \
    cp -f bnxt_re.driver /etc/libibverbs.d/ && \
    ibv_devices
RUN rm -rf bcm5760x_230.2.52.0a && \
    rm -rf bcm5760x_230.2.52.0a.zip

# Upgrade pip and install Python tools and Ray
ARG RAY_VERSION=2.54.0
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install --no-cache-dir --upgrade pip && \
    pip install --no-cache-dir \
    aiohttp-cors \
    "meson>=0.64.0" \
    opencensus \
    opentelemetry-api \
    opentelemetry-exporter-otlp \
    opentelemetry-exporter-prometheus \
    opentelemetry-sdk \
    pre-commit \
    "pybind11[global]" \
    "ray[all,default,serve]==${RAY_VERSION}"

# Build and install UCX "develop" version (ROCm-specific)
ENV ROCM_PATH=/opt/rocm
ENV PREFIX=/usr/local
ARG UCX_VERSION=develop
RUN git clone --depth 1 --branch ${UCX_VERSION} https://github.com/ROCm/ucx && \
    cd ucx && \
    ./autogen.sh && \
    mkdir -p build && \
    cd build && \
    ../configure \
        --prefix=${PREFIX} \
        --enable-shared \
        --disable-static \
        --disable-doxygen-doc \
        --enable-optimizations \
        --enable-devel-headers \
        --with-rocm=${ROCM_PATH} \
        --with-verbs \
        --with-dm \
        --enable-mt && \
    make -j$(nproc) && \
    make install && \
    ldconfig

# Verify UCX installation
RUN ucx_info -v

# Build and install libfabric
ARG LIBFABRIC_VERSION=v2.4.x
RUN git clone --depth 1 --branch ${LIBFABRIC_VERSION} https://github.com/ofiwg/libfabric.git && \
    cd libfabric && \
    ./autogen.sh && \
    ./configure \
        --prefix=${PREFIX} \
        --with-rocr=${ROCM_PATH} \
        --enable-rocr-dlopen && \
    make -j$(nproc) && \
    make install && \
    ldconfig

docker-compose.yaml:

services:
  vllm-rixl-ray:
    container_name: rixl-libfabric
    build:
      context: .
      dockerfile: Dockerfile
    entrypoint: ""
    command: tail -f /dev/null

    cap_add:
      - IPC_LOCK
      - NET_ADMIN
      - SYS_PTRACE
    devices:
      - /dev/dri
      - /dev/infiniband
      - /dev/infiniband/rdma_cm 
      - /dev/kfd
    group_add:
      - video
    ipc: host
    network_mode: host
    privileged: true
    security_opt:
      - apparmor=unconfined
      - seccomp=unconfined
    shm_size: 128G

    volumes:
      - ${HOST_HF_HUB_CACHE}:/root/.cache/huggingface
      - ~/repos:/app/repos

Testing

Tested on two AMD MI300X servers (8× GPUs each) connected via Mellanox ConnectX-6 Dx (mlx5_1 → ens51f1np1, 10.162.224.0/20 subnet) using verbs;ofi_rxm with native RDMA and FI_HMEM for GPU memory.

The machines have 8× Broadcom bnxt_re NICs (400G RoCE) with no IPv4 addresses assigned — only IPv6 link-local — and we do not have access to configure IPs on those interfaces. The 2× Mellanox mlx5 NICs are the only ones with routable addresses. NIXL_LIBFABRIC_DEVICE_FILTER is used to select mlx5_1 explicitly, which is a workaround for this topology but demonstrates the filtering mechanism that would be needed on any heterogeneous multi-NIC system.

Target (chi-mi300x-004):

FI_PROVIDER="verbs;ofi_rxm" NIXL_LOG_LEVEL=INFO NIXL_LIBFABRIC_DEVICE_FILTER="mlx5_1" \
  python3 examples/python/expanded_two_peers.py \
  --mode=target --ip=10.162.224.166 --port=4242 \
  --backend=LIBFABRIC --use_cuda=True

target-rxm.log

Initiator (chi-mi300x-003):

FI_PROVIDER="verbs;ofi_rxm" NIXL_LOG_LEVEL=INFO NIXL_LIBFABRIC_DEVICE_FILTER="mlx5_1" \
  python3 examples/python/expanded_two_peers.py \
  --mode=initiator --ip=10.162.224.166 --port=4242 \
  --backend=LIBFABRIC --use_cuda=True

initiator-rxm.log

Results:

• Both sides: Discovered 10 InfiniBand (verbs;ofi_rxm) devices, filtered to 1 via DEVICE_FILTER
• Provider with FI_HMEM support detected: Using provider with FI_HMEM support for rail 0
• Fabric: fabric_attr->name IB-0xfe80000000000000
• ROCr runtime detected: System runtime: ROCr for 8 AMD GPU(s)
• VRAM memory registration via FI_HMEM: CreateBufferChunk on Rail 0 successfully created buffer chunk (32 MB, mr_key assigned)
• 1024 recv requests pre-posted successfully
• 4 READs + 2 WRITEs completed successfully
• Target: Target data verification passed (zeros found in last 4 tensors)
• Initiator: Initiator final data verification passed
• Non-uniform NIC speed warnings (expected: 8× bnxt_re at 400G vs 2× mlx5 at 100G) — cosmetic, does not affect correctness when using DEVICE_FILTER