Skip to content

Commit 494e6a9

Browse files
micmelessemicmelesse
committed
jupyter notebook
1 parent 1640eff commit 494e6a9

10 files changed

Lines changed: 49439 additions & 118 deletions

File tree

‎benchmark/llama70b/Dockerfile.baseline‎

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
1-
# Baseline: vLLM on current upstream main with the iris all-reduce path off.
2-
# Each repo is cloned at a fixed commit and installed; aiter's runtime kernels are
3-
# prebuilt into the image. The container only runs the workload.
1+
# Baseline: the AMD production config. VLLM_ROCM_USE_AITER is left unset (vLLM
2+
# default False), so broad aiter compute is off and the all-reduce is the
3+
# QuickReduce-INT4 path (baked below via VLLM_ROCM_QUICK_REDUCE_QUANTIZATION). The
4+
# exp image flips to the iris one-shot all-reduce; the A/B isolates that. Each repo
5+
# is cloned at a fixed commit and installed; aiter's runtime kernels are prebuilt
6+
# into the image. The container only runs the workload.
47
FROM rocm/vllm-dev:nightly_main_20260612
58

69
ENV MAX_JOBS=32
@@ -45,4 +48,15 @@ RUN printf '%s\n' \
4548
> /tmp/prebuild_aiter.py \
4649
&& GPU_ARCHS="${GPU_ARCHS}" python3 /tmp/prebuild_aiter.py
4750

51+
# Server env, baked into the image (the scripts export nothing - the image IS the
52+
# environment). Shared by both arms: the AMD production config (aiter MHA off,
53+
# QuickReduce INT4 all-reduce) plus operational settings (NCCL logging, RPC/ready
54+
# timeouts). baseline bakes NO per-arm behavior flag: VLLM_ROCM_USE_AITER stays
55+
# unset (vLLM default False) = broad aiter off, QuickReduce all-reduce.
56+
ENV VLLM_ROCM_USE_AITER_MHA=0
57+
ENV VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4
58+
ENV NCCL_DEBUG=INFO
59+
ENV VLLM_RPC_TIMEOUT=1800000
60+
ENV VLLM_ENGINE_READY_TIMEOUT_S=3600
61+
4862
CMD ["/bin/bash"]

‎benchmark/llama70b/Dockerfile.exp‎

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,8 @@ FROM rocm/vllm-dev:nightly_main_20260612
66
ENV MAX_JOBS=32
77
ARG GPU_ARCHS="gfx950" # MI350
88
ARG TRITON_COMMIT="6898a3288c28d50d1f4e1f91aa5867ca0d1f3c3b"
9-
ARG VLLM_COMMIT="880341fa9e0909f847d8644a838dce5f2152d65d" # micmelesse/vllm allreduce_only
10-
ARG AITER_COMMIT="d0002d20185e9e4656f746e56ce2e766cea1c1d8" # ROCm/aiter micmelesse/allreduce_only
9+
ARG VLLM_COMMIT="4e61409d254f761994b7498b20c78c3b5ecff8ae" # micmelesse/vllm allreduce_only (make_communicator gets both cpu+device groups)
10+
ARG AITER_COMMIT="51f6c3197f775bdf93272f94335a96f534334ed4" # ROCm/aiter micmelesse/allreduce_only (make_communicator(cpu_group, device_group, ...))
1111
ARG IRIS_COMMIT="04ad32409251c7a7b55a39c8bae4629154e89c2c" # ROCm/iris muhaawad/one-shot-vllm
1212

1313
RUN git clone https://github.com/micmelesse/vllm.git /src/vllm \
@@ -45,6 +45,21 @@ RUN printf '%s\n' \
4545
> /tmp/prebuild_aiter.py \
4646
&& GPU_ARCHS="${GPU_ARCHS}" python3 /tmp/prebuild_aiter.py
4747

48+
# Server env, baked into the image (the scripts export nothing - the image IS the
49+
# environment). Shared block is IDENTICAL to Dockerfile.baseline: the AMD production
50+
# config (aiter MHA off, QuickReduce INT4) plus operational settings.
51+
ENV VLLM_ROCM_USE_AITER_MHA=0
52+
ENV VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4
53+
ENV NCCL_DEBUG=INFO
54+
ENV VLLM_RPC_TIMEOUT=1800000
55+
ENV VLLM_ENGINE_READY_TIMEOUT_S=3600
56+
57+
# The per-arm behavior - the only env that differs from baseline. All three are
58+
# required: vLLM gates the comms backend on is_comms_enabled() = USE_AITER and
59+
# USE_AITER_COMMS, so without the master toggle iris silently falls back to
60+
# QuickReduce; and make_communicator has no default, so the backend must be declared.
61+
ENV VLLM_ROCM_USE_AITER=1
4862
ENV VLLM_ROCM_USE_AITER_COMMS=1
63+
ENV AITER_COMMS_BACKEND=iris
4964

5065
CMD ["/bin/bash"]

‎benchmark/llama70b/README.md‎

Lines changed: 45 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,14 @@
11
# iris all-reduce reproducer
22

3-
Correctness and performance numbers for the iris one-shot collectives on
4-
Llama-3.3-70B-FP8 at TP=8. Each arm is a baked image (vllm, aiter, iris pinned);
5-
the scripts only run the workload.
3+
Correctness and performance for the iris one-shot collectives on
4+
Llama-3.3-70B-FP8 at TP=8. Two arms, each a baked image (vllm, aiter, iris pinned):
5+
6+
- **baseline** (`Dockerfile.baseline`) - AMD production config: broad aiter off,
7+
the all-reduce is QuickReduce INT4.
8+
- **exp** (`Dockerfile.exp`) - the iris one-shot all-reduce (`AITER_COMMS_BACKEND=iris`).
9+
10+
The A/B isolates the all-reduce path. All vLLM behavior env is baked into the images;
11+
the scripts only run the workload against the server.
612

713
## Requirements
814

@@ -17,21 +23,47 @@ RUN="docker run --rm -it \
1723
-v $(pwd):/repro -w /repro"
1824
```
1925

20-
## Baseline
26+
## Run
27+
28+
Build each image once, then run the commands you need against it. The A/B is the
29+
baseline image vs the exp image; compare their outputs.
2130

2231
```sh
2332
docker build -f Dockerfile.baseline -t iris-repro:baseline .
24-
$RUN iris-repro:baseline ./bench.sh # performance
25-
$RUN iris-repro:baseline ./test.sh # correctness
33+
docker build -f Dockerfile.exp -t iris-repro:exp .
34+
35+
$RUN iris-repro:baseline ./bench.sh # perf: serving metrics (TTFT/TPOT/E2EL/throughput)
36+
$RUN iris-repro:exp ./bench.sh
37+
38+
$RUN iris-repro:baseline ./profile.sh # traces + per-kernel tables (what data.csv needs)
39+
$RUN iris-repro:exp ./profile.sh
40+
41+
$RUN iris-repro:baseline ./eval.sh # gsm8k accuracy gate (correctness)
42+
$RUN iris-repro:exp ./eval.sh
43+
44+
$RUN iris-repro:exp ./test.sh # iris collective correctness (exp stack, no server)
2645
```
2746

28-
## Experiment
47+
The four commands share one server config (`_serve.sh`) so perf, traces, and the
48+
correctness gate all describe the same server. Each writes RAW artifacts under
49+
`output/<arm>/`: `results/` (the workload result JSON), `profile/{summary,traces,ir}/`
50+
(profiling run), and `arm.json` (the resolved operating point + installed code SHAs).
51+
52+
Default operating point is `decode64` (8192 in / 1024 out, concurrency 64), warm
53+
(`WARMUP=64`, warmup requests excluded from the metrics). Knobs: `WORKLOAD=confluence`
54+
(the guide's 1024/1024/conc-4 example), `WARMUP=0` (cold), `DATA=real` (ShareGPT).
55+
56+
## Analysis
57+
58+
`data.csv` is the flat, long-format extract of every arm's raw artifacts (one row per
59+
fact: e2e metrics, profiler-table per-kernel times, trace per-kernel times). `report.ipynb`
60+
renders the A/B from it (pandas + matplotlib only). It ships pre-built, so you can open the
61+
notebook directly.
62+
63+
To rebuild it from arms you ran yourself, point `preprocess.py` at their output dirs (each
64+
`output/<arm>-<command>/` holds `arm.json` + `results/` + `profile/`):
2965

3066
```sh
31-
docker build -f Dockerfile.exp -t iris-repro:exp .
32-
$RUN iris-repro:exp ./bench.sh # performance
33-
$RUN iris-repro:exp ./test.sh # correctness
67+
python preprocess.py output/* # parses each arm dir -> data.csv (reads arm.json for the labels)
68+
jupyter nbconvert --to notebook --execute report.ipynb # or just open report.ipynb
3469
```
35-
36-
`bench.sh` writes serving metrics (TTFT/TPOT/E2EL/throughput) to `./output/`;
37-
add `EVAL=1` for a gsm8k accuracy pass or `PROFILE=1` for profiler traces.

0 commit comments

Comments
 (0)