diff --git a/.ci/docs/ci-overview.md b/.ci/docs/ci-overview.md index ba85f98781..ec0c8759b3 100644 --- a/.ci/docs/ci-overview.md +++ b/.ci/docs/ci-overview.md @@ -169,7 +169,7 @@ their own nightly/manual trigger. They split into two groups: ### `nixl-ci-build-wheel` (dispatcher-triggered) - **Config:** `.ci/jenkins/lib/build-wheel-matrix.yaml` - **What it does:** Builds NIXL Python wheels for each Python version × architecture combination. Uses a two-stage `contrib/Dockerfile.manylinux` build: the `wheel_base` stage (all slow deps: UCX, gRPC, Rust, etc.) is pre-built and cached in Artifactory under `CI_IMAGE_TAG`; PR builds pull this pre-built image and run only the `wheel` stage (~16 steps). PR builds also pass `--torch-versions` (set via the `TORCH_VERSIONS` env in the matrix file) to limit the torch extension builds to the latest version. x86_64 wheels build in the `manylinux` (podman-in-container) runner. -- **vLLM/SGLang NIXL sanity (aarch64):** the same job also gates a 1-prefill/1-decode NIXL KV-transfer sanity for vLLM and SGLang. Each framework runs as its own aarch64 branch (`build_helper_vllm` / `build_helper_sglang`): build the aarch64 wheels (reusing the cached `wheel_base`), layer them onto the pinned framework base image (`.ci/dockerfiles/Dockerfile.{vllm,sglang}-base`, built as `category: tool` by ci-demo), then run the sanity on the `gb200nvl72_ci` dlcluster SLURM partition over SSH (`.gitlab/test_vllm_sglang_sanity.sh`). SGLang additionally asserts a gsm8k accuracy floor on `Qwen/Qwen3-8B`. The aarch64 wheels are built once per framework branch (duplicated) so the two flows stay separate, labeled branches in the Jenkins UI. +- **vLLM/SGLang NIXL sanity (aarch64):** the same job also gates a 1-prefill/1-decode NIXL KV-transfer sanity for vLLM and SGLang. Each framework runs as its own aarch64 branch (`build_helper_vllm` / `build_helper_sglang`): build the aarch64 wheels (reusing the cached `wheel_base`), layer them onto the pinned framework base image (`.ci/dockerfiles/Dockerfile.{vllm,sglang}-base`, built as `category: tool` by ci-demo), then run the sanity on the `gb200nvl72_ci` dlcluster SLURM partition over SSH (`.gitlab/test_vllm_sglang_sanity.sh`). SGLang additionally asserts a gsm8k accuracy floor on `Qwen/Qwen3-8B`. The aarch64 wheels are built once per framework branch (duplicated) so the two flows stay separate, labeled branches in the Jenkins UI. Model prefetch authenticates via the `svc-nixl-hf-token` Jenkins credential (bound as `HF_TOKEN`) to avoid HuggingFace anonymous rate limits on the shared CI egress IP. - **`CI_IMAGE_TAG`:** Same convention as the other five matrix files (also tags `build_helper_*` and the sanity `*-nixl-base` images). `contrib/Dockerfile.manylinux` is part of the `CI_FILES` list in `cidemo-init.sh`, so changing it (or any other CI file) without bumping `CI_IMAGE_TAG` in all six matrix YAMLs will fail the pre-commit check. ### `nixl-ci-build-container` (standalone) diff --git a/.ci/jenkins/lib/build-wheel-matrix.yaml b/.ci/jenkins/lib/build-wheel-matrix.yaml index e7cc761ec5..5289a4eaf0 100644 --- a/.ci/jenkins/lib/build-wheel-matrix.yaml +++ b/.ci/jenkins/lib/build-wheel-matrix.yaml @@ -76,6 +76,8 @@ credentials: - credentialsId: 'svc-nixl-new-artifactory-token' usernameVariable: 'ARTIFACTORY_USER' passwordVariable: 'ARTIFACTORY_TOKEN' + - credentialsId: 'svc-nixl-hf-token' + variable: 'HF_TOKEN' # ============================================================================= # Kubernetes Resource Configuration @@ -355,7 +357,7 @@ steps: run: run args: jobIdFile: "${JOB_ID_FILE_ROOT}/job_id_${fw}_${BUILD_NUMBER}.txt" - testScript: ".gitlab/test_vllm_sglang_sanity.sh ${fw}" + testScript: "HF_TOKEN=${HF_TOKEN} .gitlab/test_vllm_sglang_sanity.sh ${fw}" headNode: "${SLURM_HEAD_NODE}" headUser: "${SLURM_HEAD_USER}" dockerImage: "${registry_host}#${registry_path}/pr/${arch}/${fw}-nixl:${BUILD_NUMBER}" @@ -371,7 +373,7 @@ steps: run: run args: jobIdFile: "${JOB_ID_FILE_ROOT}/job_id_${fw}_${BUILD_NUMBER}.txt" - testScript: ".gitlab/test_vllm_sglang_sanity.sh ${fw}" + testScript: "HF_TOKEN=${HF_TOKEN} .gitlab/test_vllm_sglang_sanity.sh ${fw}" headNode: "${SLURM_HEAD_NODE}" headUser: "${SLURM_HEAD_USER}" dockerImage: "${registry_host}#${registry_path}/pr/${arch}/${fw}-nixl:${BUILD_NUMBER}" diff --git a/.gitlab/test_vllm_sglang_sanity.sh b/.gitlab/test_vllm_sglang_sanity.sh index d872a3cf8c..d47cd49479 100755 --- a/.gitlab/test_vllm_sglang_sanity.sh +++ b/.gitlab/test_vllm_sglang_sanity.sh @@ -96,7 +96,7 @@ python3 -c "import nixl; from importlib.metadata import version; print('nixl', v # path (burning the health-check timeout); an explicit download fails fast and clearly. # huggingface_hub ships in both framework images, so no extra install is needed. log "prefetching ${MODEL}" -python3 -c "from huggingface_hub import snapshot_download; snapshot_download('${MODEL}')" +python3 -c "from huggingface_hub import snapshot_download; snapshot_download('${MODEL}', token='${HF_TOKEN:-}' or None)" if [ "$FRAMEWORK" = "vllm" ]; then python3 -c "from importlib.metadata import version; print('vllm', version('vllm'))"