Skip to content

ci: authenticate HuggingFace model prefetch to avoid rate limits #1948

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
NirWolfer wants to merge 1 commit into
base: main
Choose a base branch
from
Open

ci: authenticate HuggingFace model prefetch to avoid rate limits #1948

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .ci/docs/ci-overview.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ their own nightly/manual trigger. They split into two groups:
### `nixl-ci-build-wheel` (dispatcher-triggered)
- **Config:** `.ci/jenkins/lib/build-wheel-matrix.yaml`
- **What it does:** Builds NIXL Python wheels for each Python version × architecture combination. Uses a two-stage `contrib/Dockerfile.manylinux` build: the `wheel_base` stage (all slow deps: UCX, gRPC, Rust, etc.) is pre-built and cached in Artifactory under `CI_IMAGE_TAG`; PR builds pull this pre-built image and run only the `wheel` stage (~16 steps). PR builds also pass `--torch-versions` (set via the `TORCH_VERSIONS` env in the matrix file) to limit the torch extension builds to the latest version. x86_64 wheels build in the `manylinux` (podman-in-container) runner.
- **vLLM/SGLang NIXL sanity (aarch64):** the same job also gates a 1-prefill/1-decode NIXL KV-transfer sanity for vLLM and SGLang. Each framework runs as its own aarch64 branch (`build_helper_vllm` / `build_helper_sglang`): build the aarch64 wheels (reusing the cached `wheel_base`), layer them onto the pinned framework base image (`.ci/dockerfiles/Dockerfile.{vllm,sglang}-base`, built as `category: tool` by ci-demo), then run the sanity on the `gb200nvl72_ci` dlcluster SLURM partition over SSH (`.gitlab/test_vllm_sglang_sanity.sh`). SGLang additionally asserts a gsm8k accuracy floor on `Qwen/Qwen3-8B`. The aarch64 wheels are built once per framework branch (duplicated) so the two flows stay separate, labeled branches in the Jenkins UI.
- **vLLM/SGLang NIXL sanity (aarch64):** the same job also gates a 1-prefill/1-decode NIXL KV-transfer sanity for vLLM and SGLang. Each framework runs as its own aarch64 branch (`build_helper_vllm` / `build_helper_sglang`): build the aarch64 wheels (reusing the cached `wheel_base`), layer them onto the pinned framework base image (`.ci/dockerfiles/Dockerfile.{vllm,sglang}-base`, built as `category: tool` by ci-demo), then run the sanity on the `gb200nvl72_ci` dlcluster SLURM partition over SSH (`.gitlab/test_vllm_sglang_sanity.sh`). SGLang additionally asserts a gsm8k accuracy floor on `Qwen/Qwen3-8B`. The aarch64 wheels are built once per framework branch (duplicated) so the two flows stay separate, labeled branches in the Jenkins UI. Model prefetch authenticates via the `svc-nixl-hf-token` Jenkins credential (bound as `HF_TOKEN`) to avoid HuggingFace anonymous rate limits on the shared CI egress IP.
- **`CI_IMAGE_TAG`:** Same convention as the other five matrix files (also tags `build_helper_*` and the sanity `*-nixl-base` images). `contrib/Dockerfile.manylinux` is part of the `CI_FILES` list in `cidemo-init.sh`, so changing it (or any other CI file) without bumping `CI_IMAGE_TAG` in all six matrix YAMLs will fail the pre-commit check.

### `nixl-ci-build-container` (standalone)
Expand Down
6 changes: 4 additions & 2 deletions .ci/jenkins/lib/build-wheel-matrix.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,8 @@ credentials:
- credentialsId: 'svc-nixl-new-artifactory-token'
usernameVariable: 'ARTIFACTORY_USER'
passwordVariable: 'ARTIFACTORY_TOKEN'
- credentialsId: 'svc-nixl-hf-token'
variable: 'HF_TOKEN'
Comment on lines +79 to +80

@coderabbitai coderabbitai Bot Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -n -C 5 'credentials:|HF_TOKEN|testScript:' .ci/jenkins .gitlab

Repository: ai-dynamo/nixl

Length of output: 34719

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== build-wheel-matrix context =="
sed -n '1,120p' .ci/jenkins/lib/build-wheel-matrix.yaml
echo
echo "== sanity steps =="
sed -n '330,390p' .ci/jenkins/lib/build-wheel-matrix.yaml
echo
echo "== slurmCI usage elsewhere with env vars in testScript =="
rg -n -C 3 'testScript: ".*\$\{[A-Z_]+\}.*"|module: slurmCI' .ci/jenkins/lib .ci/jenkins/pipeline

Repository: ai-dynamo/nixl

Length of output: 27723

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== build-wheel-matrix step map =="
ast-grep outline .ci/jenkins/lib/build-wheel-matrix.yaml --view expanded | sed -n '1,220p'
echo
echo "== slurmCI references =="
rg -n -C 4 '\bslurmCI\b|testScript:|credentialsId:' .ci/jenkins .gitlab | sed -n '1,260p'
echo
echo "== any docs or code for command logging / argument passing =="
rg -n -C 3 'command.*log|log.*command|testScript|slurmCI.*args|env.*credentials|mask' .ci/jenkins .gitlab README.md

Repository: ai-dynamo/nixl

Length of output: 30651

Scope HF_TOKEN to the sanity steps only.
This top-level credentials block makes the token available to the whole wheel job, including the build/image stages that can run PR-controlled code. Bind it only around the vLLM/SGLang sanity steps, or forward it without exposing it to the rest of the matrix.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.ci/jenkins/lib/build-wheel-matrix.yaml around lines 79 - 80, Restrict the
HF_TOKEN binding from the top-level wheel job credentials to only the
vLLM/SGLang sanity steps in the matrix. Update the relevant sanity-step wrapper
or invocation so the token is available there while remaining unavailable to
build and image stages that execute PR-controlled code.


# =============================================================================
# Kubernetes Resource Configuration
Expand Down Expand Up @@ -355,7 +357,7 @@ steps:
run: run
args:
jobIdFile: "${JOB_ID_FILE_ROOT}/job_id_${fw}_${BUILD_NUMBER}.txt"
testScript: ".gitlab/test_vllm_sglang_sanity.sh ${fw}"
testScript: "HF_TOKEN=${HF_TOKEN} .gitlab/test_vllm_sglang_sanity.sh ${fw}"
headNode: "${SLURM_HEAD_NODE}"
headUser: "${SLURM_HEAD_USER}"
dockerImage: "${registry_host}#${registry_path}/pr/${arch}/${fw}-nixl:${BUILD_NUMBER}"
Expand All @@ -371,7 +373,7 @@ steps:
run: run
args:
jobIdFile: "${JOB_ID_FILE_ROOT}/job_id_${fw}_${BUILD_NUMBER}.txt"
testScript: ".gitlab/test_vllm_sglang_sanity.sh ${fw}"
testScript: "HF_TOKEN=${HF_TOKEN} .gitlab/test_vllm_sglang_sanity.sh ${fw}"
headNode: "${SLURM_HEAD_NODE}"
headUser: "${SLURM_HEAD_USER}"
dockerImage: "${registry_host}#${registry_path}/pr/${arch}/${fw}-nixl:${BUILD_NUMBER}"
Expand Down
2 changes: 1 addition & 1 deletion .gitlab/test_vllm_sglang_sanity.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ python3 -c "import nixl; from importlib.metadata import version; print('nixl', v
# path (burning the health-check timeout); an explicit download fails fast and clearly.
# huggingface_hub ships in both framework images, so no extra install is needed.
log "prefetching ${MODEL}"
python3 -c "from huggingface_hub import snapshot_download; snapshot_download('${MODEL}')"
python3 -c "from huggingface_hub import snapshot_download; snapshot_download('${MODEL}', token='${HF_TOKEN:-}' or None)"

if [ "$FRAMEWORK" = "vllm" ]; then
python3 -c "from importlib.metadata import version; print('vllm', version('vllm'))"
Expand Down
Loading