Skip to content

fix(vllm-performance): propagate package dependencies into Ray worker runtime environment - #1264

Open
mgazz wants to merge 11 commits into
IBM:mainfrom
mgazz:fix-ray-worker-runtime
Open

fix(vllm-performance): propagate package dependencies into Ray worker runtime environment#1264
mgazz wants to merge 11 commits into
IBM:mainfrom
mgazz:fix-ray-worker-runtime

Conversation

@mgazz

@mgazz mgazz commented Jul 22, 2026

Copy link
Copy Markdown
Member

This pull request adds propagation of the ray package dependency from the job environment into the Ray worker's runtime environment.

Previously, only ado-vllm-performance and ado-core were extracted from the job environment; now ray itself is also extracted and explicitly included in the worker dependencies, ensuring version consistency between the job environment and the Ray workers.

Fixes #1257

Comment thread plugins/actuators/vllm_performance/ado_actuators/vllm_performance/actuator.py Outdated
@AlessandroPomponio

Copy link
Copy Markdown
Collaborator

You're now starting to get a multitude of packages to be passed down to the runtime env of the task. I believe you might also need to pass down the vllm version (see #1214), so I'm wondering whether it's best to drop this per-package approach and just merge the dictionaries and optionally guard against passing down certain packages (basically having an opt-out approach instead of opt-in)

@mgazz
mgazz force-pushed the fix-ray-worker-runtime branch from bb2db27 to 1493a83 Compare July 23, 2026 14:04
@mgazz
mgazz requested a review from christian-pinto July 23, 2026 14:04
@mgazz mgazz changed the title fix(vllm-performance): propagate ray package dependency into Ray worker runtime environment fix(vllm-performance): propagate package dependencies into Ray worker runtime environment Jul 23, 2026
@mgazz

mgazz commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

Following @AlessandroPomponio suggestion I went for a compromise. I kept the same control for target packages (ray, vllm, guidellm) and generalised the way we set dependencies in the runtime environment.

@mgazz mgazz left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting the following in the remote.yaml

  fromPyPI:
    - ray==2.55.1
    - vllm[bench]==0.24.0

Generates a test environment with the following dependencies

(VLLMPerformanceTest pid=13618, ip=10.129.26.19) 2026-07-23 08:21:03,246 DEBUG AsyncIO Thread: default ado_actuators.vllm_performance.actuator: submit : Experiment (test-agentic-tool-calling) - Ray task environment: {'uv': ['${RAY_RUNTIME_ENV_CREATE_WORKING_DIR}/ado_core-2.0.1.dev44+g4d2dce39.d20260723151103.dirty-py3-none-any.whl', '${RAY_RUNTIME_ENV_CREATE_WORKING_DIR}/ado_vllm_performance-1.9.7.dev42+g4d2dce39.d20260723151103.dirty-py3-none-any.whl[vllm]', 'ray==2.55.1', 'vllm[bench]==0.24.0']}

@mgazz
mgazz dismissed christian-pinto’s stale review July 23, 2026 15:53

Changes have been addressed

Comment thread ado/utilities/environment.py Outdated
mgazz added 10 commits July 24, 2026 14:43
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
…prevent substring cross-match

Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
…l dependencies the same way

Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
…allback to unpublished dev version

Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
…ckage_specs_from_job_env

Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
…m fallback pinning

Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
@mgazz
mgazz force-pushed the fix-ray-worker-runtime branch from 8f1a1d9 to 492d1e9 Compare July 24, 2026 13:43
…s_from_job_env

Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
@mgazz
mgazz requested a review from christian-pinto July 24, 2026 14:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(vllm-performance): Ray version mismatch when launching vllm_performance benchmark tasks on a remote cluster

3 participants