You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This PR introduces a versioning scheme for experiments in the vllm-performance actuator. All existing experiments have been marked as deprecated, and a new set of cleanly named, versioned replacement experiments has been added. The actuator code and executor have been updated to recognise both old and new identifiers, ensuring backward compatibility while signalling that users should migrate to the new names.
Deprecated legacy experiments: all existing experiment identifiers (e.g. test-deployment-v1, test-endpoint-guidellm-v1, and all geospatial variants) have been marked deprecated: true in the experiment YAML catalog.
New versioned experiments added: a full set of replacement experiments with cleaner identifiers has been introduced (vllm-bench-deployment, vllm-bench-endpoint, guidellm-bench-deployment, guidellm-bench-endpoint, and all geospatial equivalents), each carrying an explicit version: 1.0.0 field.
Actuator and executor updated: the routing logic in the actuator and experiment executor now recognises both old and new experiment identifiers, so existing spaces and operations remain functional.
Documentation updated: the actuator README and website docs reflect the new experiment names and deprecation notices.
Impact
Users can continue using existing discovery spaces and operations without modification — all legacy experiment identifiers remain functional. New spaces should use the new versioned experiment identifiers. The deprecated: true flag allows tooling to surface warnings for spaces still referencing the old names.
The reason will be displayed to describe this comment to others. Learn more.
In addition to fixing the test we must make sure we can still load data from the other ones and update them. we must also consider if it's possible to add a validator that changes entity data or something from the old experiment to the new one - given that nothing but a rename changed -
We must also update the documentation and examples
christian-pinto
changed the title
feat(vllm-performance): Enabled experiments versioning and deprecated old experiments
feat(vllm-performance): Update experiment naming and deprecated old experiments
Jul 28, 2026
christian-pinto
changed the title
feat(vllm-performance): Update experiment naming and deprecated old experiments
feat(vllm-performance): Update experiment naming and deprecate old experiments
Jul 28, 2026
AlessandroPomponio
changed the title
feat(vllm-performance): Update experiment naming and deprecate old experiments
feat(vllm-performance): update experiment naming and versioning; deprecate old experiments
Jul 28, 2026
AlessandroPomponio
changed the title
feat(vllm-performance): update experiment naming and versioning; deprecate old experiments
refactor(vllm-performance): update experiment naming and versioning; deprecate old experiments
Jul 28, 2026
also consider if it's possible to add a validator that changes entity data or something from the old experiment to the new one
We just have to pay the price of changing the names. We shouldn't allow retroactive edit of spaces and the data collected from them as this breaks many guarantees we try to keep (spaces don't change, the schema implied by a space (observed property names etc.) doesn't change, the data associated with the space is the data collected by the ops etc.)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces a versioning scheme for experiments in the
vllm-performanceactuator. All existing experiments have been marked as deprecated, and a new set of cleanly named, versioned replacement experiments has been added. The actuator code and executor have been updated to recognise both old and new identifiers, ensuring backward compatibility while signalling that users should migrate to the new names.Resolves #1203
High-level Changes
test-deployment-v1,test-endpoint-guidellm-v1, and all geospatial variants) have been markeddeprecated: truein the experiment YAML catalog.vllm-bench-deployment,vllm-bench-endpoint,guidellm-bench-deployment,guidellm-bench-endpoint, and all geospatial equivalents), each carrying an explicitversion: 1.0.0field.Impact
Users can continue using existing discovery spaces and operations without modification — all legacy experiment identifiers remain functional. New spaces should use the new versioned experiment identifiers. The
deprecated: trueflag allows tooling to surface warnings for spaces still referencing the old names.