refactor(vllm-performance): update experiment naming and versioning; deprecate old experiments

docs/resources/sample-stores.md

@@ -275,7 +275,7 @@ copyFrom:
       generatorIdentifier: "vllm-benchmark-run"
       identifierColumn: "config"
       experiments:
-        - experimentIdentifier: "test-deployment-v1"
+        - experimentIdentifier: "vllm-bench-deployment"
           actuatorIdentifier: "vllm_performance" # Specify the actual actuator
           propertyFormat: "target" # if the columns for observed properties use target property names or observed property names
 ```

docs/user-guide/actuators/vllm-performance.md

@@ -50,41 +50,72 @@ The `vllm_performance` actuator implements twelve experiments:
 
 **Standard LLM Benchmarking:**
 
-- `test-deployment-v1`: This experiment can test the full vLLM workload
+- `vllm-bench-deployment`: This experiment can test the full vLLM workload
   configuration, including resource requests and server deployment
   configuration. It deploys servers with given configuration on kubernetes and
   runs vLLM's built-in benchmarking tool (`vllm bench serve`) on them with the
   given parameters.
-- `test-endpoint-v1`: This experiment is equivalent to running
+- `vllm-bench-endpoint`: This experiment is equivalent to running
   `vllm bench serve` against an endpoint.
-- `test-deployment-guidellm-v1`: Similar to `test-deployment-v1`, but uses
+- `guidellm-bench-deployment`: Similar to `vllm-bench-deployment`, but uses
   GuideLLM (`guidellm benchmark run`) for benchmarking instead of vLLM's
   built-in benchmarking tool.
-- `test-endpoint-guidellm-v1`: Similar to `test-endpoint-v1`, but uses GuideLLM
+- `guidellm-bench-endpoint`: Similar to `vllm-bench-endpoint`, but uses GuideLLM
   (`guidellm benchmark run`) for benchmarking instead of vLLM's built-in
   benchmarking tool.
 
 **Geospatial Model Benchmarking:**
 
-- `test-geospatial-deployment-v1`: Deploy and benchmark geospatial models
+- `geospatial-vllm-bench-deployment`: Deploy and benchmark geospatial models
   (IBM-NASA Prithvi) using pre-packaged datasets for flood detection tasks with
   vLLM's built-in benchmarking tool.
-- `test-geospatial-endpoint-v1`: Benchmark existing geospatial model endpoints
+- `geospatial-vllm-bench-endpoint`: Benchmark existing geospatial model endpoints
   using pre-packaged datasets with vLLM's built-in benchmarking tool.
-- `test-geospatial-deployment-guidellm-v1`: Deploy and benchmark geospatial
+- `geospatial-guidellm-bench-deployment`: Deploy and benchmark geospatial
   models using pre-packaged datasets with GuideLLM.
-- `test-geospatial-endpoint-guidellm-v1`: Benchmark existing geospatial model
+- `geospatial-guidellm-bench-endpoint`: Benchmark existing geospatial model
   endpoints using pre-packaged datasets with GuideLLM.
-- `test-geospatial-deployment-custom-dataset-v1`: Deploy and benchmark
+- `geospatial-vllm-bench-deployment-custom-dataset`: Deploy and benchmark
   geospatial models with custom datasets using vLLM's built-in benchmarking
   tool.
-- `test-geospatial-endpoint-custom-dataset-v1`: Benchmark existing geospatial
+- `geospatial-vllm-bench-endpoint-custom-dataset`: Benchmark existing geospatial
   model endpoints with custom datasets using vLLM's built-in benchmarking tool.
-- `test-geospatial-deployment-guidellm-custom-dataset-v1`: Deploy and benchmark
+- `geospatial-guidellm-bench-deployment-custom-dataset`: Deploy and benchmark
   geospatial models with custom datasets using GuideLLM.
-- `test-geospatial-endpoint-guidellm-custom-dataset-v1`: Benchmark existing
+- `geospatial-guidellm-bench-endpoint-custom-dataset`: Benchmark existing
   geospatial model endpoints with custom datasets using GuideLLM.
 
+> [!NOTE] Deprecated experiment names
+>
+> Earlier versions of the actuator used different names for the same experiments.
+> The behaviour and output are identical — only the names changed. If you have
+> existing discoveryspace or operation YAML files that reference the old names,
+> update them to the current names shown above.
+>
+> **Standard LLM experiments:**
+>
+> | Deprecated name | Current name |
+> | --- | --- |
+> | `test-deployment-v1` | `vllm-bench-deployment` |
+> | `test-endpoint-v1` | `vllm-bench-endpoint` |
+> | `test-deployment-guidellm-v1` | `guidellm-bench-deployment` |
+> | `test-endpoint-guidellm-v1` | `guidellm-bench-endpoint` |
+>
+> **Geospatial experiments:**
+>
+> | Deprecated name | Current name |
+> | --- | --- |
+> | `test-geospatial-deployment-v1` | `geospatial-vllm-bench-deployment` |
+> | `test-geospatial-endpoint-v1` | `geospatial-vllm-bench-endpoint` |
+> | `test-geospatial-deployment-guidellm-v1` | `geospatial-guidellm-bench-deployment` |
+> | `test-geospatial-endpoint-guidellm-v1` | `geospatial-guidellm-bench-endpoint` |
+> | `test-geospatial-deployment-custom-dataset-v1` | `geospatial-vllm-bench-deployment-custom-dataset` |
+> | `test-geospatial-endpoint-custom-dataset-v1` | `geospatial-vllm-bench-endpoint-custom-dataset` |
+> | `test-geospatial-deployment-guidellm-custom-dataset-v1` | `geospatial-guidellm-bench-deployment-custom-dataset` |
+> | `test-geospatial-endpoint-guidellm-custom-dataset-v1` | `geospatial-guidellm-bench-endpoint-custom-dataset` |
+
+<!-- markdownlint-disable-next-line MD028 -->
+
 > [!NOTE] Threadpool Support for Geospatial Models
 >
 > Geospatial experiments support threadpool pre/post processing for improved
@@ -149,7 +180,7 @@ entity:
   request_rate: 50
 experiments:
   - actuatorIdentifier: vllm_performance
-    experimentIdentifier: test-endpoint-v1
+    experimentIdentifier: vllm-bench-endpoint
 ```
 
 Then run:
@@ -195,7 +226,7 @@ entity:
   request_rate: 10
 experiments:
   - actuatorIdentifier: vllm_performance
-    experimentIdentifier: test-deployment-v1
+    experimentIdentifier: vllm-bench-deployment
 ```
 
 Then run:
@@ -281,8 +312,8 @@ ado create actuatorconfiguration -f vllm_config.yaml
 > [!WARNING] GPU type
 >
 > The GPU type to use in an experiment is set via the experiment itself
-> (test-deployment-v1). **Do not** set this via the `node_selector` parameter of
-> the configuration.
+> (`vllm-bench-deployment`). **Do not** set this via the `node_selector`
+> parameter of the configuration.
 
 <!-- markdownlint-disable-next-line MD028 -->
 
@@ -339,10 +370,10 @@ experiments can execute even if these packages are not pre-installed in your Ray
 cluster or local environment. The actuator intelligently selects which tool to
 install based on the experiment type:
 
-- For experiments using vLLM's built-in benchmarking (`test-deployment-v1`,
-  `test-endpoint-v1`), it installs `ado-vllm-performance[vllm]`
-- For experiments using GuideLLM (`test-deployment-guidellm-v1`,
-  `test-endpoint-guidellm-v1`), it installs `ado-vllm-performance[guidellm]`
+- For experiments using vLLM's built-in benchmarking (`vllm-bench-deployment`,
+  `vllm-bench-endpoint`), it installs `ado-vllm-performance[vllm]`
+- For experiments using GuideLLM (`guidellm-bench-deployment`,
+  `guidellm-bench-endpoint`), it installs `ado-vllm-performance[guidellm]`
 
 > [!NOTE] Deployment vs. Benchmark environment
 >
@@ -399,7 +430,7 @@ parameters:
 
 The `in_cluster` option in your `actuatorconfiguration` tells the
 `vllm_performance` actuator how to communicate with the target Kubernetes or
-OpenShift cluster when running `test-deployment-v1`.
+OpenShift cluster when running `vllm-bench-deployment`.
 
 If running `ado` from outside the Kubernetes/OpenShift cluster where the
 deployments will be created, leave `in_cluster: false` (the default).

docs/user-guide/examples/example_yamls/geospatial_flood_detection_space.yaml

@@ -45,4 +45,4 @@ entitySpace:
         #     values: [32, 64]
 measurementSpace:
   - actuatorIdentifier: vllm_performance
-    experimentIdentifier: test-geospatial-deployment-v1
+    experimentIdentifier: geospatial-vllm-bench-deployment

docs/user-guide/examples/index.md

@@ -73,8 +73,8 @@ to full Kubernetes/OpenShift deployments and specialised geospatial models.
 - **[Exploring vLLM deployment configurations](vllm-performance-full.md)** —
   Evaluate different vLLM server deployment configurations (GPU type, batch
   size, memory limits) on Kubernetes/OpenShift by combining the
-  `test-deployment-v1` experiment with the `random_walk` operator.
+  `vllm-bench-deployment` experiment with the `random_walk` operator.
 - **[Benchmarking geospatial models with vLLM](vllm-performance-geospatial.md)**
   — Benchmark IBM-NASA Prithvi geospatial models for Earth observation tasks
   (flood detection, land-use classification) using the
-  `test-geospatial-deployment-v1` experiment.
+  `geospatial-vllm-bench-deployment` experiment.

docs/user-guide/examples/vllm-performance-endpoint.md

@@ -92,7 +92,7 @@ entitySpace:
       interval: 1
 experiments:
   - actuatorIdentifier: vllm_performance
-    experimentIdentifier: test-endpoint-v1
+    experimentIdentifier: vllm-bench-endpoint
 ```
 
 Create the space with:
@@ -227,7 +227,7 @@ and the best region is unlikely to be visited.
 - Try running the same operation with the
   [GuideLLM](https://github.com/vllm-project/guidellm) benchmarking tool by
   setting the `experimentIdentifier` field in the entity space definition to
-  `test-endpoint-guidellm-v1`.
+  `guidellm-bench-endpoint`.
 - Use `ado describe experiment vllm_performance_endpoint` to see what other
   parameters can be explored
 - Try varying **`burstiness`** or **`number_input_tokens`**, or adding them as

docs/user-guide/examples/vllm-performance-full.md

@@ -16,7 +16,7 @@
 > explore the deployment parameter space. In this example:
 >
 > - We will define a space of vLLM deployment configurations to test with the
->   `vllm_performance` actuator's `test-deployment-v1` experiment
+>   `vllm_performance` actuator's `vllm-bench-deployment` experiment
 >   - This experiment can create and characterize a vLLM deployment on
 >     Kubernetes
 > - Use the [`random_walk` operator](../operators/random-walk.md) to explore the
@@ -205,7 +205,7 @@ ado show measurements space --output csv --use-latest > entities.csv
 - Try running the same operation with the
   [GuideLLM](https://github.com/vllm-project/guidellm) benchmarking tool by
   setting the `experimentIdentifier` field in the entity space definition to
-  `test-deployment-guidellm-v1`.
+  `guidellm-bench-deployment`.
 - Try varying **`max_batch_tokens`** or **`gpu_memory_utilization`** to explore
   the impact on throughput.
 - Try creating a different `actuatorconfiguration` with more `max_environments`

docs/user-guide/examples/vllm-performance-geospatial.md

@@ -19,7 +19,7 @@
 > In this example:
 >
 > - We will define a space of geospatial model deployment configurations to test
-> - Use the `test-geospatial-deployment-v1` experiment to create and benchmark
+> - Use the `geospatial-vllm-bench-deployment` experiment to create and benchmark
 >   vLLM deployments serving Prithvi models
 > - Explore how deployment parameters affect inference latency for flood
 >   detection tasks
@@ -65,9 +65,9 @@ experiments:
 ado get experiments --details
 ```
 
-You should see experiments including `test-geospatial-deployment-v1`,
-`test-geospatial-endpoint-v1`, `test-geospatial-deployment-custom-dataset-v1`,
-and `test-geospatial-endpoint-custom-dataset-v1`.
+You should see experiments including `geospatial-vllm-bench-deployment`,
+`geospatial-vllm-bench-endpoint`, `geospatial-vllm-bench-deployment-custom-dataset`,
+and `geospatial-vllm-bench-endpoint-custom-dataset`.
 
 ## Create an actuator configuration
 
@@ -212,7 +212,7 @@ Prithvi-EO-2.0 flood detection models.
 ## Using Custom Datasets
 
 To use your own geospatial datasets, use the
-`test-geospatial-deployment-custom-dataset-v1` experiment. Your dataset should
+`geospatial-vllm-bench-deployment-custom-dataset` experiment. Your dataset should
 be a JSONL (JSON Lines) file where each line is a JSON object with this
 structure:
 
@@ -251,7 +251,7 @@ Update your space definition to use the custom dataset experiment:
 ```yaml
 measurementSpace:
   - actuatorIdentifier: vllm_performance
-    experimentIdentifier: test-geospatial-deployment-custom-dataset-v1
+    experimentIdentifier: geospatial-vllm-bench-deployment-custom-dataset
 ```
 
 And add the dataset path to your entity space:
@@ -298,7 +298,7 @@ versions (0.20.0+) and fail early with a clear error if there's a mismatch.
 - Try the **600M parameter Prithvi model** by changing the model identifier to
   `ibm-nasa-geospatial/Prithvi-EO-2.0-600M-TL-Sen1Floods11`
 - Explore different **GPU types** if your cluster has multiple options
-- Test **endpoint benchmarking** with `test-geospatial-endpoint-v1` if you have
+- Test **endpoint benchmarking** with `geospatial-vllm-bench-endpoint` if you have
   an existing deployment
 - **Enable threadpool rendering** with vLLM 0.20.0+ to improve inference
   performance for geospatial models