feat(vllm-performance): add prefix_repetition benchmark dataset support - #1274
Open
mgazz wants to merge 4 commits into
Open
feat(vllm-performance): add prefix_repetition benchmark dataset support#1274mgazz wants to merge 4 commits into
prefix_repetition benchmark dataset support#1274mgazz wants to merge 4 commits into
Conversation
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
mgazz
requested review from
AlessandroPomponio and
christian-pinto
and removed request for
christian-pinto
July 24, 2026 10:03
AlessandroPomponio
requested changes
Jul 24, 2026
AlessandroPomponio
left a comment
AlessandroPomponio
left a comment
Collaborator
There was a problem hiding this comment.
This requires a bump in the VERSION for vllm-performance
christian-pinto
requested changes
Jul 24, 2026
…lm_benchmark and add defaults for prefix_repetition params Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
Member
Author
|
Comments address as following:
|
Comment on lines
+39
to
+46
| prefix_repetition_prefix_len: Annotated[int | None, pydantic.Field()] = None | ||
| prefix_repetition_suffix_len: Annotated[int | None, pydantic.Field()] = None | ||
| prefix_repetition_num_prefixes: Annotated[int | None, pydantic.Field()] = None | ||
| prefix_repetition_output_len: Annotated[int | None, pydantic.Field()] = None | ||
| burstiness: Annotated[float, pydantic.Field()] = 1.0 | ||
| dataset: Annotated[str | None, pydantic.Field()] = "random" | ||
| dataset: Annotated[ | ||
| Literal["random", "prefix_repetition"] | None, pydantic.Field() | ||
| ] = "random" |
Collaborator
There was a problem hiding this comment.
The fields here are still optional while in other places they aren't optional anymore
| "--random-output-len": max_output_tokens, | ||
| } | ||
|
|
||
| if dataset == "prefix_repetition": |
Collaborator
There was a problem hiding this comment.
There is still nothing actually checking that dataset is only "random" or "prefix_repetition"
| dataset=benchmark_parameters.dataset, | ||
| burstiness=benchmark_parameters.burstiness, | ||
| ) | ||
| elif benchmark_parameters.dataset == "bfcl": |
Collaborator
There was a problem hiding this comment.
It looks like the dataset can be more than just "random" and "prefix_repetition"
Comment on lines
+600
to
+602
| dataset: Literal["random", "prefix_repetition"] = ( | ||
| benchmark_parameters.dataset or "random" | ||
| ) |
Collaborator
There was a problem hiding this comment.
This still allows values other than the two defined to go through, unless I'm missing validation somewhere else
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat(vllm-performance): add
prefix_repetitionbenchmark dataset supportAdds support for the
prefix_repetitionbenchmark dataset to thevllm_performanceactuator. This dataset simulates KV-cache prefix-caching workloads by generating requests with a shared cached prefix followed by a unique suffix, making it useful for evaluating prefix-cache hit rates and their effect on throughput and latency.Changes
benchmark_models.py— Added four new optional fields toBenchmarkParameters:prefix_repetition_prefix_len,prefix_repetition_suffix_len,prefix_repetition_num_prefixes, andprefix_repetition_output_len.execute_benchmark.py— Extendedexecute_random_benchmarkto accept the four new parameters and, whendataset="prefix_repetition", swap thecustom_argsdict passed to the vLLM benchmark runner from the--random-*flags to the corresponding--prefix-repetition-*flags.experiment_executor.py— Threaded the new parameters through bothrun_resource_and_workload_experimentandrun_workload_experimentcall sites.performance_testing.yaml— Addedprefix_repetitionas a validdatasetvalue across all five experiment definitions (test-deployment-v1,test-endpoint-v1,test-deployment-guidellm-v1,test-endpoint-guidellm-v1,test-agentic-tool-calling). Declared the four new properties with discrete domains and sensible defaults (prefix len: 2048, suffix len: 128, num prefixes: 10, output len: 128).