chore: sync text generation launcher with vLLM 0.26 - #524
Merged
Conversation
Two narrow fixes surfaced by the vLLM v0.26 audit (#515): Pass the model reference as a positional argument to `vllm serve` rather than `--model`. v0.26 warns on every service start: "With `vllm serve`, you should provide the model as a positional argument or in a config file instead of via the `--model` option. The `--model` option will be removed in a future version." Migrating now avoids a hard break on the next vLLM bump. Applies to `text_generation_slurm.sh` and both branches of `text_generation_local.sh`. Remove the Disable Custom Kernels toggle. It has been a phantom option for a while: the toggle sets container state and passes through the API payload, but neither the active `text_generation_*.sh` templates nor the backend translate it into a vLLM flag (only the deprecated templates mention `--disable-custom-kernels`). The toggle told users they were changing behavior; they were not. Cleans up UI, initial state, propTypes, tests, and dev fixtures. Closes #515
ReviewSmall, well-scoped PR — both changes do exactly what the description says, and I verified there's no leftover state:
Minor/non-blocking observations:
No security, performance, or test-coverage concerns beyond the manual-verification gap noted above. Good cleanup. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two narrow fixes surfaced by the vLLM v0.26 audit.
Migrate `--model` to a positional argument. v0.26 emits a deprecation warning on every service start:
Migrating now avoids a hard break on the next vLLM bump. Applies to `text_generation_slurm.sh` and both branches of `text_generation_local.sh` (docker + apptainer).
Remove the Disable Custom Kernels toggle. It has been a phantom option for a while: the toggle sets container state and appears in the request body, but neither the active `text_generation_*.sh` templates nor the backend translate it into a vLLM flag — only the `templates/deprecated/` copies mention `--disable-custom-kernels`. The toggle told users they were changing behavior; they were not. Cleans up UI, initial state, propTypes, tests, and dev fixtures.
Test plan
Notes
Closes #515