Surfaced during the vLLM v0.26 audit (#515).
Context
vLLM v0.22 added `thinking_token_budget` — a per-request cap on how many tokens the model spends "thinking" before producing the visible answer. It complements the existing Disable Thinking toggle: instead of the binary on/off choice, users could allow reasoning but bound its cost.
The Disable Thinking toggle maps to a launch-time flag (`--default-chat-template-kwargs`). `thinking_token_budget` is per-request, so it belongs on the completions/chat body, not the launcher — likely in the parameters sidebar (`SidebarContainer`'s Parameters section for text-generation), alongside `temperature` and `max_tokens`.
Scope
Notes
- Complements Disable Thinking rather than replacing it. Disable Thinking is a launch-time choice (affects the chat template); `thinking_token_budget` is a per-request choice.
- Not a bug — pure feature request.
Surfaced during the vLLM v0.26 audit (#515).
Context
vLLM v0.22 added `thinking_token_budget` — a per-request cap on how many tokens the model spends "thinking" before producing the visible answer. It complements the existing Disable Thinking toggle: instead of the binary on/off choice, users could allow reasoning but bound its cost.
The Disable Thinking toggle maps to a launch-time flag (`--default-chat-template-kwargs`). `thinking_token_budget` is per-request, so it belongs on the completions/chat body, not the launcher — likely in the parameters sidebar (`SidebarContainer`'s Parameters section for text-generation), alongside `temperature` and `max_tokens`.
Scope
Notes