Repository navigation
fix(llm): keep finish_reason and empty content for tool responses without calls - #5633
Merged
qinxuye merged 1 commit intoOct 5, 2026
Conversation
…hout calls When a non-streaming request with tools ends without a tool call, _post_process_completion always reported finish_reason "stop", and when the output only contained reasoning (e.g. max_tokens reached inside the thinking block) the raw output was returned as content as well as reasoning_content. Use the engine's finish_reason when there is no tool call and always use the reasoning parser's content, matching the request without tools and the streaming tool path.
qinxuye
approved these changes
Oct 5, 2026
qinxuye
left a comment
Contributor
There was a problem hiding this comment.
LGTM. Verified the reasoning-only empty-content handling and preservation of the engine finish_reason when no tool is called. All 176 focused utility and tool-parser tests pass locally. The sole CI failure is test_fatal_engine_error_closes_model in the image engine, outside the changed LLM post-processing path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
ChatModelMixin._post_process_completion(non-streaming chat withtools, used by the vLLM, SGLang, MLX, llama.cpp and transformers engines) has two problems when the model does not call a tool:finish_reasonis always"stop", so a response cut off bymax_tokensis reported as"stop"instead of"length". The streaming tool path and the request without tools keep"length".max_tokensis reached inside the thinking block), the reasoning parser returns empty content, but theif processed_content:check then keeps the raw output, so the whole reasoning is returned again ascontent.Example with Qwen3 (
reasoning_content=True), output cut off while thinking:Before:
After:
Fix
Always use the reasoning parser's content for tool parsing, and use the engine's
finish_reasonwhen there is no tool call ("stop"if it is missing). Responses with tool calls are unchanged.Tests
test_post_process_completion_stopped_while_thinking(fails before the fix, passes after).pytest xinference/model/llm/tests/test_utils.py xinference/model/llm/tool_parsers/testspasses.