fix(extensions): raise on truncated empty Chat Completions responses - #4886
fix(extensions): raise on truncated empty Chat Completions responses#4886CoralGarden52 wants to merge 1 commit into
Conversation
seratch
left a comment
There was a problem hiding this comment.
Thanks for the fix. The new condition also rejects length-truncated responses containing reasoning but no assistant text. The existing adapters can return that reasoning as a ResponseReasoningItem, so this may change more than empty-response handling. Please provide a real-provider comparison for a reasoning-only truncated response through the affected adapters, showing the base and head outputs. We need that evidence to assess compatibility before merging.
|
I independently reproduced the review concern in a local fork: a I tested a reasoning-aware variant at fscfede-beep/openai-agents-python@504dcf9:
Branch: https://github.com/fscfede-beep/openai-agents-python/tree/fix/empty-length-chat-completions-4885 I have not run the requested live paid-provider base/head comparison, so this is only code-level compatibility evidence, not a substitute for that live verification. |
|
Without confirming real-world impact, we won't move forward with this possibly breaking change to existing use cases. |
|
I now have a real-provider base/head comparison for the reasoning-only truncation case. Provider/model used:
Raw provider behavior is the case from the review concern: the OpenAI-compatible endpoint returns Pinned code states:
AnyLLM, end-to-end against the real Ollama provider:
LiteLLM, end-to-end via Ollama's real OpenAI-compatible
I also independently called So this is a concrete compatibility break in an existing supported path: before #4886, a length-truncated reasoning-only completion is surfaced as a reasoning item; at the current PR head it becomes an exception. The reasoning-aware guard at This is local real-provider evidence, not a hosted/paid-provider claim. The branch remains: https://github.com/fscfede-beep/openai-agents-python/tree/fix/empty-length-chat-completions-4885 |
Summary
This pull request fixes AnyLLM and LiteLLM non-streaming Chat Completions adapters returning successful empty responses when a completion terminates with
finish_reason="length"before emitting assistant text, refusal, or tool calls.Both adapters now raise
ModelBehaviorErrorafter usage and tracing data are recorded, matching the built-in OpenAI Chat Completions adapter. Responses containing partial text, refusals, or tool calls remain supported.Test plan
content=Noneandcontent="".finish_reason.ModelBehaviorErroris raised.Live verification:
gpt-5.6-lunatemperature=0max_tokens=256The raw Chat Completions response returned
finish_reason="length"withcontent=null. The built-in adapter raisedModelBehaviorError, while AnyLLM and LiteLLM previously returned successful empty outputs.Issue number
Closes #4885
Checks
.agents/skills/code-change-verification/scripts/run.sh/reviewbefore submitting this PR