Repository navigation
Python: Invoke the model one more time after auto invoke has exhausted - #13236
Tao Chen (TaoChenOSU) wants to merge 3 commits into
Conversation
Tao Chen (TaoChenOSU)
added
python
github-actions
Bot
changed the title
777f3f0 to
4383883
Compare
There was a problem hiding this comment.
Pull request overview
Aligns streaming auto function invocation behavior with the non streaming implementation by performing one final model call after the auto invoke attempt limit is exhausted, so streaming scenarios can still produce a final text response.
Changes:
- Simplifies local imports of function result merge helpers.
- Adds a
for ... elsefinal streaming call path aftermaximum_auto_invoke_attemptsis reached, resetting function calling settings before the final call.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Do a final call, without function calling when the max has been reached. | ||
| self._reset_function_choice_settings(settings) | ||
| async for streaming_chat_message_contents in self._inner_get_streaming_chat_message_contents( | ||
| chat_history, settings |
There was a problem hiding this comment.
In the final non function calling call, _inner_get_streaming_chat_message_contents is invoked without passing function_invoke_attempt, so derived connectors will default it to 0. That can produce duplicate function_invoke_attempt values across attempts and may break consumers that group, dedupe, or combine streaming messages by attempt. Pass an explicit attempt index for this final call, for example settings.function_choice_behavior.maximum_auto_invoke_attempts.
| chat_history, settings | |
| chat_history, | |
| settings, | |
| function_invoke_attempt=settings.function_choice_behavior.maximum_auto_invoke_attempts, |
| else: | ||
| # Do a final call, without function calling when the max has been reached. | ||
| self._reset_function_choice_settings(settings) | ||
| async for streaming_chat_message_contents in self._inner_get_streaming_chat_message_contents( | ||
| chat_history, settings | ||
| ): | ||
| yield streaming_chat_message_contents |
There was a problem hiding this comment.
This adds new control flow for streaming auto invoke exhaustion (the for ... else final call after maximum_auto_invoke_attempts). Please add a unit test that exercises this path and asserts that a final model invocation happens after the last tool call and that function calling is disabled for that final invocation (via _reset_function_choice_settings).
GitHub Actions
Inactive
Motivation and Context
Related to: #13006
Description
Invoke the chat completion streaming service one more time after the auto function invocation limit has been reached to generate a text response. This has been done in the non-streaming case.
TODO: unit tests
Contribution Checklist