Repository navigation
fix(spring-boot): keep the finish reason when usage arrives in the last chunk - #2276
Open
elgamal-ahmed wants to merge 1 commit into
Open
elgamal-ahmed wants to merge 1 commit into
elgamal-ahmed wants to merge 1 commit into
Conversation
…st chunk SpringAiLLMClient.executeStreaming held a single ChatResponse reference that every collected chunk overwrote, then read the finish reason and the token usage off it once the stream had ended. With Spring AI's stream-usage option the terminal chunk carries the usage and an empty results list, so the finish reason from the chunk before it was dropped and the End frame reported null. Keep each value as it arrives instead, the way the OpenAI-like clients already do. A chunk that reports neither carries an empty finish reason and zero token counts rather than nulls, and Mistral AI and DeepSeek pad every streamed chunk that way, so neither is allowed to replace what an earlier chunk reported. koog-spring-ai-v2 carried the same code and gets the same fix. Closes JetBrains#2109
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Keeps the finish reason and the token usage when Spring AI reports them in separate stream chunks, so the terminal End frame carries both. Closes #2109
executeStreamingheld a singleChatResponsereference that every collected chunk overwrote, then read both values off it in thefinallyblock. Withstream-usageenabled the last chunk carries the usage and an emptyresultslist, so the finish reason from the chunk before it was lost and the End frame reportednull; with the option off the symptom inverts and the token counts stay at zero. Each value is now kept as it arrives, the same shape PR #1404 used for the OpenAI-like clients. The one judgement call is what a chunk reporting neither looks like:ChatResponseMetadata.getUsage()is never null but reports zeros, and Mistral AI and DeepSeek pad every streamed chunk with an empty finish reason, so an empty reason is skipped and a zero usage only fills in while nothing has been counted yet, which keeps the End frame for a stream that reports no usage at all exactly as it is today.koog-spring-ai-v2carried the same code and gets the same fix.Tests: four per module, covering the usage-only terminal chunk, usage followed by a chunk that reports none, a padded empty finish reason after a real one, and the unchanged zero counts when nothing reports usage. The first three fail on
developwithout the production change.closes KG-857