Skip to content

Fix: always send stream flag explicitly in LLM request body#438

Merged
altic-dev merged 2 commits into
altic-dev:mainfrom
Kayaba-Attribution:fix/issue-295-stream-flag-ollama
Jun 28, 2026
Merged

Fix: always send stream flag explicitly in LLM request body#438
altic-dev merged 2 commits into
altic-dev:mainfrom
Kayaba-Attribution:fix/issue-295-stream-flag-ollama

Conversation

@Kayaba-Attribution

@Kayaba-Attribution Kayaba-Attribution commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes a bug where Ollama and other OpenAI-compatible providers would silently fail AI enhancement when streaming was disabled.

When `config.streaming` was `false`, the `stream` key was omitted entirely from the JSON request body. Providers like Ollama treat an absent `stream` key as `true` — so they returned SSE-encoded chunks, the non-streaming parser couldn't handle them, and the call failed with `LLMError.invalidResponse`. The enhancement would silently fall back to the raw transcript.

The fix is to always send the key explicitly in both `buildChatCompletionsBody` and `buildResponsesBody`:

```swift
// Before — key absent when false; Ollama reads absence as true
if config.streaming {
body["stream"] = true
}

// After — always explicit
body["stream"] = config.streaming
```

Type of Change

  • 🐞 Bug fix (non-breaking change which fixes an issue)

Related Issues

Testing

  • Tested on Apple Silicon Mac (M-series)
  • Tested on macOS Sequoia 15
  • 4 unit tests added in `LLMClientRequestBodyTests` asserting the `stream` key is present and correct for both endpoints (`/chat/completions` and `/responses`) and both streaming states
  • Tested on Intel Mac
  • Ran linter locally: `swiftlint --strict --config .swiftlint.yml Sources` — 0 violations
  • Ran formatter locally: `swiftformat --config .swiftformat Sources/Fluid/Services/LLMClient.swift` — 1 cosmetic fix (`//` → `///` on an existing comment)

Notes

  • `buildChatCompletionsBody` and `buildResponsesBody` were changed from `private` to `internal` to allow direct unit testing. No behaviour changes outside the module.
  • The `stream: false` fix is most impactful for Ollama and self-hosted providers. OpenAI and Groq already default to non-streaming when the key is absent, so this is a no-op for them.

…#295)

Providers like Ollama and other OpenAI-compatible endpoints treat an
absent `stream` key as `true`. When streaming was disabled, the key was
omitted entirely from the request body, causing the server to return
SSE-encoded chunks that the non-streaming parser couldn't handle,
resulting in `LLMError.invalidResponse` and silent enhancement failure.

Change `buildChatCompletionsBody` and `buildResponsesBody` to always
include `stream` regardless of its value:
  - Before: `if config.streaming { body["stream"] = true }`
  - After:  `body["stream"] = config.streaming`

Also relaxes both methods from `private` to `internal` to allow direct
unit testing. Four regression tests added in `LLMClientRequestBodyTests`
asserting the stream key is present and correct for both endpoints and
both streaming states.
@altic-dev altic-dev merged commit 96b77e9 into altic-dev:main Jun 28, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[🐞 BUG] AI Enhancements fail when provider defaults to streaming (stream flag omitted from request body)

2 participants