Skip to content

fix: ignore empty SSE data frames in OpenAI-compatible and Google providers - #762

Open
harshitgavita-07 wants to merge 3 commits into
huggingface:mainfrom
harshitgavita-07:fix-empty-sse-data-frames
Open

harshitgavita-07 wants to merge 3 commits into
huggingface:mainfrom
harshitgavita-07:fix-empty-sse-data-frames

Conversation

@harshitgavita-07

Copy link
Copy Markdown

Motivation

An SSE frame with an empty payload (data: followed by nothing) is valid and is sent as a keep-alive by some proxies. #680 made the Anthropic parser ignore these frames. The OpenAI-compatible and Google parsers still return "" for them, _loads_object("") fails, and the stream ends with a fatal "Provider returned invalid JSON chunk" (or "Google returned an invalid JSON stream chunk") error even though the rest of the response is fine. This is the same failure reported for the Anthropic path in #679.

Changes

  • src/tau_ai/openai_compatible.py and src/tau_ai/google.py: _parse_sse_line returns None when the payload after data: is empty, same as anthropic.py. The OpenAI-compatible change covers both the chat completions and Responses paths since they share the helper.
  • tests/test_sse_empty_data_frames.py: streams a response with an empty data: frame in the middle through the OpenAI-compatible, Google and Mistral providers and expects a normal completed message.

No behavior change for non-empty frames, [DONE], or other lines.

Tests

  • New test: 3 cases. On current main the OpenAI-compatible and Google cases fail with the invalid JSON chunk error; the Mistral case already passes and is included as a guard. All 3 pass with the change.
  • 475 related tests pass (-k "tau_ai or provider or sse or openai or google or chat_channels or prompt_caching or multimodal").
  • ruff check, ruff format --check and mypy pass.
  • The full pytest run was stopped at about 76% for time; the remaining tests were not run for this PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant