Skip to content

Add the fidelity payload and replay the exact reasoning field the upstream produced - #159

Merged
hiyouga merged 5 commits into
devfrom
fix/reasoning-field-fidelity
Jul 20, 2026
Merged

Add the fidelity payload and replay the exact reasoning field the upstream produced#159
hiyouga merged 5 commits into
devfrom
fix/reasoning-field-fidelity

Conversation

@hiyouga

@hiyouga hiyouga commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Problem

OpenAI Chat Completions-compatible servers spell the streamed thinking field differently: vLLM & SiliconFlow use reasoning_content, OpenRouter uses reasoning. When sending assistant history back, the openai, glm5_1, and kimi_k2_6 clients always set both fields on the message. Strict upstreams reject the spelling they did not emit — e.g. a server that streamed reasoning_content refuses a request that also carries reasoning — which breaks multi-turn conversations.

Design: the fidelity field

Fixing this needs a place to record which wire field carried the thinking. Instead of overloading signature, content items now carry one dedicated field:

  • fidelity (dict[str, Any] / Record<string, any>, optional) — an arbitrary JSON-style object of wire-level data the client records to reproduce the original message on replay. Opaque to consumers: pass it back verbatim.

It replaces and absorbs the former item-level signature and phase fields:

Client Before After
claude5 / claude4_6 signature: <sig> (also redacted-thinking data) fidelity: {"signature": <sig>}
gemini3 signature: <thought_signature> on text/thinking/inline/tool_call, key present even when null fidelity: {"signature": ...}, omitted when absent
gpt5_5 signature: JSON.stringify({id, encrypted_content}); phase on text fidelity: {"id", "encrypted_content"} (no JSON-in-a-string); fidelity: {"phase": ...}
openai / glm5_1 / kimi_k2_6 / deepseek_v4 nothing recorded; both reasoning spellings replayed fidelity: {"reasoning_field": "reasoning_content" | "reasoning"}

The reasoning-field fix

On receive, the OpenAI-compatible clients record the wire field name per thinking delta; on send, the conversion replays thinking through exactly that field. Fallbacks keep the old maximum-compatibility behavior: thinking without a recorded field (hand-written histories, foreign-protocol fidelity), mixed fields in one message, and the ambiguous case where one chunk carries both spellings (recorded with no fidelity) all still send both fields.

Concatenation rules

concat_uni_events_to_uni_message / concatUniEventsToUniMessage now key on fidelity:

  • text: a phase change starts a new item; same-phase and phaseless deltas merge (per the GPT-5.5 phase guide, snapshotted into llmsdk_docs/gpt5_5/docs/reasoning.md); other fidelity (signatures) merges into the open item's fidelity and finishes it.
  • thinking: an incoming fidelity finishes the open item (Claude signature deltas, GPT-5.5 reasoning markers); a run of deltas with equal fidelity concatenates into one item (the per-delta reasoning_field tags).

Breaking change

Histories recorded by earlier versions carry signature/phase at the item top level; clients no longer read those fields. Migration notes are in changelog/2026-07-20-reasoning-field-fidelity.md.

Tests

Offline fake-stream suites src_py/tests/test_reasoning_fidelity.py and src_ts/tests/reasoning-fidelity.test.ts, parameterized over openai / glm-5.1 / kimi-k2.6:

  • upstream streams reasoning_content → one thinking item with fidelity: {reasoning_field: "reasoning_content"}; replay contains only reasoning_content
  • upstream streams reasoning → replay contains only reasoning
  • chunk carries both fields → item has no fidelity; replay sends both
  • thinking without fidelity in history → replay sends both

Verified locally: ruff check + ruff format --check clean; eslint + tsc + prettier clean; 71 Python and 56 TypeScript offline tests pass (no API quota used). Docs (skills/*/reference/data-models.md, skills/*/SKILL.md, README.md, agenthub-dev skill) updated to the new field.

🤖 Generated with Claude Code

https://claude.ai/code/session_01J8bNMvyPJ7muyDs4Fh9ZLF

OpenAI-compatible servers spell the thinking field differently
(reasoning_content for vLLM & siliconflow, reasoning for openrouter), and
the openai, glm5_1, and kimi_k2_6 clients always sent both spellings back
with assistant history. Strict upstreams reject the field they did not
emit, breaking multi-turn conversations.

The clients now stamp each thinking delta's signature with the wire field
that carried it and replay the thinking through exactly that field.
Unsigned, mixed, or ambiguous (both fields in one chunk) thinking still
falls back to sending both spellings. Consecutive thinking deltas sharing
the same reasoning-field signature now concatenate into a single item;
other signature values keep the close-on-signature behavior unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8bNMvyPJ7muyDs4Fh9ZLF
Copilot AI review requested due to automatic review settings July 20, 2026 09:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves multi-turn compatibility with OpenAI Chat Completions–compatible streaming servers by recording which upstream “thinking” wire field was used (reasoning_content vs reasoning) and replaying exactly that field when sending assistant history back (with compatibility fallbacks preserved). Python and TypeScript implementations are kept in sync, and new offline fake-stream tests validate the behavior across openai, glm-5.1, and kimi-k2.6.

Changes:

  • Stamp streamed thinking deltas with a signature indicating the upstream reasoning field name, and replay through only that recorded field when unambiguous.
  • Relax concat_uni_events_to_uni_message / concatUniEventsToUniMessage to merge consecutive thinking deltas when the reasoning-field signature matches.
  • Add offline fake-stream regression tests and update reference docs + changelog entries.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src_ts/tests/reasoning-fidelity.test.ts Adds offline regression tests for reasoning-field fidelity and fallback behavior.
src_ts/src/utils.ts Introduces REASONING_FIELD_SIGNATURES constants for controlled signature merging logic.
src_ts/src/openai/client.ts Records reasoning-field signature on receive and replays only the recorded field on send (with fallbacks).
src_ts/src/kimi_k2_6/client.ts Same reasoning-field signature record/replay behavior as OpenAI client.
src_ts/src/glm5_1/client.ts Same reasoning-field signature record/replay behavior as OpenAI client.
src_ts/src/deepseek_v4/client.ts Stamps reasoning_content deltas so DeepSeek history replays correctly via other OpenAI-compatible clients.
src_ts/src/baseClient.ts Updates thinking-item concatenation to allow merging for matching reasoning-field signatures.
src_py/tests/test_reasoning_fidelity.py Adds offline regression tests for reasoning-field fidelity and fallback behavior.
src_py/agenthub/utils.py Adds REASONING_FIELD_SIGNATURES constants for controlled signature merging logic.
src_py/agenthub/openai/client.py Records reasoning-field signature on receive and replays only the recorded field on send (with fallbacks).
src_py/agenthub/kimi_k2_6/client.py Same reasoning-field signature record/replay behavior as OpenAI client.
src_py/agenthub/glm5_1/client.py Same reasoning-field signature record/replay behavior as OpenAI client.
src_py/agenthub/deepseek_v4/client.py Stamps reasoning_content deltas so DeepSeek history replays correctly via other OpenAI-compatible clients.
src_py/agenthub/base_client.py Updates thinking-item concatenation to allow merging for matching reasoning-field signatures.
skills/agenthub-typescript/reference/data-models.md Documents that thinking signature may record the upstream reasoning wire field name for replay fidelity.
skills/agenthub-python/reference/data-models.md Same documentation update as TypeScript.
changelog/2026-07-20-reasoning-field-fidelity.md Adds detailed changelog entry describing the fidelity fix and fallback behaviors.
CHANGELOG.md Adds a one-line changelog entry pointing to the detailed note.

Comment thread src_ts/src/baseClient.ts Outdated
Comment thread src_py/agenthub/base_client.py
…payload

Content items now carry a single optional fidelity field: an arbitrary
JSON-style object of wire-level data a client records to reproduce the
original message on replay. Consumers pass it back unchanged.

It absorbs the former signature and phase fields: Claude and Gemini store
{"signature": ...}, GPT-5.5 stores {"id", "encrypted_content"} directly
instead of a JSON string crammed into signature, plus {"phase": ...} on
text, and the OpenAI-compatible clients record the upstream reasoning
field as {"reasoning_field": "reasoning_content" | "reasoning"} without
overloading signature semantics.

Concatenation now keys on fidelity: an incoming payload finishes the open
thinking item, a run of deltas with equal fidelity concatenates into one
item, and a fidelity phase starts a new text item. Histories recorded by
earlier versions carry signature/phase at the item top level and need
those moved under fidelity to replay (breaking).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8bNMvyPJ7muyDs4Fh9ZLF
@hiyouga hiyouga changed the title fix: replay the exact reasoning field the upstream produced Add the fidelity payload and replay the exact reasoning field the upstream produced Jul 20, 2026
hiyouga and others added 3 commits July 20, 2026 03:22
A delta that announces a phase always starts a new text item, even when
the phase is unchanged, so that two same-phase wire output items stay two
entries on replay; phaseless deltas merge into the open item.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8bNMvyPJ7muyDs4Fh9ZLF
Per the official phase guide, phase only takes "commentary" or
"final_answer", and adjacent assistant text with an unchanged phase
belongs to one item: concatenation now splits text items only when the
phase actually changes, and the GPT-5.5 replay groups consecutive
same-phase items into one wire entry. Snapshot the reasoning guide into
llmsdk_docs/gpt5_5/docs/reasoning.md and use the official phase values
in tests and docs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8bNMvyPJ7muyDs4Fh9ZLF
@hiyouga
hiyouga merged commit db6ad68 into dev Jul 20, 2026
2 of 3 checks passed
@hiyouga
hiyouga deleted the fix/reasoning-field-fidelity branch July 20, 2026 11:30
@hiyouga hiyouga mentioned this pull request Jul 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants