Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/skills/agenthub-dev/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ CHANGELOG.md Brief one-line entries linking into changelog/
- Any difference between generations, even a single key name, means a separate folder per generation (e.g. `claude4_6/` vs `claude5/`).
- Only an identical wire protocol may share a folder; name it after the newest generation (rename and reroute if needed). This is how `claude5/` serves Claude 4.7, 4.8, and 5.
- `auto_client.py` / `autoClient.ts` route model names by explicit version matching only, never a bare substring like `"claude" in model`.
- Conversion must be bijective: a wire message converted to `UniMessage`/`UniEvent` and back must reproduce the original exactly, including thinking signatures, phase labels, and tool-call IDs. Verify against the captured exchange.
- Conversion must be bijective: a wire message converted to `UniMessage`/`UniEvent` and back must reproduce the original exactly, including `fidelity` payloads (thinking signatures, phase labels, reasoning field names) and tool-call IDs. Verify against the captured exchange.
- `UniConfig` keys rarely map one-to-one onto provider config keys. **Stop and ask**: list every non-obvious mapping and confirm it with the user before coding. Never decide silently.
- Implement Python and TypeScript together with identical behavior.

Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

Here, we record the addition and removal times of models, major functional updates, bug fixes, and release times of key versions. Each entry keeps one brief line; the full details of a change live in a file under [changelog/](changelog/).

- [2026-07-20] Content items now carry an opaque `fidelity` payload that absorbs the former `signature`/`phase` fields (breaking), and OpenAI-compatible clients use it to replay thinking through exactly the reasoning field the upstream produced instead of always sending both spellings. ([details](changelog/2026-07-20-reasoning-field-fidelity.md))

- [2026-07-17] Add the `agenthub-dev` skill that fixes the model-support development workflow, and the `changelog/` details directory. ([details](changelog/2026-07-17-agenthub-dev-skill.md))

- [2026-07-14] Raise `EmptyResponseError` when a model completes a response with thinking output only, since sending it back would fail with a 400 error. It and `ToolCallArgumentParseError` now inherit the new `AgentHubError` base class.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -433,7 +433,7 @@ Example UniMessage:
{"type": "text", "text": "How are you doing?"},
{"type": "image_url", "image_url": "https://example.com/image.jpg"},
{"type": "inline_data", "mime_type": "image/jpeg", "data": "base64-encoded-image"},
{"type": "thinking", "thinking": "I am thinking.", "signature": "0x123456"},
{"type": "thinking", "thinking": "I am thinking.", "fidelity": {"signature": "0x123456"}},
{"type": "inline_thinking", "mime_type": "image/jpeg", "data": "base64-encoded-image"},
{"type": "tool_call", "name": "math", "arguments": {"expression": "2 + 3"}, "tool_call_id": "123"},
{"type": "tool_result", "text": "2 + 3 = 5", "images": [], "tool_call_id": "123"}
Expand Down
39 changes: 39 additions & 0 deletions changelog/2026-07-20-reasoning-field-fidelity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Add the `fidelity` field and replay the exact reasoning field the upstream produced

## The bug

OpenAI Chat Completions-compatible servers spell the streamed thinking field differently — vLLM & SiliconFlow use `reasoning_content` while OpenRouter uses `reasoning` — and when sending assistant history back, the `openai`, `glm5_1`, and `kimi_k2_6` clients always set **both** fields on the message. Strict upstreams reject the spelling they did not emit (e.g. a server that returned `reasoning_content` refuses a request containing `reasoning`), breaking multi-turn conversations.

## The `fidelity` field

Fixing this needs a place to record which wire field carried the thinking. Rather than overloading `signature`, content items now carry a single dedicated field:

- `fidelity` (`dict[str, Any]` / `Record<string, any>`, optional) — an arbitrary JSON-style object of wire-level data a client records to reproduce the original message on replay. Opaque to consumers: pass it back unchanged.

It replaces and absorbs the former item-level `signature` and `phase` fields:

| Client | Old | New |
| --- | --- | --- |
| `claude5` / `claude4_6` | `signature: <sig>` on thinking (also holds redacted-thinking data) | `fidelity: {"signature": <sig>}` |
| `gemini3` | `signature: <thought_signature>` on text / thinking / inline / tool_call items (key present even when `None`) | `fidelity: {"signature": <thought_signature>}`, omitted entirely when absent |
| `gpt5_5` | `signature: json.dumps({"id": ..., "encrypted_content": ...})` on thinking; `phase: <p>` on text | `fidelity: {"id": ..., "encrypted_content": ...}` (no more JSON-in-a-string); `fidelity: {"phase": <p>}` |
| `openai` / `glm5_1` / `kimi_k2_6` / `deepseek_v4` | nothing recorded; both reasoning spellings sent back | `fidelity: {"reasoning_field": "reasoning_content" \| "reasoning"}` per thinking delta |

## The reasoning-field fix

On receive, the OpenAI-compatible clients record the wire field name that carried each thinking delta. On send, the message conversion replays the thinking through exactly that field. Fallbacks keep the old maximum-compatibility behavior: thinking without a recorded `reasoning_field` (hand-written histories, foreign-protocol fidelity), mixed fields within one message, and the ambiguous case where one chunk carries both spellings (such deltas record no fidelity) all still send both fields.

## Concatenation rules

`concat_uni_events_to_uni_message` / `concatUniEventsToUniMessage` now key on `fidelity`:

- text: a phase change starts a new item (same-phase and phaseless deltas merge, per the GPT-5.5 `phase` guide); an incoming fidelity payload (e.g. a signature) merges into the open item's fidelity and finishes it.
- thinking: an incoming fidelity payload finishes the open item (Claude signature deltas, GPT-5.5 reasoning markers), and a run of deltas carrying **equal** fidelity concatenates into one item (the OpenAI-compatible per-delta `reasoning_field` tags).

## Breaking change

Histories recorded by earlier versions carry `signature` / `phase` at the top level of content items; clients no longer read those fields. To replay an old history, move each item's `signature`/`phase` into `fidelity` (`{"signature": ...}` for Claude/Gemini, `{"id": ..., "encrypted_content": ...}` parsed from the GPT-5.5 JSON string, `{"phase": ...}` for GPT-5.5 text).

## Tests

Offline fake-stream suites `src_py/tests/test_reasoning_fidelity.py` and `src_ts/tests/reasoning-fidelity.test.ts` cover both reasoning field spellings, the ambiguous both-fields case, and the no-fidelity fallback across the `openai`, `glm5_1`, and `kimi_k2_6` clients.
1 change: 1 addition & 0 deletions llmsdk_docs/gpt5_5/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ The `docs/` folder contains detailed guides on various GPT-5.5 features:
- [images-vision.md](./docs/images-vision.md) - Image and vision capabilities
- [latest-model.md](./docs/latest-model.md) - Latest model features and updates
- [migrate-to-responses.md](./docs/migrate-to-responses.md) - Migration guide to Responses API
- [reasoning.md](./docs/reasoning.md) - Reasoning effort, summaries, encrypted reasoning, and the `phase` parameter
- [text.md](./docs/text.md) - Text generation and completion

## Examples
Expand Down
58 changes: 58 additions & 0 deletions llmsdk_docs/gpt5_5/docs/reasoning.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Reasoning models

Source: https://developers.openai.com/api/docs/guides/reasoning (Responses API; applies to GPT-5.6 variants, GPT-5.5, and GPT-5.4)

Page outline: Get started with reasoning / Reasoning effort / Reasoning mode / How reasoning works / Managing the context window / Controlling costs / Allocating space for reasoning / Handling incomplete responses / Preserve reasoning across calls / Continue reasoning with stored responses / Preserve reasoning without stored responses / Reasoning summaries / `phase` parameter / Advice on prompting / Use case examples

## Reasoning effort

The `reasoning.effort` parameter guides how much the model thinks: `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, or `max`. Lower values prioritize speed and cost; higher values enable more thorough analysis. Defaults vary by model; GPT-5.5 defaults to `medium`.

## Reasoning summaries

Models can emit summaries of their internal reasoning when `reasoning.summary` is set (`concise`, `detailed`, or `auto`). Summaries appear in the `summary` array of reasoning output items and require explicit opt-in.

## Preserve reasoning without stored responses

When `store` is `false`, reasoning items include an `encrypted_content` property by default. Pass these encrypted tokens back on later calls to preserve reasoning context without storing responses server-side.

## `phase` parameter

> For long-running or tool-heavy flows with GPT-5.5 and GPT-5.4 in the Responses API, use the assistant message `phase` field to avoid early stopping.

- Use `phase: "commentary"` for intermediate assistant updates, such as preambles before tool calls.
- Use `phase: "final_answer"` for the completed answer.
- These are the only two `phase` values.
- Don't add `phase` to user messages.
- `phase` is optional at the API level, but OpenAI recommends using it. When replaying assistant history manually, preserve each original `phase` value. Missing or dropped `phase` can cause preambles to be treated as final answers in those workflows.
- Using `previous_response_id` is usually the simplest path because prior assistant state is preserved.

Official example (input array with `phase` on assistant messages):

```python
from openai import OpenAI

client = OpenAI()

response = client.responses.create(
model="gpt-5.6",
input=[
{
"role": "assistant",
"phase": "commentary",
"content": "I'll inspect the logs and then summarize root cause and remediation.",
},
{
"role": "assistant",
"phase": "final_answer",
"content": "Root cause: cache invalidation race.",
},
{
"role": "user",
"content": "Great—now give me a rollout-safe fix plan.",
},
],
)

print(response.output_text)
```
2 changes: 1 addition & 1 deletion skills/agenthub-python/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ Agent loop rules:

- Send every tool result with the exact `tool_call_id` from its originating `tool_call`. Do not invent, normalize, or reuse IDs across unrelated tool calls.
- If streamed tool-call arguments cannot be parsed, AgentHub raises `ToolCallArgumentParseError`. Do not execute the tool from partial arguments; let the agent runtime retry or re-prompt the model.
- Preserve `thinking` and `inline_thinking` items. Do not strip `phase` or `signature` fields.
- Preserve `thinking` and `inline_thinking` items. Do not strip or modify `fidelity` fields.
- Do not accumulate `usage_metadata` across events. Take the latest `usage_metadata` as the usage of the current request.
- For embedding models, each `UniMessage` in the `messages` array produces **one embedding vector**. Within a single message, all items in `content_items` are aggregated into a single embedding. Set `embedding_config.dimensions` in the config to control vector size.

Expand Down
22 changes: 11 additions & 11 deletions skills/agenthub-python/reference/data-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,12 +54,12 @@ Fields:
message = {
"role": "user",
"content_items": [
{"type": "text", "text": "Hello", "phase": None, "signature": "sig"},
{"type": "text", "text": "Hello", "fidelity": {"phase": "commentary"}},
{"type": "image_url", "image_url": "https://example.com/image.jpg"},
{"type": "inline_data", "data": b"...", "mime_type": "image/png", "signature": "sig"},
{"type": "thinking", "thinking": "Reasoning", "signature": "sig"},
{"type": "inline_thinking", "data": b"...", "mime_type": "image/png", "signature": "sig"},
{"type": "tool_call", "name": "get_weather", "arguments": {"location": "Paris"}, "tool_call_id": "call_1", "signature": "sig"},
{"type": "inline_data", "data": b"...", "mime_type": "image/png", "fidelity": {"signature": "sig"}},
{"type": "thinking", "thinking": "Reasoning", "fidelity": {"signature": "sig"}},
{"type": "inline_thinking", "data": b"...", "mime_type": "image/png", "fidelity": {"signature": "sig"}},
{"type": "tool_call", "name": "get_weather", "arguments": {"location": "Paris"}, "tool_call_id": "call_1", "fidelity": {"signature": "sig"}},
{"type": "tool_result", "text": "22 C", "tool_call_id": "call_1"},
{"type": "embedding", "embedding": [0.1, 0.2]},
],
Expand All @@ -76,16 +76,16 @@ Fields:

Content items:

- `text`: Text chunk; `phase` marks sub-stage; `signature` verifies signed content.
- `text`: Text chunk; may carry `fidelity`.
- `image_url`: Image URL or data URI.
- `inline_data`: Inline media bytes with MIME type; may carry `signature`.
- `thinking`: Text reasoning content; may carry `signature`.
- `inline_thinking`: Binary reasoning artifact; may carry `signature`.
- `tool_call`: Complete model tool request with name, args, ID, and optional `signature`.
- `inline_data`: Inline media bytes with MIME type; may carry `fidelity`.
- `thinking`: Text reasoning content; may carry `fidelity`.
- `inline_thinking`: Binary reasoning artifact; may carry `fidelity`.
- `tool_call`: Complete model tool request with name, args, ID, and optional `fidelity`.
- `tool_result`: Tool output text for a `tool_call_id`; may include image URLs.
- `embedding`: Numeric embedding vector.

Preserve `phase` and `signature`; never drop either field.
`fidelity` is an arbitrary JSON object of wire-level data the client recorded to reproduce the original message on replay — thinking signatures, phase labels, the upstream reasoning field name, and the like. It is opaque: pass it back unchanged, never modify or drop it.

## UniEvent

Expand Down
2 changes: 1 addition & 1 deletion skills/agenthub-typescript/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ Keep these points in mind for agent loops:

- Send every tool result with the exact `tool_call_id` from its originating `tool_call`. Do not invent, normalize, or reuse IDs across unrelated tool calls.
- If streamed tool-call arguments cannot be parsed, AgentHub raises `ToolCallArgumentParseError`. Do not execute the tool from partial arguments; let the agent runtime retry or re-prompt the model.
- Preserve `thinking` and `inline_thinking` items. Do not strip `phase` or `signature` fields.
- Preserve `thinking` and `inline_thinking` items. Do not strip or modify `fidelity` fields.
- Do not accumulate `usage_metadata` across events. Take the latest `usage_metadata` as the usage of the current request.
- For embedding models, each `UniMessage` in the `messages` array produces **one embedding vector**. Within a single message, all items in `content_items` are aggregated into a single embedding. Set `embedding_config.dimensions` in the config to control vector size.

Expand Down
22 changes: 11 additions & 11 deletions skills/agenthub-typescript/reference/data-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,12 +54,12 @@ Fields:
const message = {
role: "user",
content_items: [
{ type: "text", text: "Hello", phase: null, signature: "sig" },
{ type: "text", text: "Hello", fidelity: { phase: "commentary" } },
{ type: "image_url", image_url: "https://example.com/image.jpg" },
{ type: "inline_data", data: Buffer.from("..."), mime_type: "image/png", signature: "sig" },
{ type: "thinking", thinking: "Reasoning", signature: "sig" },
{ type: "inline_thinking", data: Buffer.from("..."), mime_type: "image/png", signature: "sig" },
{ type: "tool_call", name: "get_weather", arguments: { location: "Paris" }, tool_call_id: "call_1", signature: "sig" },
{ type: "inline_data", data: Buffer.from("..."), mime_type: "image/png", fidelity: { signature: "sig" } },
{ type: "thinking", thinking: "Reasoning", fidelity: { signature: "sig" } },
{ type: "inline_thinking", data: Buffer.from("..."), mime_type: "image/png", fidelity: { signature: "sig" } },
{ type: "tool_call", name: "get_weather", arguments: { location: "Paris" }, tool_call_id: "call_1", fidelity: { signature: "sig" } },
{ type: "tool_result", text: "22 C", tool_call_id: "call_1" },
{ type: "embedding", embedding: [0.1, 0.2] },
],
Expand All @@ -76,16 +76,16 @@ Fields:

Content items:

- `text`: Text chunk; `phase` marks sub-stage; `signature` verifies signed content.
- `text`: Text chunk; may carry `fidelity`.
- `image_url`: Image URL or data URI.
- `inline_data`: Inline media bytes with MIME type; may carry `signature`.
- `thinking`: Text reasoning content; may carry `signature`.
- `inline_thinking`: Binary reasoning artifact; may carry `signature`.
- `tool_call`: Complete model tool request with name, args, ID, and optional `signature`.
- `inline_data`: Inline media bytes with MIME type; may carry `fidelity`.
- `thinking`: Text reasoning content; may carry `fidelity`.
- `inline_thinking`: Binary reasoning artifact; may carry `fidelity`.
- `tool_call`: Complete model tool request with name, args, ID, and optional `fidelity`.
- `tool_result`: Tool output text for a `tool_call_id`; may include image URLs.
- `embedding`: Numeric embedding vector.

Preserve `phase` and `signature`; never drop either field.
`fidelity` is an arbitrary JSON object of wire-level data the client recorded to reproduce the original message on replay — thinking signatures, phase labels, the upstream reasoning field name, and the like. It is opaque: pass it back unchanged, never modify or drop it.

## UniEvent

Expand Down
Loading
Loading