Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions .agents/skills/agenthub-dev/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ src_ts/src/<protocol>/ TypeScript client, mirrors the Python folder
src_ts/src/autoClient.ts TypeScript routing, mirrors auto_client.py
src_py/tests/test_client.py Parameterized e2e tests (env-gated AVAILABLE_MODELS)
src_ts/tests/client.test.ts Same for TypeScript
changelog/ One detail file per CHANGELOG.md entry
CHANGELOG.md Brief one-line entries linking into changelog/
changelog/<version>/ Release summary (README.md) plus one detail file per entry
CHANGELOG.md One brief line per release linking into changelog/
```

## Stage 1 — Sync official docs into `llmsdk_docs/`
Expand All @@ -43,22 +43,32 @@ CHANGELOG.md Brief one-line entries linking into changelog/
- One folder per wire protocol, named after the newest model generation that uses it. Diff the new protocol (capture + docs) against the closest existing folder:
- Any difference between generations, even a single key name, means a separate folder per generation (e.g. `claude4_6/` vs `claude5/`).
- Only an identical wire protocol may share a folder; name it after the newest generation (rename and reroute if needed). This is how `claude5/` serves Claude 4.7, 4.8, and 5.
- When the old and new generations' implementations differ, keep the old model supported: leave its client folder and routing in place and add a new client folder for the new model. Never delete or rewire away an old model's client unless the user explicitly instructs it.
- `auto_client.py` / `autoClient.ts` route model names by explicit version matching only, never a bare substring like `"claude" in model`.
- Conversion must be bijective: a wire message converted to `UniMessage`/`UniEvent` and back must reproduce the original exactly, including `fidelity` payloads (thinking signatures, phase labels, reasoning field names) and tool-call IDs. Verify against the captured exchange.
- `UniConfig` keys rarely map one-to-one onto provider config keys. **Stop and ask**: list every non-obvious mapping and confirm it with the user before coding. Never decide silently.
- Every `ThinkingLevel` must stay usable on every client — never raise for a thinking level. Map each level to the closest level the model supports and degrade silently when a level has no exact equivalent (e.g. `gemini3` maps `NONE` to `MINIMAL`; `kimi_k3` maps `NONE` to `low` because K3 cannot disable reasoning).
- `temperature` and `tool_choice` (and other unsupported parameter values, e.g. `prompt_caching`) may reject with an exception, but must raise the AgentHub-specific `UnsupportedParameterError` from `errors.py` / `errors.ts`, never a bare `ValueError`/`Error`. Keep the message wording consistent with existing clients (containing "not support").
- Implement Python and TypeScript together with identical behavior.

## Stage 4 — Verify

- Register the model in the env-gated `AVAILABLE_MODELS` lists of both test files with correct capability flags. Do not add model-specific test functions or files.
- `AVAILABLE_MODELS` keeps only the newest version of each model family per provider block (e.g. gemini-3.6-flash, not gemini-3.5-flash or 3.5-flash-lite as well). When a newer generation lands, replace the older entry — the old client folder stays supported and routed (see Stage 3) but is no longer e2e-tested.
- Static checks: `make lint` in `src_py/`; `npm run lint` and `npm run build` in `src_ts/`.
- Run only the new model's e2e tests; the full suites are slow and spend real API quota:
- `cd src_py && uv run pytest -vvv tests/test_client.py -k "<model-name>"`
- `cd src_ts && npm run test -- -t "<model-name>"`
- Leave unrelated tests to CI.

## Supported-model registry

- `src_py/agenthub/registry.py` / `src_ts/src/registry.ts` list the supported models as entries of (model, base_url, client) plus input/output modalities, context window, and USD-stored pricing keyed by AgentHub's usage buckets. Keep both languages identical; the registry unit test constructs every entry through `AutoLLMClient`.
- For OpenRouter-hosted entries, pull authoritative data from the live models API `GET https://openrouter.ai/api/v1/models` (docs: https://openrouter.ai/docs/api/api-reference/models/list-all-models-and-their-properties): `pricing.prompt`/`completion` are USD per token (multiply by 1e6), plus `context_length` and `architecture.input_modalities`/`output_modalities`. The API lists chat models only — embedding models are absent and must be checked via their model pages.
- SiliconFlow publishes no pricing API; declare official CNY list prices with the `cny()` initializer (converted to USD storage at 7 CNY/USD).

## Record and ship

- Write `changelog/YYYY-MM-DD-<slug>.md` with the specifics: protocol differences found, config mapping decisions, notable capture findings.
- Add one brief line at the top of `CHANGELOG.md` linking to that file. The root file keeps a single line per change.
- Write `changelog/<version>/YYYY-MM-DD-<slug>.md` (folder of the upcoming release) with the specifics: protocol differences found, config mapping decisions, notable capture findings.
- Add one brief line at the top of that version's `changelog/<version>/README.md` linking to the file; the root `CHANGELOG.md` keeps one line per release, added at release preparation.
- Commit on a feature branch and open a PR with `gh pr create --base dev`; direct pushes to `dev` are rejected.
48 changes: 8 additions & 40 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,49 +1,17 @@
# Changelog

Here, we record the addition and removal times of models, major functional updates, bug fixes, and release times of key versions. Each entry keeps one brief line; the full details of a change live in a file under [changelog/](changelog/).
Here, we record the addition and removal times of models, major functional updates, bug fixes, and release times of key versions. Each release keeps one brief line here; the per-entry summaries live in `changelog/<version>/README.md`, and every entry links its detail file.

- [2026-07-20] Release version 0.4.0. Content items now carry an opaque `fidelity` payload that absorbs the former `signature`/`phase` fields (breaking), and OpenAI-compatible clients use it to replay thinking through exactly the reasoning field the upstream produced. ([details](changelog/2026-07-20-reasoning-field-fidelity.md))
- [2026-07-22] [Version 0.4.1](changelog/0.4.1/README.md): Kimi K3, the Gemini 3.6 generation (gemini-3.6-flash, gemini-3.5-flash-lite), and GLM-5.2 support, a supported-model registry with USD/CNY pricing, context windows, and modalities, and the `UnsupportedParameterError` parameter error class.

- [2026-07-17] Add the `agenthub-dev` skill that fixes the model-support development workflow, and the `changelog/` details directory. ([details](changelog/2026-07-17-agenthub-dev-skill.md))
- [2026-07-20] [Version 0.4.0](changelog/0.4.0/README.md): the `fidelity` content-item payload replaces `signature`/`phase` (breaking), OpenAI-compatible clients replay the exact upstream reasoning field, Claude 5 support, and hardened tool-call streaming.

- [2026-07-14] Raise `EmptyResponseError` when a model completes a response with thinking output only, since sending it back would fail with a 400 error. It and `ToolCallArgumentParseError` now inherit the new `AgentHubError` base class.
- [2026-06-01] [Version 0.3.3](changelog/0.3.3/README.md): OpenAI-compatible embedding input format.

- [2026-06-10] Support Claude 5 models.
- [2026-05-30] [Version 0.3.2](changelog/0.3.2/README.md): Claude 4.8, a generic OpenAI Chat Completions-compatible client, abort support, agent skills, and a broad model refresh.

- [2026-06-01] Release version 0.3.3. Support OpenAI-compatible embedding input format.
- [2026-04-28] [Version 0.3.1](changelog/0.3.1/README.md): Gemini TTS and image generation, GPT-5.5, the UModelVerse vendor, and automatic Claude caching.

- [2026-05-30] Release version 0.3.2.
- [2026-03-11] [Version 0.3.0](changelog/0.3.0/README.md): Claude 4.6 with adaptive thinking, GPT-5.4 with phase labels, Claude on Amazon Bedrock, and GLM-5.

- [2026-05-30] Support Claude 4.8 models and an OpenAI Chat Completions API-compatible client.

- [2026-05-28] Add abort support and agent skills.

- [2026-05-27] Support Gemini 3.5, Gemini Embedding 2, Claude 4.7, Kimi-K2.6, GLM-5.1, DeepSeek V4 and Qwen3.6 models. Qwen3 models are deprecated.

- [2026-04-28] Release version 0.3.1.

- [2026-04-28] Support Gemini 3.1 Flash TTS and GPT-5.5 models. Add UModelVerse vendor.

- [2026-04-22] Gemini 3.1 Flash Image (Nano Banana 2) model is supported.

- [2026-04-02] Switch to automatic caching for Claude 4.6 (but not for bedrock yet). Add message timestamp and round index to the tracer tool.

- [2026-03-11] GPT-5.4 is supported. We now add `phase` labels to assistant messages, and preserve and send them to the server. GPT-5.2 is deprecated.

- [2026-03-04] Claude 4.6 is supported. We switch to using the adaptive thinking and `effort` parameter instead of the thinking budget. Supports Gemini on Vertex AI. Add Kimi-K2.5 model. Claude 4.5 models are deprecated.

- [2026-02-26] Supports Claude on Amazon Bedrock. Bedrock requires image base64 encoding, we convert images to base64 in the client.

- [2026-02-15] Fix encrypted thinking message in Claude models. It needs to be preserved and sent to the server.

- [2026-02-15] Fix the calculation of token usage in from OpenRouter provider.

- [2026-02-13] Support GLM-5 model, GLM-4.7 is deprecated.

- [2026-01-21] Supports GPT-5.2 via the Responses API. Add Qwen3 models support.

- [2026-01-20] Support prompt caching for Claude 4.5 models.

- [2026-01-19] Support Claude 4.5 and GLM-4.7 models.

- [2026-01-16] Support Gemini 3 models.
- [2026-01-22] [Version 0.2.0](changelog/0.2.0/README.md): Gemini 3, Claude 4.5, GLM-4.7, GPT-5.2, and Qwen3 models, with prompt caching for Claude.
10 changes: 7 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,14 +42,18 @@ https://github.com/user-attachments/assets/c49a21a1-5bf9-4768-a76d-f73c9a03ca87

| Model Name | Vendor | Example Model ID | Input Modalities | Output Modalities |
| -------------- | ----------------------------------- | ---------------------- | ---------------- | ------------------------------ |
| Gemini 3-3.5 | Official/Google Vertex AI | `gemini-3.5-flash` | Text, Image | Text, Image, Speech, Embedding |
| Gemini 3-3.6 | Official/Google Vertex AI | `gemini-3.6-flash` | Text, Image | Text, Image, Speech, Embedding |
| Claude 4.6-5 | Official/Amazon Bedrock/UModelVerse | `claude-opus-4-8` | Text, Image | Text |
| GPT-5.4/5.5 | Official/UModelVerse | `gpt-5.5` | Text, Image | Text, Embedding |
| Kimi-K2.5/K2.6 | Official/OpenRouter/SiliconFlow | `kimi-k2.6` | Text, Image | Text |
| Kimi-K2.5/K2.6/K3 | Official/OpenRouter/SiliconFlow | `kimi-k3` | Text, Image | Text |
| DeepSeek V4 | Official/OpenRouter/SiliconFlow | `deepseek-v4-pro` | Text | Text |
| GLM-5.1 | Official/OpenRouter/SiliconFlow | `glm-5.1` | Text | Text |
| GLM-5.1/5.2 | Official/OpenRouter/SiliconFlow | `glm-5.2` | Text | Text |
| Qwen3.6 | OpenRouter/SiliconFlow/vLLM | `qwen/qwen3.6-35b-a3b` | Text, Image | Text, Embedding |

The full machine-readable list — model, base URL, client, input/output modalities, context
window, and per-million-token pricing in USD or CNY — is available via
`agenthub.list_supported_models()` (Python) / `listSupportedModels()` (TypeScript).

## Installation

### Python package
Expand Down
5 changes: 5 additions & 0 deletions changelog/0.2.0/2026-01-16-gemini-3.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Support Gemini 3 models

- Gemini 3 is supported through the Google GenAI SDK; this is the first supported model family.

*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.*
5 changes: 5 additions & 0 deletions changelog/0.2.0/2026-01-19-claude-4-5-glm-4-7.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Support Claude 4.5 and GLM-4.7 models

- Claude 4.5 and GLM-4.7 are supported (TypeScript Claude 4.5 client in #40).

*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.*
5 changes: 5 additions & 0 deletions changelog/0.2.0/2026-01-20-claude-prompt-caching.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Support prompt caching for Claude 4.5 models

- Prompt caching is enabled for Claude 4.5 via `cache_control`, exposed through the `prompt_caching` config.

*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.*
6 changes: 6 additions & 0 deletions changelog/0.2.0/2026-01-21-gpt-5-2-qwen3.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Support GPT-5.2 via the Responses API; add Qwen3 models

- GPT-5.2 is supported through the OpenAI Responses API (TypeScript client in #42).
- Added Qwen3 model support (TypeScript client in #44).

*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.*
11 changes: 11 additions & 0 deletions changelog/0.2.0/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Version 0.2.0

Released on 2026-01-22.

- [2026-01-21] Supports GPT-5.2 via the Responses API. Add Qwen3 models support. ([details](2026-01-21-gpt-5-2-qwen3.md))

- [2026-01-20] Support prompt caching for Claude 4.5 models. ([details](2026-01-20-claude-prompt-caching.md))

- [2026-01-19] Support Claude 4.5 and GLM-4.7 models. ([details](2026-01-19-claude-4-5-glm-4-7.md))

- [2026-01-16] Support Gemini 3 models. ([details](2026-01-16-gemini-3.md))
6 changes: 6 additions & 0 deletions changelog/0.3.0/2026-02-13-glm-5.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Support GLM-5; deprecate GLM-4.7

- The GLM client folder was renamed from `glm4_7` to `glm5` with multi-version routing (#71).
- GLM-4.7 is deprecated.

*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.*
5 changes: 5 additions & 0 deletions changelog/0.3.0/2026-02-15-claude-encrypted-thinking-fix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Fix encrypted thinking messages in Claude models

- Encrypted (redacted) thinking blocks from Claude must be preserved in history and sent back to the server unchanged; the client no longer drops them (#74).

*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.*
5 changes: 5 additions & 0 deletions changelog/0.3.0/2026-02-15-openrouter-usage-fix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Fix token usage calculation from the OpenRouter provider

- OpenRouter occasionally omits reasoning tokens from completion tokens; the usage metadata calculation compensates for it in both Python and TypeScript (#73).

*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.*
6 changes: 6 additions & 0 deletions changelog/0.3.0/2026-02-26-claude-bedrock.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Support Claude on Amazon Bedrock

- Claude models are supported through Amazon Bedrock (#79).
- Bedrock does not accept image URLs, so the client fetches images and converts them to base64 before sending.

*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.*
8 changes: 8 additions & 0 deletions changelog/0.3.0/2026-03-04-claude-4-6-adaptive-thinking.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Support Claude 4.6 with adaptive thinking; Vertex AI; Kimi-K2.5; deprecate Claude 4.5

- Claude 4.6 is supported using the adaptive thinking and `effort` parameter instead of the thinking budget (#82).
- Gemini is supported on Vertex AI.
- Added the Kimi-K2.5 model (#81).
- Claude 4.5 models are deprecated.

*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.*
7 changes: 7 additions & 0 deletions changelog/0.3.0/2026-03-11-gpt-5-4-phase-labels.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Support GPT-5.4 with phase labels; deprecate GPT-5.2

- GPT-5.4 is supported via the Responses API (#87).
- Assistant messages now carry `phase` labels, which are preserved and sent back to the server on replay.
- GPT-5.2 is deprecated.

*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.*
15 changes: 15 additions & 0 deletions changelog/0.3.0/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Version 0.3.0

Released on 2026-03-11.

- [2026-03-11] GPT-5.4 is supported. We now add `phase` labels to assistant messages, and preserve and send them to the server. GPT-5.2 is deprecated. ([details](2026-03-11-gpt-5-4-phase-labels.md))

- [2026-03-04] Claude 4.6 is supported. We switch to using the adaptive thinking and `effort` parameter instead of the thinking budget. Supports Gemini on Vertex AI. Add Kimi-K2.5 model. Claude 4.5 models are deprecated. ([details](2026-03-04-claude-4-6-adaptive-thinking.md))

- [2026-02-26] Supports Claude on Amazon Bedrock. Bedrock requires image base64 encoding, we convert images to base64 in the client. ([details](2026-02-26-claude-bedrock.md))

- [2026-02-15] Fix encrypted thinking message in Claude models. It needs to be preserved and sent to the server. ([details](2026-02-15-claude-encrypted-thinking-fix.md))

- [2026-02-15] Fix the calculation of token usage in from OpenRouter provider. ([details](2026-02-15-openrouter-usage-fix.md))

- [2026-02-13] Support GLM-5 model, GLM-4.7 is deprecated. ([details](2026-02-13-glm-5.md))
6 changes: 6 additions & 0 deletions changelog/0.3.1/2026-04-02-claude-auto-caching-tracer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Automatic caching for Claude 4.6; tracer timestamps and round index

- Claude 4.6 switched to automatic prompt caching by moving `cache_control` from message content items to a top-level API parameter (#95); Bedrock still uses per-message cache control.
- `UniMessage`/`UniEvent` gained `created_at` timestamps, and the tracer tracks message rounds (#96).

*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.*
5 changes: 5 additions & 0 deletions changelog/0.3.1/2026-04-22-nano-banana-2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Support Gemini 3.1 Flash Image (Nano Banana 2)

- Gemini image generation is supported, including aspect ratio and image size configuration via `image_config` (#108).

*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.*
7 changes: 7 additions & 0 deletions changelog/0.3.1/2026-04-28-gemini-tts-gpt-5-5-modelverse.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Support Gemini 3.1 Flash TTS and GPT-5.5; add the UModelVerse vendor

- Gemini TTS is supported across clients with speaker/voice configuration (#110, #113).
- GPT-5.5 models are supported and became the default GPT option; GPT-5.4 routes through the GPT-5.5 client (#112).
- Added the UModelVerse vendor.

*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.*
9 changes: 9 additions & 0 deletions changelog/0.3.1/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Version 0.3.1

Released on 2026-04-28.

- [2026-04-28] Support Gemini 3.1 Flash TTS and GPT-5.5 models. Add UModelVerse vendor. ([details](2026-04-28-gemini-tts-gpt-5-5-modelverse.md))

- [2026-04-22] Gemini 3.1 Flash Image (Nano Banana 2) model is supported. ([details](2026-04-22-nano-banana-2.md))

- [2026-04-02] Switch to automatic caching for Claude 4.6 (but not for bedrock yet). Add message timestamp and round index to the tracer tool. ([details](2026-04-02-claude-auto-caching-tracer.md))
6 changes: 6 additions & 0 deletions changelog/0.3.2/2026-05-27-model-refresh.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Model refresh: Gemini 3.5, Gemini Embedding 2, Claude 4.7, Kimi-K2.6, GLM-5.1, DeepSeek V4, Qwen3.6

- Added support for Gemini 3.5, Gemini Embedding 2 (#126, #127), Claude 4.7, Kimi-K2.6, GLM-5.1, DeepSeek V4 (#124), and Qwen3.6 (#137) models.
- Qwen3 models are deprecated.

*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.*
6 changes: 6 additions & 0 deletions changelog/0.3.2/2026-05-28-abort-support-and-skills.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Add abort support and agent skills

- Streaming requests accept an abort signal in both Python and TypeScript (#128), the abort waiter is reused during streaming (#133), and the playground gained an abort control (#130).
- Added the `agenthub-python` and `agenthub-typescript` SDK usage skills under `skills/` (#121, #129).

*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.*
6 changes: 6 additions & 0 deletions changelog/0.3.2/2026-05-30-claude-4-8-openai-compatible.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Support Claude 4.8 and an OpenAI Chat Completions-compatible client

- Claude 4.8 models are supported.
- Added a generic OpenAI Chat Completions API-compatible client with explicit `client_type` routing (#138), so any Chat Completions-style endpoint can be used without a dedicated protocol folder.

*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.*
Loading
Loading