diff --git a/.agents/skills/agenthub-dev/SKILL.md b/.agents/skills/agenthub-dev/SKILL.md index 97b04500..9d66401c 100644 --- a/.agents/skills/agenthub-dev/SKILL.md +++ b/.agents/skills/agenthub-dev/SKILL.md @@ -18,8 +18,8 @@ src_ts/src// TypeScript client, mirrors the Python folder src_ts/src/autoClient.ts TypeScript routing, mirrors auto_client.py src_py/tests/test_client.py Parameterized e2e tests (env-gated AVAILABLE_MODELS) src_ts/tests/client.test.ts Same for TypeScript -changelog/ One detail file per CHANGELOG.md entry -CHANGELOG.md Brief one-line entries linking into changelog/ +changelog// Release summary (README.md) plus one detail file per entry +CHANGELOG.md One brief line per release linking into changelog/ ``` ## Stage 1 — Sync official docs into `llmsdk_docs/` @@ -43,22 +43,32 @@ CHANGELOG.md Brief one-line entries linking into changelog/ - One folder per wire protocol, named after the newest model generation that uses it. Diff the new protocol (capture + docs) against the closest existing folder: - Any difference between generations, even a single key name, means a separate folder per generation (e.g. `claude4_6/` vs `claude5/`). - Only an identical wire protocol may share a folder; name it after the newest generation (rename and reroute if needed). This is how `claude5/` serves Claude 4.7, 4.8, and 5. + - When the old and new generations' implementations differ, keep the old model supported: leave its client folder and routing in place and add a new client folder for the new model. Never delete or rewire away an old model's client unless the user explicitly instructs it. - `auto_client.py` / `autoClient.ts` route model names by explicit version matching only, never a bare substring like `"claude" in model`. - Conversion must be bijective: a wire message converted to `UniMessage`/`UniEvent` and back must reproduce the original exactly, including `fidelity` payloads (thinking signatures, phase labels, reasoning field names) and tool-call IDs. Verify against the captured exchange. - `UniConfig` keys rarely map one-to-one onto provider config keys. **Stop and ask**: list every non-obvious mapping and confirm it with the user before coding. Never decide silently. +- Every `ThinkingLevel` must stay usable on every client — never raise for a thinking level. Map each level to the closest level the model supports and degrade silently when a level has no exact equivalent (e.g. `gemini3` maps `NONE` to `MINIMAL`; `kimi_k3` maps `NONE` to `low` because K3 cannot disable reasoning). +- `temperature` and `tool_choice` (and other unsupported parameter values, e.g. `prompt_caching`) may reject with an exception, but must raise the AgentHub-specific `UnsupportedParameterError` from `errors.py` / `errors.ts`, never a bare `ValueError`/`Error`. Keep the message wording consistent with existing clients (containing "not support"). - Implement Python and TypeScript together with identical behavior. ## Stage 4 — Verify - Register the model in the env-gated `AVAILABLE_MODELS` lists of both test files with correct capability flags. Do not add model-specific test functions or files. +- `AVAILABLE_MODELS` keeps only the newest version of each model family per provider block (e.g. gemini-3.6-flash, not gemini-3.5-flash or 3.5-flash-lite as well). When a newer generation lands, replace the older entry — the old client folder stays supported and routed (see Stage 3) but is no longer e2e-tested. - Static checks: `make lint` in `src_py/`; `npm run lint` and `npm run build` in `src_ts/`. - Run only the new model's e2e tests; the full suites are slow and spend real API quota: - `cd src_py && uv run pytest -vvv tests/test_client.py -k ""` - `cd src_ts && npm run test -- -t ""` - Leave unrelated tests to CI. +## Supported-model registry + +- `src_py/agenthub/registry.py` / `src_ts/src/registry.ts` list the supported models as entries of (model, base_url, client) plus input/output modalities, context window, and USD-stored pricing keyed by AgentHub's usage buckets. Keep both languages identical; the registry unit test constructs every entry through `AutoLLMClient`. +- For OpenRouter-hosted entries, pull authoritative data from the live models API `GET https://openrouter.ai/api/v1/models` (docs: https://openrouter.ai/docs/api/api-reference/models/list-all-models-and-their-properties): `pricing.prompt`/`completion` are USD per token (multiply by 1e6), plus `context_length` and `architecture.input_modalities`/`output_modalities`. The API lists chat models only — embedding models are absent and must be checked via their model pages. +- SiliconFlow publishes no pricing API; declare official CNY list prices with the `cny()` initializer (converted to USD storage at 7 CNY/USD). + ## Record and ship -- Write `changelog/YYYY-MM-DD-.md` with the specifics: protocol differences found, config mapping decisions, notable capture findings. -- Add one brief line at the top of `CHANGELOG.md` linking to that file. The root file keeps a single line per change. +- Write `changelog//YYYY-MM-DD-.md` (folder of the upcoming release) with the specifics: protocol differences found, config mapping decisions, notable capture findings. +- Add one brief line at the top of that version's `changelog//README.md` linking to the file; the root `CHANGELOG.md` keeps one line per release, added at release preparation. - Commit on a feature branch and open a PR with `gh pr create --base dev`; direct pushes to `dev` are rejected. diff --git a/CHANGELOG.md b/CHANGELOG.md index e6404698..ffa6a00d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,49 +1,17 @@ # Changelog -Here, we record the addition and removal times of models, major functional updates, bug fixes, and release times of key versions. Each entry keeps one brief line; the full details of a change live in a file under [changelog/](changelog/). +Here, we record the addition and removal times of models, major functional updates, bug fixes, and release times of key versions. Each release keeps one brief line here; the per-entry summaries live in `changelog//README.md`, and every entry links its detail file. -- [2026-07-20] Release version 0.4.0. Content items now carry an opaque `fidelity` payload that absorbs the former `signature`/`phase` fields (breaking), and OpenAI-compatible clients use it to replay thinking through exactly the reasoning field the upstream produced. ([details](changelog/2026-07-20-reasoning-field-fidelity.md)) +- [2026-07-22] [Version 0.4.1](changelog/0.4.1/README.md): Kimi K3, the Gemini 3.6 generation (gemini-3.6-flash, gemini-3.5-flash-lite), and GLM-5.2 support, a supported-model registry with USD/CNY pricing, context windows, and modalities, and the `UnsupportedParameterError` parameter error class. -- [2026-07-17] Add the `agenthub-dev` skill that fixes the model-support development workflow, and the `changelog/` details directory. ([details](changelog/2026-07-17-agenthub-dev-skill.md)) +- [2026-07-20] [Version 0.4.0](changelog/0.4.0/README.md): the `fidelity` content-item payload replaces `signature`/`phase` (breaking), OpenAI-compatible clients replay the exact upstream reasoning field, Claude 5 support, and hardened tool-call streaming. -- [2026-07-14] Raise `EmptyResponseError` when a model completes a response with thinking output only, since sending it back would fail with a 400 error. It and `ToolCallArgumentParseError` now inherit the new `AgentHubError` base class. +- [2026-06-01] [Version 0.3.3](changelog/0.3.3/README.md): OpenAI-compatible embedding input format. -- [2026-06-10] Support Claude 5 models. +- [2026-05-30] [Version 0.3.2](changelog/0.3.2/README.md): Claude 4.8, a generic OpenAI Chat Completions-compatible client, abort support, agent skills, and a broad model refresh. -- [2026-06-01] Release version 0.3.3. Support OpenAI-compatible embedding input format. +- [2026-04-28] [Version 0.3.1](changelog/0.3.1/README.md): Gemini TTS and image generation, GPT-5.5, the UModelVerse vendor, and automatic Claude caching. -- [2026-05-30] Release version 0.3.2. +- [2026-03-11] [Version 0.3.0](changelog/0.3.0/README.md): Claude 4.6 with adaptive thinking, GPT-5.4 with phase labels, Claude on Amazon Bedrock, and GLM-5. -- [2026-05-30] Support Claude 4.8 models and an OpenAI Chat Completions API-compatible client. - -- [2026-05-28] Add abort support and agent skills. - -- [2026-05-27] Support Gemini 3.5, Gemini Embedding 2, Claude 4.7, Kimi-K2.6, GLM-5.1, DeepSeek V4 and Qwen3.6 models. Qwen3 models are deprecated. - -- [2026-04-28] Release version 0.3.1. - -- [2026-04-28] Support Gemini 3.1 Flash TTS and GPT-5.5 models. Add UModelVerse vendor. - -- [2026-04-22] Gemini 3.1 Flash Image (Nano Banana 2) model is supported. - -- [2026-04-02] Switch to automatic caching for Claude 4.6 (but not for bedrock yet). Add message timestamp and round index to the tracer tool. - -- [2026-03-11] GPT-5.4 is supported. We now add `phase` labels to assistant messages, and preserve and send them to the server. GPT-5.2 is deprecated. - -- [2026-03-04] Claude 4.6 is supported. We switch to using the adaptive thinking and `effort` parameter instead of the thinking budget. Supports Gemini on Vertex AI. Add Kimi-K2.5 model. Claude 4.5 models are deprecated. - -- [2026-02-26] Supports Claude on Amazon Bedrock. Bedrock requires image base64 encoding, we convert images to base64 in the client. - -- [2026-02-15] Fix encrypted thinking message in Claude models. It needs to be preserved and sent to the server. - -- [2026-02-15] Fix the calculation of token usage in from OpenRouter provider. - -- [2026-02-13] Support GLM-5 model, GLM-4.7 is deprecated. - -- [2026-01-21] Supports GPT-5.2 via the Responses API. Add Qwen3 models support. - -- [2026-01-20] Support prompt caching for Claude 4.5 models. - -- [2026-01-19] Support Claude 4.5 and GLM-4.7 models. - -- [2026-01-16] Support Gemini 3 models. +- [2026-01-22] [Version 0.2.0](changelog/0.2.0/README.md): Gemini 3, Claude 4.5, GLM-4.7, GPT-5.2, and Qwen3 models, with prompt caching for Claude. diff --git a/README.md b/README.md index f0a98103..0cda3330 100644 --- a/README.md +++ b/README.md @@ -42,14 +42,18 @@ https://github.com/user-attachments/assets/c49a21a1-5bf9-4768-a76d-f73c9a03ca87 | Model Name | Vendor | Example Model ID | Input Modalities | Output Modalities | | -------------- | ----------------------------------- | ---------------------- | ---------------- | ------------------------------ | -| Gemini 3-3.5 | Official/Google Vertex AI | `gemini-3.5-flash` | Text, Image | Text, Image, Speech, Embedding | +| Gemini 3-3.6 | Official/Google Vertex AI | `gemini-3.6-flash` | Text, Image | Text, Image, Speech, Embedding | | Claude 4.6-5 | Official/Amazon Bedrock/UModelVerse | `claude-opus-4-8` | Text, Image | Text | | GPT-5.4/5.5 | Official/UModelVerse | `gpt-5.5` | Text, Image | Text, Embedding | -| Kimi-K2.5/K2.6 | Official/OpenRouter/SiliconFlow | `kimi-k2.6` | Text, Image | Text | +| Kimi-K2.5/K2.6/K3 | Official/OpenRouter/SiliconFlow | `kimi-k3` | Text, Image | Text | | DeepSeek V4 | Official/OpenRouter/SiliconFlow | `deepseek-v4-pro` | Text | Text | -| GLM-5.1 | Official/OpenRouter/SiliconFlow | `glm-5.1` | Text | Text | +| GLM-5.1/5.2 | Official/OpenRouter/SiliconFlow | `glm-5.2` | Text | Text | | Qwen3.6 | OpenRouter/SiliconFlow/vLLM | `qwen/qwen3.6-35b-a3b` | Text, Image | Text, Embedding | +The full machine-readable list — model, base URL, client, input/output modalities, context +window, and per-million-token pricing in USD or CNY — is available via +`agenthub.list_supported_models()` (Python) / `listSupportedModels()` (TypeScript). + ## Installation ### Python package diff --git a/changelog/0.2.0/2026-01-16-gemini-3.md b/changelog/0.2.0/2026-01-16-gemini-3.md new file mode 100644 index 00000000..bafd4e4b --- /dev/null +++ b/changelog/0.2.0/2026-01-16-gemini-3.md @@ -0,0 +1,5 @@ +# Support Gemini 3 models + +- Gemini 3 is supported through the Google GenAI SDK; this is the first supported model family. + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.2.0/2026-01-19-claude-4-5-glm-4-7.md b/changelog/0.2.0/2026-01-19-claude-4-5-glm-4-7.md new file mode 100644 index 00000000..48346e3a --- /dev/null +++ b/changelog/0.2.0/2026-01-19-claude-4-5-glm-4-7.md @@ -0,0 +1,5 @@ +# Support Claude 4.5 and GLM-4.7 models + +- Claude 4.5 and GLM-4.7 are supported (TypeScript Claude 4.5 client in #40). + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.2.0/2026-01-20-claude-prompt-caching.md b/changelog/0.2.0/2026-01-20-claude-prompt-caching.md new file mode 100644 index 00000000..0eb1a3b5 --- /dev/null +++ b/changelog/0.2.0/2026-01-20-claude-prompt-caching.md @@ -0,0 +1,5 @@ +# Support prompt caching for Claude 4.5 models + +- Prompt caching is enabled for Claude 4.5 via `cache_control`, exposed through the `prompt_caching` config. + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.2.0/2026-01-21-gpt-5-2-qwen3.md b/changelog/0.2.0/2026-01-21-gpt-5-2-qwen3.md new file mode 100644 index 00000000..3588f7d7 --- /dev/null +++ b/changelog/0.2.0/2026-01-21-gpt-5-2-qwen3.md @@ -0,0 +1,6 @@ +# Support GPT-5.2 via the Responses API; add Qwen3 models + +- GPT-5.2 is supported through the OpenAI Responses API (TypeScript client in #42). +- Added Qwen3 model support (TypeScript client in #44). + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.2.0/README.md b/changelog/0.2.0/README.md new file mode 100644 index 00000000..9bcd4bac --- /dev/null +++ b/changelog/0.2.0/README.md @@ -0,0 +1,11 @@ +# Version 0.2.0 + +Released on 2026-01-22. + +- [2026-01-21] Supports GPT-5.2 via the Responses API. Add Qwen3 models support. ([details](2026-01-21-gpt-5-2-qwen3.md)) + +- [2026-01-20] Support prompt caching for Claude 4.5 models. ([details](2026-01-20-claude-prompt-caching.md)) + +- [2026-01-19] Support Claude 4.5 and GLM-4.7 models. ([details](2026-01-19-claude-4-5-glm-4-7.md)) + +- [2026-01-16] Support Gemini 3 models. ([details](2026-01-16-gemini-3.md)) diff --git a/changelog/0.3.0/2026-02-13-glm-5.md b/changelog/0.3.0/2026-02-13-glm-5.md new file mode 100644 index 00000000..2bc0f3f1 --- /dev/null +++ b/changelog/0.3.0/2026-02-13-glm-5.md @@ -0,0 +1,6 @@ +# Support GLM-5; deprecate GLM-4.7 + +- The GLM client folder was renamed from `glm4_7` to `glm5` with multi-version routing (#71). +- GLM-4.7 is deprecated. + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.3.0/2026-02-15-claude-encrypted-thinking-fix.md b/changelog/0.3.0/2026-02-15-claude-encrypted-thinking-fix.md new file mode 100644 index 00000000..48a3b458 --- /dev/null +++ b/changelog/0.3.0/2026-02-15-claude-encrypted-thinking-fix.md @@ -0,0 +1,5 @@ +# Fix encrypted thinking messages in Claude models + +- Encrypted (redacted) thinking blocks from Claude must be preserved in history and sent back to the server unchanged; the client no longer drops them (#74). + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.3.0/2026-02-15-openrouter-usage-fix.md b/changelog/0.3.0/2026-02-15-openrouter-usage-fix.md new file mode 100644 index 00000000..3f1e9c20 --- /dev/null +++ b/changelog/0.3.0/2026-02-15-openrouter-usage-fix.md @@ -0,0 +1,5 @@ +# Fix token usage calculation from the OpenRouter provider + +- OpenRouter occasionally omits reasoning tokens from completion tokens; the usage metadata calculation compensates for it in both Python and TypeScript (#73). + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.3.0/2026-02-26-claude-bedrock.md b/changelog/0.3.0/2026-02-26-claude-bedrock.md new file mode 100644 index 00000000..bfb46c55 --- /dev/null +++ b/changelog/0.3.0/2026-02-26-claude-bedrock.md @@ -0,0 +1,6 @@ +# Support Claude on Amazon Bedrock + +- Claude models are supported through Amazon Bedrock (#79). +- Bedrock does not accept image URLs, so the client fetches images and converts them to base64 before sending. + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.3.0/2026-03-04-claude-4-6-adaptive-thinking.md b/changelog/0.3.0/2026-03-04-claude-4-6-adaptive-thinking.md new file mode 100644 index 00000000..6125cf29 --- /dev/null +++ b/changelog/0.3.0/2026-03-04-claude-4-6-adaptive-thinking.md @@ -0,0 +1,8 @@ +# Support Claude 4.6 with adaptive thinking; Vertex AI; Kimi-K2.5; deprecate Claude 4.5 + +- Claude 4.6 is supported using the adaptive thinking and `effort` parameter instead of the thinking budget (#82). +- Gemini is supported on Vertex AI. +- Added the Kimi-K2.5 model (#81). +- Claude 4.5 models are deprecated. + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.3.0/2026-03-11-gpt-5-4-phase-labels.md b/changelog/0.3.0/2026-03-11-gpt-5-4-phase-labels.md new file mode 100644 index 00000000..dbc2dbcd --- /dev/null +++ b/changelog/0.3.0/2026-03-11-gpt-5-4-phase-labels.md @@ -0,0 +1,7 @@ +# Support GPT-5.4 with phase labels; deprecate GPT-5.2 + +- GPT-5.4 is supported via the Responses API (#87). +- Assistant messages now carry `phase` labels, which are preserved and sent back to the server on replay. +- GPT-5.2 is deprecated. + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.3.0/README.md b/changelog/0.3.0/README.md new file mode 100644 index 00000000..efb63f72 --- /dev/null +++ b/changelog/0.3.0/README.md @@ -0,0 +1,15 @@ +# Version 0.3.0 + +Released on 2026-03-11. + +- [2026-03-11] GPT-5.4 is supported. We now add `phase` labels to assistant messages, and preserve and send them to the server. GPT-5.2 is deprecated. ([details](2026-03-11-gpt-5-4-phase-labels.md)) + +- [2026-03-04] Claude 4.6 is supported. We switch to using the adaptive thinking and `effort` parameter instead of the thinking budget. Supports Gemini on Vertex AI. Add Kimi-K2.5 model. Claude 4.5 models are deprecated. ([details](2026-03-04-claude-4-6-adaptive-thinking.md)) + +- [2026-02-26] Supports Claude on Amazon Bedrock. Bedrock requires image base64 encoding, we convert images to base64 in the client. ([details](2026-02-26-claude-bedrock.md)) + +- [2026-02-15] Fix encrypted thinking message in Claude models. It needs to be preserved and sent to the server. ([details](2026-02-15-claude-encrypted-thinking-fix.md)) + +- [2026-02-15] Fix the calculation of token usage in from OpenRouter provider. ([details](2026-02-15-openrouter-usage-fix.md)) + +- [2026-02-13] Support GLM-5 model, GLM-4.7 is deprecated. ([details](2026-02-13-glm-5.md)) diff --git a/changelog/0.3.1/2026-04-02-claude-auto-caching-tracer.md b/changelog/0.3.1/2026-04-02-claude-auto-caching-tracer.md new file mode 100644 index 00000000..ea2a8d5b --- /dev/null +++ b/changelog/0.3.1/2026-04-02-claude-auto-caching-tracer.md @@ -0,0 +1,6 @@ +# Automatic caching for Claude 4.6; tracer timestamps and round index + +- Claude 4.6 switched to automatic prompt caching by moving `cache_control` from message content items to a top-level API parameter (#95); Bedrock still uses per-message cache control. +- `UniMessage`/`UniEvent` gained `created_at` timestamps, and the tracer tracks message rounds (#96). + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.3.1/2026-04-22-nano-banana-2.md b/changelog/0.3.1/2026-04-22-nano-banana-2.md new file mode 100644 index 00000000..a446bb85 --- /dev/null +++ b/changelog/0.3.1/2026-04-22-nano-banana-2.md @@ -0,0 +1,5 @@ +# Support Gemini 3.1 Flash Image (Nano Banana 2) + +- Gemini image generation is supported, including aspect ratio and image size configuration via `image_config` (#108). + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.3.1/2026-04-28-gemini-tts-gpt-5-5-modelverse.md b/changelog/0.3.1/2026-04-28-gemini-tts-gpt-5-5-modelverse.md new file mode 100644 index 00000000..70b84ac1 --- /dev/null +++ b/changelog/0.3.1/2026-04-28-gemini-tts-gpt-5-5-modelverse.md @@ -0,0 +1,7 @@ +# Support Gemini 3.1 Flash TTS and GPT-5.5; add the UModelVerse vendor + +- Gemini TTS is supported across clients with speaker/voice configuration (#110, #113). +- GPT-5.5 models are supported and became the default GPT option; GPT-5.4 routes through the GPT-5.5 client (#112). +- Added the UModelVerse vendor. + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.3.1/README.md b/changelog/0.3.1/README.md new file mode 100644 index 00000000..1f8119a2 --- /dev/null +++ b/changelog/0.3.1/README.md @@ -0,0 +1,9 @@ +# Version 0.3.1 + +Released on 2026-04-28. + +- [2026-04-28] Support Gemini 3.1 Flash TTS and GPT-5.5 models. Add UModelVerse vendor. ([details](2026-04-28-gemini-tts-gpt-5-5-modelverse.md)) + +- [2026-04-22] Gemini 3.1 Flash Image (Nano Banana 2) model is supported. ([details](2026-04-22-nano-banana-2.md)) + +- [2026-04-02] Switch to automatic caching for Claude 4.6 (but not for bedrock yet). Add message timestamp and round index to the tracer tool. ([details](2026-04-02-claude-auto-caching-tracer.md)) diff --git a/changelog/0.3.2/2026-05-27-model-refresh.md b/changelog/0.3.2/2026-05-27-model-refresh.md new file mode 100644 index 00000000..8cfc1d3a --- /dev/null +++ b/changelog/0.3.2/2026-05-27-model-refresh.md @@ -0,0 +1,6 @@ +# Model refresh: Gemini 3.5, Gemini Embedding 2, Claude 4.7, Kimi-K2.6, GLM-5.1, DeepSeek V4, Qwen3.6 + +- Added support for Gemini 3.5, Gemini Embedding 2 (#126, #127), Claude 4.7, Kimi-K2.6, GLM-5.1, DeepSeek V4 (#124), and Qwen3.6 (#137) models. +- Qwen3 models are deprecated. + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.3.2/2026-05-28-abort-support-and-skills.md b/changelog/0.3.2/2026-05-28-abort-support-and-skills.md new file mode 100644 index 00000000..cdb2c604 --- /dev/null +++ b/changelog/0.3.2/2026-05-28-abort-support-and-skills.md @@ -0,0 +1,6 @@ +# Add abort support and agent skills + +- Streaming requests accept an abort signal in both Python and TypeScript (#128), the abort waiter is reused during streaming (#133), and the playground gained an abort control (#130). +- Added the `agenthub-python` and `agenthub-typescript` SDK usage skills under `skills/` (#121, #129). + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.3.2/2026-05-30-claude-4-8-openai-compatible.md b/changelog/0.3.2/2026-05-30-claude-4-8-openai-compatible.md new file mode 100644 index 00000000..549aa569 --- /dev/null +++ b/changelog/0.3.2/2026-05-30-claude-4-8-openai-compatible.md @@ -0,0 +1,6 @@ +# Support Claude 4.8 and an OpenAI Chat Completions-compatible client + +- Claude 4.8 models are supported. +- Added a generic OpenAI Chat Completions API-compatible client with explicit `client_type` routing (#138), so any Chat Completions-style endpoint can be used without a dedicated protocol folder. + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.3.2/README.md b/changelog/0.3.2/README.md new file mode 100644 index 00000000..d097758d --- /dev/null +++ b/changelog/0.3.2/README.md @@ -0,0 +1,9 @@ +# Version 0.3.2 + +Released on 2026-05-30. + +- [2026-05-30] Support Claude 4.8 models and an OpenAI Chat Completions API-compatible client. ([details](2026-05-30-claude-4-8-openai-compatible.md)) + +- [2026-05-28] Add abort support and agent skills. ([details](2026-05-28-abort-support-and-skills.md)) + +- [2026-05-27] Support Gemini 3.5, Gemini Embedding 2, Claude 4.7, Kimi-K2.6, GLM-5.1, DeepSeek V4 and Qwen3.6 models. Qwen3 models are deprecated. ([details](2026-05-27-model-refresh.md)) diff --git a/changelog/0.3.3/2026-06-01-openai-embedding.md b/changelog/0.3.3/2026-06-01-openai-embedding.md new file mode 100644 index 00000000..cb50d046 --- /dev/null +++ b/changelog/0.3.3/2026-06-01-openai-embedding.md @@ -0,0 +1,5 @@ +# Support OpenAI-compatible embedding input format + +- Added text embedding support for OpenAI-compatible endpoints (#145), allowed empty embedding inputs (#146), and split the embedding client into its own `openai_embedding/` folder with the `openai-embedding-compatible` client type (#148). + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.3.3/README.md b/changelog/0.3.3/README.md new file mode 100644 index 00000000..c4955b8a --- /dev/null +++ b/changelog/0.3.3/README.md @@ -0,0 +1,5 @@ +# Version 0.3.3 + +Released on 2026-06-01. + +- [2026-06-01] Support OpenAI-compatible embedding input format. ([details](2026-06-01-openai-embedding.md)) diff --git a/changelog/0.4.0/2026-06-10-claude-5.md b/changelog/0.4.0/2026-06-10-claude-5.md new file mode 100644 index 00000000..8bcd3a9d --- /dev/null +++ b/changelog/0.4.0/2026-06-10-claude-5.md @@ -0,0 +1,5 @@ +# Support Claude 5 models + +- Claude 5 is served by the `claude5/` protocol folder, which also routes Claude 4.7 and 4.8 since the wire protocol is identical (#149). + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/0.4.0/2026-07-14-empty-response-error.md b/changelog/0.4.0/2026-07-14-empty-response-error.md new file mode 100644 index 00000000..e8d7bd66 --- /dev/null +++ b/changelog/0.4.0/2026-07-14-empty-response-error.md @@ -0,0 +1,6 @@ +# Reject thinking-only responses and introduce the AgentHubError base class + +- A response that completes with thinking output only now raises `EmptyResponseError` as soon as the stream ends, because replaying a thinking-only assistant message on the next turn fails with a 400 error (#157). +- `EmptyResponseError` and `ToolCallArgumentParseError` (raised for malformed streamed tool arguments, #155) now inherit the new `AgentHubError` base class, so callers can catch all AgentHub-raised errors in one place. + +*Backfilled when the changelog was reorganized by release version; see the git history of the release range for full context.* diff --git a/changelog/2026-07-17-agenthub-dev-skill.md b/changelog/0.4.0/2026-07-17-agenthub-dev-skill.md similarity index 100% rename from changelog/2026-07-17-agenthub-dev-skill.md rename to changelog/0.4.0/2026-07-17-agenthub-dev-skill.md diff --git a/changelog/2026-07-20-reasoning-field-fidelity.md b/changelog/0.4.0/2026-07-20-reasoning-field-fidelity.md similarity index 100% rename from changelog/2026-07-20-reasoning-field-fidelity.md rename to changelog/0.4.0/2026-07-20-reasoning-field-fidelity.md diff --git a/changelog/0.4.0/README.md b/changelog/0.4.0/README.md new file mode 100644 index 00000000..82a9c427 --- /dev/null +++ b/changelog/0.4.0/README.md @@ -0,0 +1,11 @@ +# Version 0.4.0 + +Released on 2026-07-20. + +- [2026-07-20] Content items now carry an opaque `fidelity` payload that absorbs the former `signature`/`phase` fields (breaking), and OpenAI-compatible clients use it to replay thinking through exactly the reasoning field the upstream produced. ([details](2026-07-20-reasoning-field-fidelity.md)) + +- [2026-07-17] Add the `agenthub-dev` skill that fixes the model-support development workflow, and the `changelog/` details directory. ([details](2026-07-17-agenthub-dev-skill.md)) + +- [2026-07-14] Raise `EmptyResponseError` when a model completes a response with thinking output only, since sending it back would fail with a 400 error. It and `ToolCallArgumentParseError` now inherit the new `AgentHubError` base class. ([details](2026-07-14-empty-response-error.md)) + +- [2026-06-10] Support Claude 5 models. ([details](2026-06-10-claude-5.md)) diff --git a/changelog/0.4.1/2026-07-22-kimi-k3-gemini-3-6-registry.md b/changelog/0.4.1/2026-07-22-kimi-k3-gemini-3-6-registry.md new file mode 100644 index 00000000..90aeb9b1 --- /dev/null +++ b/changelog/0.4.1/2026-07-22-kimi-k3-gemini-3-6-registry.md @@ -0,0 +1,90 @@ +# Kimi K3, Gemini 3.6 generation, supported-model registry, and UnsupportedParameterError + +## What changed + +- New `kimi_k3/` clients (Python and TypeScript) for Moonshot's `kimi-k3`. +- New `gemini3_6/` clients for the Gemini 3.6 protocol generation: `gemini-3.6-flash` and + `gemini-3.5-flash-lite` (which belongs to this generation despite its 3.5 name). +- New `glm5_2/` clients for Z.AI's GLM-5.2 (`glm5_1/` stays untouched): adds the top-level + `reasoning_effort` parameter next to the `extra_body.thinking` config. Mapping is 1:1 + because the official schema documents server-side compatibility mapping (`low`/`medium` + map to `high`, `xhigh` to `max`): NONE keeps `thinking={"type": "disabled"}` and sends no + effort, LOW/MEDIUM/HIGH/XHIGH send `reasoning_effort` `low`/`medium`/`high`/`xhigh` with + `thinking={"type": "enabled", "clear_thinking": false}` (preserved thinking; the client + replays `reasoning_content` unmodified). `tool_choice` stays auto-only and temperature + passes through, as in GLM-5.1. Verified against live captures on the official endpoint + (including a thinking-disabled probe) plus OpenRouter and SiliconFlow; the gateway + `glm-5.2` ids now auto-route to the new client. +- Gemini image endpoint migration: `gemini-3.1-flash-image-preview` is deprecated and + replaced by `gemini-3.1-flash-image` (`gemini-3-pro-image-preview` likewise becomes + `gemini-3-pro-image`) across the registry, tests, and skill references; image generation + re-verified live on the new endpoint. +- New supported-model registry: `agenthub.list_supported_models(currency="USD"|"CNY")` / + `listSupportedModels(currency)` returns one entry per supported (model, platform) pair: + the `(model, base_url, client)` triple (mapping directly onto the `AutoLLMClient` + constructor) plus input/output modalities (Text/Image/Video/Audio/Embed), context window, + and per-million-token list pricing stored in the vendor's official currency and converted + at 7 CNY/USD on request. Covers official endpoints plus OpenRouter and SiliconFlow + (including `z-ai/glm-5.2`, `zai-org/GLM-5.2`, `moonshotai/kimi-k3`, and the OpenRouter + entries from the AgentHub app catalog: Claude/GPT/Gemini/MiniMax/Grok/Step/Hy3/MiMo/ + Nemotron via the generic OpenAI client, DeepSeek via its native client). OpenRouter + prices, context windows, and modality flags were pulled from the live `/models` API; + SiliconFlow prices from the vendors' official CNY price lists. Modalities record what is + usable through the routed AgentHub client, not the raw upstream capability, and are + written inline per entry. Pricing keys mirror AgentHub's usage buckets (`cached_tokens`, + `prompt_tokens`, `thoughts_tokens`, `response_tokens`; thoughts and response both carry + the vendor's output price), stored in USD and multiplied by 7 when CNY is requested; + CNY-denominated official list prices are declared with a `cny()` initializer that + converts at 7 CNY/USD on write. Gemini cache-hit prices from the official pricing page + (gemini-3.6-flash $0.15, gemini-3.5-flash-lite $0.03). `qwen/qwen3-embedding-4b` stays listed (embedding models are absent from + OpenRouter's chat-only `/models` API but the model page is live). Every newly added entry + passed a live streaming smoke call, except the official Anthropic entries, which could + not be smoke-tested locally (invalid local `ANTHROPIC_API_KEY`; their ids and prices were + verified via the OpenRouter passthrough listings). +- New `UnsupportedParameterError` (subclass of `AgentHubError`, hence still a `ValueError` in + Python) raised for unsupported `temperature`/`tool_choice`/`prompt_caching` values across + all clients; messages are unchanged, so existing `except ValueError` / message matching + keeps working. +- Docs snapshots: `llmsdk_docs/kimi_k3/` (raw pages from platform.kimi.com) and + `llmsdk_docs/gemini3_6/`. + +## Protocol differences found + +- **Kimi K3 vs K2.6**: reasoning is configured with the top-level `reasoning_effort` + parameter (`low`/`high`/`max`, default `max`) instead of `extra_body.thinking`, and cannot + be disabled. `tool_choice` gains `required`; forcing a specific function is incompatible + with the always-on reasoning. Context caching is fully automatic (no `prompt_cache_key`). + Everything else on the wire matches K2.6 (`reasoning_content` deltas, standard incremental + tool-call chunks, `completion_tokens_details.reasoning_tokens`); the K3 stream additionally + embeds a non-standard usage object inside the finishing choice, which the existing + top-level-usage accumulation already handles. +- **Gemini 3.6 generation vs Gemini 3**: identical wire format (thought signatures, usage + fields, event shapes verified by capture). The generation deprecates `temperature`/ + `top_p`/`top_k` — the API silently ignores them today (verified live) and will return + HTTP 400 in future generations — and disallows requests ending with a non-empty model + turn. The `gemini3_6` client therefore rejects `temperature` instead of sending a no-op. + +## Config mapping decisions (user-confirmed) + +- K3 `thinking_level` → `reasoning_effort`: NONE→`low` (cannot disable; degrade, do not + raise), LOW→`low`, MEDIUM→`high`, HIGH→`high`, XHIGH→`max`; unset → not sent (server + default `max`). +- K3 drops `trace_id` → `prompt_cache_key` (caching is automatic). +- Gemini 3.6 generation rejects `temperature` via `UnsupportedParameterError`; old + `gemini3/` client and its models are left untouched. + +## Test-matrix policy + +- `AVAILABLE_MODELS` now keeps only the newest version of each model family per provider + block: gemini-3.6-flash replaces gemini-3.5-flash and 3.5-flash-lite (Vertex included), + glm-5.2 replaces glm-5.1 (official, OpenRouter, SiliconFlow), and kimi-k3 replaces + kimi-k2.6 (official, OpenRouter; SiliconFlow keeps Kimi-K2.6 as its newest available + Kimi). Older clients stay supported and routed but are no longer e2e-tested. + +## Policy changes recorded in the dev skill + +- When old and new generations differ, keep the old model's client and routing and add a + new client folder; never remove old model support unless explicitly instructed. +- Every `ThinkingLevel` must stay usable on every client (map to the closest supported + level, never raise). `temperature`/`tool_choice` may reject values, but only with + `UnsupportedParameterError`. diff --git a/changelog/0.4.1/README.md b/changelog/0.4.1/README.md new file mode 100644 index 00000000..0483d42e --- /dev/null +++ b/changelog/0.4.1/README.md @@ -0,0 +1,3 @@ +# 0.4.1 + +- [2026-07-22] Kimi K3, Gemini 3.6 generation (gemini-3.6-flash, gemini-3.5-flash-lite), and GLM-5.2 support, a supported-model registry (`list_supported_models` with USD/CNY pricing, context windows, and modalities), and the `UnsupportedParameterError` parameter error class. ([details](2026-07-22-kimi-k3-gemini-3-6-registry.md)) diff --git a/changelog/README.md b/changelog/README.md index 19e37f94..bb6a8ea8 100644 --- a/changelog/README.md +++ b/changelog/README.md @@ -1,7 +1,7 @@ # Changelog Details -One file per entry in [../CHANGELOG.md](../CHANGELOG.md). The root file keeps a single brief line per change; the full story lives here. +The root [../CHANGELOG.md](../CHANGELOG.md) keeps one brief line per release. Each release owns a folder here: -- File name: `YYYY-MM-DD-short-slug.md`, matching the entry date. -- Content: what changed and why, affected modules, and decisions worth keeping (protocol differences, config mappings, migration notes). -- Link the root entry to its file: `- [YYYY-MM-DD] Brief description. ([details](changelog/YYYY-MM-DD-short-slug.md))` +- `/README.md` — the release summary: one brief line per change, each linking its detail file, e.g. `- [YYYY-MM-DD] Brief description. ([details](YYYY-MM-DD-short-slug.md))` +- `/YYYY-MM-DD-short-slug.md` — one detail file per entry, named by the entry date: what changed and why, affected modules, and decisions worth keeping (protocol differences, config mappings, migration notes). +- Changes not yet released go into the upcoming version's folder; during release preparation, rename the folder if the number changed and add the release line to the root file. diff --git a/llmsdk_docs/README.md b/llmsdk_docs/README.md index 20f33db7..66cc941f 100644 --- a/llmsdk_docs/README.md +++ b/llmsdk_docs/README.md @@ -11,8 +11,11 @@ To use a specific model, please refer to its dedicated README: - **[Claude 4.8](./claude4_8/README.md)** - Anthropic's Claude 4.8 API documentation and examples - **[DeepSeek V4](./deepseek_v4/README.md)** - DeepSeek V4 API documentation and OpenAI-compatible usage guides - **[Gemini 3](./gemini3/README.md)** - Google's Gemini 3 API documentation and examples +- **[Gemini 3.6](./gemini3_6/README.md)** - Google's Gemini 3.6 generation (gemini-3.6-flash, gemini-3.5-flash-lite): sampling-parameter deprecation and thinking levels - **[GLM-5.1](./glm5_1/README.md)** - Z.AI's GLM-5.1 API documentation and examples +- **[GLM-5.2](./glm5_2/README.md)** - Z.AI's GLM-5.2 API documentation (reasoning_effort, thinking modes, tool streaming) - **[GPT-5.5](./gpt5_5/README.md)** - OpenAI's GPT-5.5 API documentation and examples +- **[Kimi K3](./kimi_k3/README.md)** - Moonshot's Kimi K3 API documentation (reasoning_effort, tool calling, vision, caching) Each model directory contains: - `docs/` - Detailed documentation for the model's features and capabilities diff --git a/llmsdk_docs/gemini3_6/README.md b/llmsdk_docs/gemini3_6/README.md new file mode 100644 index 00000000..815b8d20 --- /dev/null +++ b/llmsdk_docs/gemini3_6/README.md @@ -0,0 +1,26 @@ +# Gemini 3.6 SDK Documentation + +This directory documents the Gemini 3.6 protocol generation, which starts with +`gemini-3.6-flash` and `gemini-3.5-flash-lite` and applies to all future Gemini model +releases. Content is snapshotted from the official documentation +(https://ai.google.dev/gemini-api/docs/latest-model). + +The request/response wire format is shared with the Gemini 3 generation (see +[../gemini3/](../gemini3/README.md)); this generation changes the parameter contract: + +- **Sampling parameters are deprecated**: `temperature`, `top_p`, and `top_k` are ignored by + the API for these models and will return an HTTP 400 error in future model generations. +- **Model turn prefill is disallowed**: API requests ending with a non-empty `model` role turn + return an HTTP 400 error. +- Thinking is configured with the `thinking_level` enum (no `thinking_budget`). + +## Documentation + +- [latest-model.md](./docs/latest-model.md) - The API changes introduced by this generation +- [gemini-3.6-flash.md](./docs/gemini-3.6-flash.md) - Gemini 3.6 Flash model spec +- [gemini-3.5-flash-lite.md](./docs/gemini-3.5-flash-lite.md) - Gemini 3.5 Flash-Lite model spec +- [thinking.md](./docs/thinking.md) - Thinking levels across Gemini 3.x models + +For the SDK usage guides (function calling, streaming, thought signatures, TTS, image +generation, embeddings), refer to [../gemini3/docs/](../gemini3/README.md); they apply +unchanged to this generation. diff --git a/llmsdk_docs/gemini3_6/docs/gemini-3.5-flash-lite.md b/llmsdk_docs/gemini3_6/docs/gemini-3.5-flash-lite.md new file mode 100644 index 00000000..e3942251 --- /dev/null +++ b/llmsdk_docs/gemini3_6/docs/gemini-3.5-flash-lite.md @@ -0,0 +1,18 @@ +# Gemini 3.5 Flash-Lite + +> Source: https://ai.google.dev/gemini-api/docs/models/gemini-3.5-flash-lite (snapshot 2026-07-22) + +- **Model code:** `gemini-3.5-flash-lite` (stable) +- **Input token limit:** 1,048,576 +- **Output token limit:** 65,536 +- **Input modalities:** Text, Image, Video, Audio, and PDF +- **Output modality:** Text +- **Thinking:** Supported (default `minimal`; supports `minimal`, `low`, `medium`, `high`) +- **Function calling:** Supported +- **Structured outputs:** Supported +- **Caching:** Supported +- **Batch API / Flex inference / Priority inference:** Supported +- **Not supported:** Live API, image generation, audio generation, computer use +- **Sampling parameters:** `temperature`/`top_p`/`top_k` are deprecated and ignored + (HTTP 400 in future generations) — this model belongs to the Gemini 3.6 protocol + generation despite its 3.5 name; see [latest-model.md](./latest-model.md) diff --git a/llmsdk_docs/gemini3_6/docs/gemini-3.6-flash.md b/llmsdk_docs/gemini3_6/docs/gemini-3.6-flash.md new file mode 100644 index 00000000..135a3b07 --- /dev/null +++ b/llmsdk_docs/gemini3_6/docs/gemini-3.6-flash.md @@ -0,0 +1,15 @@ +# Gemini 3.6 Flash + +> Source: https://ai.google.dev/gemini-api/docs/models/gemini-3.6-flash (snapshot 2026-07-22) + +- **Model code:** `gemini-3.6-flash` (stable) +- **Input token limit:** 1,048,576 +- **Output token limit:** 65,536 +- **Input modalities:** Text, Image, Video, Audio, and PDF +- **Output modality:** Text +- **Thinking:** Supported (default `medium`; supports `minimal`, `low`, `medium`, `high`) +- **Function calling:** Supported +- **Structured outputs:** Supported +- **Search grounding:** Supported +- **Sampling parameters:** `temperature`/`top_p`/`top_k` are deprecated and ignored + (HTTP 400 in future generations) — see [latest-model.md](./latest-model.md) diff --git a/llmsdk_docs/gemini3_6/docs/latest-model.md b/llmsdk_docs/gemini3_6/docs/latest-model.md new file mode 100644 index 00000000..58fec135 --- /dev/null +++ b/llmsdk_docs/gemini3_6/docs/latest-model.md @@ -0,0 +1,45 @@ +# Latest Gemini models + +> Source: https://ai.google.dev/gemini-api/docs/latest-model (snapshot 2026-07-22) + +## Gemini 3.6 Flash + +- **Model ID:** `gemini-3.6-flash` (stable) +- **Pricing:** $1.50 / 1M input tokens, $7.50 / 1M output tokens +- **Context window:** 1,048,576 input tokens; 65,536 output tokens +- **Capabilities:** thinking (default `medium` level), tool calling, structured outputs, + multimodal input (text, image, video, audio, PDF), Computer Use, code generation, + spatial reasoning +- Balances speed with intelligence for agentic and multimodal tasks. + +## Gemini 3.5 Flash-Lite + +- **Model ID:** `gemini-3.5-flash-lite` (stable) +- **Pricing:** $0.30 / 1M input tokens, $2.50 / 1M output tokens +- **Context window:** 1,048,576 input tokens; 65,536 output tokens +- **Capabilities:** thinking (default `minimal` level), tool calling, structured outputs, + multimodal input, Computer Use, data extraction, structured JSON parsing +- The fastest, lowest-cost 3.5 model for high-throughput execution. + +## API changes + +> "Starting with Gemini 3.6 Flash and Gemini 3.5 Flash-Lite, the following API changes +> apply to these models and all future Gemini model releases." + +1. **Sampling parameter deprecation.** `temperature`, `top_p`, and `top_k` are deprecated. + The API currently ignores these parameters; in future model generations, supplying them + returns an HTTP 400 error. +2. **Prefilled model turn validation.** API requests ending with a non-empty `model` role + turn are disallowed and return an HTTP 400 error. +3. **Thinking configuration.** Use the `thinking_level` enum instead of `thinking_budget`. + See [thinking.md](./thinking.md) for supported levels per model. + +## Live capture notes (api_captures/gemini3_6/) + +- Both models stream the same event shapes as the Gemini 3 generation: parts with optional + `thought_signature` on function-call and final text parts, `usage_metadata` with + `thoughts_token_count`, finish reason `STOP`. +- Thought summaries may be absent for simple requests; a signature can arrive on a + non-thought part and must be replayed as received. +- A `temperature=0.5` probe against both models succeeded (silently ignored), matching the + documented "ignored today, error later" behavior. diff --git a/llmsdk_docs/gemini3_6/docs/thinking.md b/llmsdk_docs/gemini3_6/docs/thinking.md new file mode 100644 index 00000000..5f31f077 --- /dev/null +++ b/llmsdk_docs/gemini3_6/docs/thinking.md @@ -0,0 +1,26 @@ +# Thinking levels across Gemini 3.x + +> Source: https://ai.google.dev/gemini-api/docs/thinking (snapshot 2026-07-22) + +Thinking is controlled with the `thinking_level` enum in the generation config. Supported +values and defaults per model: + +| Model | Default thinking | Supported levels | +| --------------------------- | ---------------- | ---------------------------- | +| gemini-3.6-flash | medium | minimal, low, medium, high | +| gemini-3.5-flash | medium | minimal, low, medium, high | +| gemini-3.5-flash-lite | minimal | minimal, low, medium, high | +| gemini-3.1-pro-preview | high | low, medium, high | +| gemini-3.1-flash-lite-image | minimal | minimal, high | +| gemini-3-flash-preview | high | minimal, low, medium, high | +| gemini-3-pro-preview | high | low, high | + +Notes: + +- Thought summaries are requested via the thinking config (`include_thoughts` in the + google-genai SDK). A thought block may contain **only a signature with no summary** for + simple requests or thought content types without text summaries. +- When managing conversation state yourself (stateless mode), you **must** resend all + thought blocks and thought signatures exactly as they were received to maintain + reasoning continuity. See [../../gemini3/docs/thought-signatures.md](../../gemini3/docs/thought-signatures.md). +- Thought tokens are reported in usage metadata (`thoughts_token_count` in the SDK). diff --git a/llmsdk_docs/glm5_2/README.md b/llmsdk_docs/glm5_2/README.md new file mode 100644 index 00000000..4a3d83f0 --- /dev/null +++ b/llmsdk_docs/glm5_2/README.md @@ -0,0 +1,28 @@ +# GLM-5.2 SDK Documentation + +This directory contains documentation for Z.AI's GLM-5.2 API, snapshotted as raw markdown +from the official documentation (https://docs.z.ai/). + +## Documentation + +- [glm-5.2.md](./docs/glm-5.2.md) - GLM-5.2 model guide (1M context, 128K max output) +- [migrate-to-glm-new.md](./docs/migrate-to-glm-new.md) - Migration guide from GLM-5.1/5/4.x +- [concept-param.md](./docs/concept-param.md) - Core request parameters +- [thinking.md](./docs/thinking.md) - Deep thinking capability +- [thinking-mode.md](./docs/thinking-mode.md) - Thinking modes (interleaved, preserved, turn-level) +- [stream-tool.md](./docs/stream-tool.md) - Streaming tool-call parameters (`tool_stream`) +- [chat-completion.md](./docs/chat-completion.md) - Chat Completion API reference (OpenAPI schema) + +## Key protocol differences vs GLM-5.1 + +- New top-level `reasoning_effort` parameter (GLM-5.2 only, default `max`; effective when + thinking is enabled). The server maps compatibility values itself: `none`/`minimal` skip + thinking, `low`/`medium` map to `high`, and `xhigh` maps to `max`. +- `thinking` stays `{"type": "enabled"|"disabled"}` (default enabled); when enabled the + model decides for itself whether to think. `"clear_thinking": false` still turns on + preserved thinking on the standard endpoint and requires replaying `reasoning_content` + unmodified. +- Maximum context grows to 1M tokens and maximum output to 128K tokens. +- `tool_choice` still supports only `auto`; `tool_stream: true` still enables streaming + tool-call arguments; streaming fields (`reasoning_content`/`content`/`tool_calls`) are + unchanged. diff --git a/llmsdk_docs/glm5_2/docs/chat-completion.md b/llmsdk_docs/glm5_2/docs/chat-completion.md new file mode 100644 index 00000000..09a75ee4 --- /dev/null +++ b/llmsdk_docs/glm5_2/docs/chat-completion.md @@ -0,0 +1,1161 @@ +> ## Documentation Index +> Fetch the complete documentation index at: https://docs.z.ai/llms.txt +> Use this file to discover all available pages before exploring further. + +# Chat Completion + +> Create a chat completion model that generates AI replies for given conversation messages. It supports multimodal inputs (text, images, audio, video, file), offers configurable parameters (like temperature, max tokens, tool use), and supports both streaming and non-streaming output modes. + + + +## OpenAPI + +````yaml POST /paas/v4/chat/completions +openapi: 3.0.1 +info: + title: Z.AI API + description: Z.AI API available endpoints + license: + name: Z.AI Developer Agreement and Policy + url: https://chat.z.ai/legal-agreement/terms-of-service + version: 1.0.0 + contact: + name: Z.AI Developers + url: https://chat.z.ai/legal-agreement/privacy-policy + email: user_feedback@z.ai +servers: + - url: https://api.z.ai/api + description: Production server +security: + - bearerAuth: [] +paths: + /paas/v4/chat/completions: + post: + description: >- + Create a chat completion model that generates AI replies for given + conversation messages. It supports multimodal inputs (text, images, + audio, video, file), offers configurable parameters (like temperature, + max tokens, tool use), and supports both streaming and non-streaming + output modes. + parameters: + - $ref: '#/components/parameters/AcceptLanguage' + requestBody: + content: + application/json: + schema: + oneOf: + - $ref: '#/components/schemas/ChatCompletionTextRequest' + title: Text Model + - $ref: '#/components/schemas/ChatCompletionVisionRequest' + title: Vision Model + examples: + Basic Example: + value: + model: glm-5.2 + messages: + - role: system + content: You are a helpful coding assistant. + - role: user + content: >- + Write a Python function to calculate the factorial of a + number. + temperature: 1 + stream: false + Stream Example: + value: + model: glm-5.2 + messages: + - role: system + content: You are a helpful coding assistant. + - role: user + content: >- + Write a Python function to calculate the factorial of a + number. + temperature: 1 + stream: true + Thinking Example: + value: + model: glm-5.2 + messages: + - role: system + content: You are a helpful coding assistant. + - role: user + content: >- + Write a Python function to calculate the factorial of a + number. + thinking: + type: enabled + stream: true + Multi Conversation: + value: + model: glm-5.2 + messages: + - role: system + content: You are a professional programming assistant. + - role: user + content: What is recursion? + - role: assistant + content: >- + Recursion is a programming technique where a function + calls itself to solve a problem... What is recursion + - role: user + content: Can you give me an example of Python recursion? + stream: true + Image Visual Example: + value: + model: glm-5v-turbo + messages: + - role: user + content: + - type: image_url + image_url: + url: https://cdn.bigmodel.cn/static/logo/register.png + - type: image_url + image_url: + url: https://cdn.bigmodel.cn/static/logo/api-key.png + - type: text + text: What are the pics talk about? + Video Visual Example: + value: + model: glm-5v-turbo + messages: + - role: user + content: + - type: video_url + video_url: + url: >- + https://cdn.bigmodel.cn/agent-demos/lark/113123.mov + - type: text + text: What are the video show about? + File Visual Example: + value: + model: glm-5v-turbo + messages: + - role: user + content: + - type: file_url + file_url: + url: https://cdn.bigmodel.cn/static/demo/demo2.txt + - type: file_url + file_url: + url: https://cdn.bigmodel.cn/static/demo/demo1.pdf + - type: text + text: What are the files show about? + Function Call Example: + value: + model: glm-5.2 + messages: + - role: user + content: >- + Is there an example of how the weather in Beijing is + today? + tools: + - type: function + function: + name: get_weather + description: Get weather information for the specified city. + parameters: + type: object + properties: + city: + type: string + description: City Name + required: + - city + tool_choice: auto + temperature: 0.3 + required: true + responses: + '200': + description: Processing successful + content: + application/json: + schema: + $ref: '#/components/schemas/ChatCompletionResponse' + default: + description: The request has failed. + content: + application/json: + schema: + $ref: '#/components/schemas/Error' +components: + parameters: + AcceptLanguage: + name: Accept-Language + in: header + schema: + type: string + description: Config desired response language for HTTP requests. + default: en-US,en + example: en-US,en + enum: + - en-US,en + required: false + schemas: + ChatCompletionTextRequest: + required: + - model + - messages + type: object + properties: + model: + type: string + description: >- + The model code to be called. GLM-5.2, GLM-5.1, GLM-5-Turbo are the + latest flagship model series, foundational models specifically + designed for agent applications. + example: glm-5.2 + default: glm-5.2 + enum: + - glm-5.2 + - glm-5.1 + - glm-5-turbo + - glm-5 + - glm-4.7 + - glm-4.7-flash + - glm-4.7-flashx + - glm-4.6 + - glm-4.5 + - glm-4.5-air + - glm-4.5-x + - glm-4.5-airx + - glm-4.5-flash + - glm-4-32b-0414-128k + messages: + type: array + description: >- + The current conversation message list as the model’s prompt input, + provided in JSON array format, e.g.,`{“role”: “user”, “content”: + “Hello”}`. Possible message types include system messages, user + messages, assistant messages, and tool messages. Note: The input + must not consist of system messages or assistant messages only. + items: + oneOf: + - title: User Message + type: object + properties: + role: + type: string + enum: + - user + description: Role of the message author + default: user + content: + oneOf: + - type: string + description: Text message content + example: >- + What opportunities and challenges will the Chinese + large model industry face in 2025? + required: + - role + - content + - title: System Message + type: object + properties: + role: + type: string + enum: + - system + description: Role of the message author + default: system + content: + oneOf: + - type: string + description: Message text content + example: You are a helpful assistant. + required: + - role + - content + - title: Assistant Message + type: object + description: Can include tool calls + properties: + role: + type: string + enum: + - assistant + description: Role of the message author + default: assistant + content: + oneOf: + - type: string + description: Text message content + example: I'll help you with that analysis. + tool_calls: + type: array + description: >- + Tool call messages generated by the model. When this field + is provided, content is usually empty. + items: + type: object + properties: + id: + type: string + description: Tool call ID + type: + type: string + description: Tool type, supports web_search, retrieval, function + enum: + - function + - web_search + - retrieval + function: + type: object + description: >- + Function call information, not empty when type is + function + properties: + name: + type: string + description: Function name + arguments: + type: string + description: Function parameters, JSON format string + required: + - name + - arguments + required: + - id + - type + required: + - role + - title: Tool Message + type: object + properties: + role: + type: string + enum: + - tool + description: Role of the message author + default: tool + content: + oneOf: + - type: string + description: Message text content + example: 'Function executed successfully with result: ...' + tool_call_id: + type: string + description: Indicates the tool call ID corresponding to this message + required: + - role + - content + - tool_call_id + minItems: 1 + do_sample: + type: boolean + example: true + default: true + description: >- + When do_sample is true, sampling strategy is enabled; when do_sample + is false, sampling strategy parameters such as temperature and top_p + will not take effect. Default value is `true`. + stream: + type: boolean + example: false + default: false + description: >- + This parameter should be set to false or omitted when using + synchronous call. It indicates that the model returns all content at + once after generating all content. Default value is false. If set to + true, the model will return the generated content in chunks via + standard Event Stream. When the Event Stream ends, a `data: [DONE]` + message will be returned. + thinking: + $ref: '#/components/schemas/ChatThinking' + reasoning_effort: + type: string + description: >- + Controls the model's reasoning effort level, takes effect when + `thinking` is enabled. Default is `max`. Only supported by + `GLM-5.2`. For compatibility with other protocols, passing `none` or + `minimal` will cause the model to skip thinking; `low` and `medium` + will be mapped to `high`; `xhigh` will be mapped to `max`. + example: max + default: max + enum: + - max + - xhigh + - high + - medium + - low + - minimal + - none + temperature: + type: number + description: >- + Sampling temperature, controls the randomness of the output, must be + a positive number within the range: `[0.0, 1.0]`. The GLM-5.2, + GLM-5.1, GLM-5, GLM-4.7, GLM-4.6 series default value is `1.0`, + GLM-4.5 series default value is `0.6`, GLM-4-32B-0414-128K default + value is `0.75`. + format: float + example: 1 + default: 1 + minimum: 0 + maximum: 1 + top_p: + type: number + description: >- + Another method of temperature sampling, value range is: `[0.01, + 1.0]`. The GLM-5.2, GLM-5.1, GLM-5, GLM-4.7, GLM-4.6, GLM-4.5 series + default value is `0.95`, GLM-4-32B-0414-128K default value is `0.9`. + format: float + example: 0.95 + default: 0.95 + minimum: 0.01 + maximum: 1 + max_tokens: + type: integer + description: >- + The maximum number of tokens for model output, the GLM-5.2, GLM-5.1, + GLM-5, GLM-4.7, GLM-4.6 series supports 128K maximum output, the + GLM-4.5 series supports 96K maximum output, the GLM-4.6v series + supports 32K maximum output, the GLM-4.5v series supports 16K + maximum output, GLM-4-32B-0414-128K supports 16K maximum output. + example: 1024 + minimum: 1 + maximum: 131072 + tool_stream: + type: boolean + example: false + default: false + description: >- + Whether to enable streaming response for Function Calls. Default + value is false. Only supported by GLM-4.6 and above. Refer the + [Stream Tool Call](/guides/tools/stream-tool) + tools: + type: array + description: > + A list of tools the model may call. Currently, only functions are + supported as a tool. Use this to provide a list of functions the + model may generate JSON inputs for. A max of 128 functions are + supported. + items: + anyOf: + - $ref: '#/components/schemas/FunctionToolSchema' + - $ref: '#/components/schemas/RetrievalToolSchema' + - $ref: '#/components/schemas/WebSearchToolSchema' + tool_choice: + oneOf: + - type: string + enum: + - auto + description: >- + Used to control how the model selects which function to call. + This is only applicable when the tool type is function. The + default value is auto, and only auto is supported. + description: Controls how the model selects a tool. + stop: + type: array + description: >- + Stop word list. Generation stops when the model encounters any + specified string. Currently, only one stop word is supported, in the + format ["stop_word1"]. + items: + type: string + maxItems: 4 + response_format: + type: object + description: >- + Specifies the response format of the model. Defaults to text. + Supports two formats:{ "type": "text" } plain text mode, returns + natural language text, { "type": "json_object" } JSON mode, returns + valid JSON data. When using JSON mode, it’s recommended to clearly + request JSON output in the prompt. + properties: + type: + type: string + enum: + - text + - json_object + default: text + description: >- + Output format type: text for plain text, json_object for + JSON-formatted output. + required: + - type + request_id: + type: string + description: >- + Passed by the user side, needs to be unique; used to distinguish + each request, 6–64 characters. If not provided by the user side, the + platform will generate one by default. + minLength: 6 + maxLength: 64 + user_id: + type: string + description: >- + Unique ID for the end user, 6–128 characters. Avoid using sensitive + information. + minLength: 6 + maxLength: 128 + ChatCompletionVisionRequest: + required: + - model + - messages + type: object + properties: + model: + type: string + description: >- + The model code to be called. GLM-5V-Turbo are the new generation of + visual reasoning models. `AutoGLM-Phone-Multilingual` is mobile + intelligent assistant model. + example: glm-5v-turbo + default: glm-5v-turbo + enum: + - glm-5v-turbo + - glm-4.6v + - autoglm-phone-multilingual + - glm-4.6v-flash + - glm-4.6v-flashx + - glm-4.5v + messages: + type: array + description: >- + The current conversation message list as the model’s prompt input, + provided in JSON array format, e.g.,`{“role”: “user”, “content”: + “Hello”}`. Possible message types include system messages, user + messages. Note: The input must not consist of system or assistant + messages only. + items: + oneOf: + - title: User Message + type: object + properties: + role: + type: string + enum: + - user + description: Role of the message author + default: user + content: + oneOf: + - type: array + description: >- + Multimodal message content, supports text, images, + video, file + items: + $ref: '#/components/schemas/VisionMultimodalContentItem' + - type: string + description: >- + Text message content (can switch to multimodal message + above) + example: >- + What opportunities and challenges will the Chinese + large model industry face in 2025? + required: + - role + - content + - title: System Message + type: object + properties: + role: + type: string + enum: + - system + description: Role of the message author + default: system + content: + oneOf: + - type: string + description: Message text content + example: You are a helpful assistant. + required: + - role + - content + - title: Assistant Message + type: object + description: Can include tool calls + properties: + role: + type: string + enum: + - assistant + description: Role of the message author + default: assistant + content: + oneOf: + - type: string + description: Text message content + example: I'll help you with that analysis. + required: + - role + minItems: 1 + do_sample: + type: boolean + example: true + default: true + description: >- + When do_sample is true, sampling strategy is enabled; when do_sample + is false, sampling strategy parameters such as temperature and top_p + will not take effect. Default value is `true`. + stream: + type: boolean + example: false + default: false + description: >- + This parameter should be set to false or omitted when using + synchronous call. It indicates that the model returns all content at + once after generating all content. Default value is false. If set to + true, the model will return the generated content in chunks via + standard Event Stream. When the Event Stream ends, a `data: [DONE]` + message will be returned. + thinking: + $ref: '#/components/schemas/ChatThinking' + temperature: + type: number + description: >- + Sampling temperature, controls the randomness of the output, must be + a positive number within the range: `[0.0, 1.0]`. The GLM-5V-Turbo, + GLM-4.6V, GLM-4.5V series default value is `0.8`, the + autoglm-phone-multilingual default value is `0.0`. + format: float + example: 0.8 + default: 0.8 + minimum: 0 + maximum: 1 + top_p: + type: number + description: >- + Another method of temperature sampling, value range is: `[0.01, + 1.0]`, value range is: `[0.01, 1.0]`. The GLM-5V-Turbo, GLM-4.6V, + GLM-4.5V series default value is `0.6`, the + autoglm-phone-multilingual default value is `0.85`. + format: float + example: 0.6 + default: 0.6 + minimum: 0.01 + maximum: 1 + max_tokens: + type: integer + description: >- + The maximum number of tokens for model output, the GLM-5V-Turbo + supports 128K maximum output, GLM-4.6V series supports 32K maximum + output, the GLM-4.5V series supports 16K maximum output, the + autoglm-phone-multilingual supports 4K maximum output. + example: 1024 + minimum: 1 + maximum: 131072 + tools: + type: array + description: > + A list of tools the model may call. Only support by GLM-4.6V series + and autoglm-phone-multilingual. Use this to provide a list of + functions the model may generate JSON inputs for. A max of 128 + functions are supported. + items: + anyOf: + - $ref: '#/components/schemas/FunctionToolSchema' + tool_choice: + oneOf: + - type: string + enum: + - auto + description: >- + Used to control how the model selects which function to call. + This is only applicable when the tool type is function. The + default value is auto, and only auto is supported. + description: Controls how the model selects a tool. + stop: + type: array + description: >- + Stop word list. Generation stops when the model encounters any + specified string. Currently, only one stop word is supported, in the + format ["stop_word1"]. + items: + type: string + maxItems: 4 + request_id: + type: string + description: >- + Passed by the user side, needs to be unique; used to distinguish + each request, 6–64 characters. If not provided by the user side, the + platform will generate one by default. + minLength: 6 + maxLength: 64 + user_id: + type: string + description: >- + Unique ID for the end user, 6–128 characters. Avoid using sensitive + information. + minLength: 6 + maxLength: 128 + ChatCompletionResponse: + type: object + properties: + id: + type: string + description: Task ID + request_id: + description: Request ID + type: string + created: + description: Request creation time, Unix timestamp in seconds + type: integer + model: + description: Model name + type: string + choices: + type: array + description: List of model responses + items: + type: object + properties: + index: + type: integer + description: Result index. + message: + $ref: '#/components/schemas/ChatCompletionResponseMessage' + finish_reason: + type: string + description: >- + Reason for model inference termination. Can be `stop`, + `tool_calls`, `length`, `sensitive`, + `model_context_window_exceeded` or `network_error`. + usage: + type: object + description: Token usage statistics returned when the model call ends. + properties: + prompt_tokens: + type: number + description: Number of tokens in user input + completion_tokens: + type: number + description: Number of output tokens + prompt_tokens_details: + type: object + properties: + cached_tokens: + type: number + description: Number of tokens served from cache + total_tokens: + type: integer + description: Total number of tokens + web_search: + description: Search results. + type: array + items: + $ref: '#/components/schemas/WebSearchObjectResponse' + Error: + required: + - code + - message + type: object + description: The request has failed. + properties: + code: + type: integer + format: int32 + description: Error code. + message: + type: string + description: Error message. + ChatThinking: + type: object + description: >- + Only supported by GLM-4.5 series and higher models. This parameter is + used to control whether the model enable the chain of thought. + properties: + type: + type: string + description: >- + Whether to enable the chain of thought(When enabled, GLM-5.2 GLM-5.1 + GLM-5 GLM-5-Turbo GLM-5V-Turbo GLM-4.6 GLM-4.5 and others will + automatically determine whether to think, while GLM-4.7 and GLM-4.5V + will think compulsorily), default: enabled + default: enabled + enum: + - enabled + - disabled + clear_thinking: + type: boolean + description: >- + Default value is True. Controls whether to clear `reasoning_content` + from previous conversation turns. View more in [Thinking + Mode](/guides/capabilities/thinking-mode). + - `true` (default): For this request, the system ignores/removes `reasoning_content` from prior turns, and only keeps non-reasoning context (e.g., user/assistant visible text, tool calls, and tool results). This is recommended for general chat or lightweight tasks to reduce context length and cost. + - `false`: Retains `reasoning_content` from prior turns and includes it in the context sent to the model. To enable Preserved Thinking, you must forward the full, unmodified, and correctly ordered historical `reasoning_content` in `messages`. Missing, truncated, rewritten, or reordered blocks may degrade performance or prevent the feature from taking effect. + - Notes: This parameter only affects cross-turn historical thinking blocks; it does not change whether the model generates/returns thinking in the current turn. + default: true + example: true + FunctionToolSchema: + type: object + title: Function Call + properties: + type: + type: string + default: function + enum: + - function + function: + $ref: '#/components/schemas/FunctionObject' + required: + - type + - function + additionalProperties: false + RetrievalToolSchema: + type: object + title: Retrieval + properties: + type: + type: string + default: retrieval + enum: + - retrieval + retrieval: + $ref: '#/components/schemas/RetrievalObject' + required: + - type + - retrieval + additionalProperties: false + WebSearchToolSchema: + type: object + title: Web Search + properties: + type: + type: string + default: web_search + enum: + - web_search + web_search: + $ref: '#/components/schemas/WebSearchObject' + required: + - type + - web_search + additionalProperties: false + VisionMultimodalContentItem: + oneOf: + - title: Text + type: object + properties: + type: + type: string + enum: + - text + description: Content type is text + default: text + text: + type: string + description: Text content + required: + - type + - text + additionalProperties: false + - title: Image + type: object + properties: + type: + type: string + enum: + - image_url + description: Content type is image URL + default: image_url + image_url: + type: object + description: Image information + properties: + url: + type: string + description: >- + Image URL or Base64 encoding. Image size limit is under 5M + per image, with pixels not exceeding 6000*6000. GLM-5V + GLM4.6V series are limited to 150 sheets, GLM4.5V limit 50 + sheets. Supports jpg, png, jpeg formats. + required: + - url + additionalProperties: false + required: + - type + - image_url + additionalProperties: false + - title: Video + type: object + properties: + type: + type: string + enum: + - video_url + description: Content type is video URL + default: video_url + video_url: + type: object + description: Video information. + properties: + url: + type: string + description: >- + Video URL address.The video size is limited to within 200 + MB, GLM-5V GLM4.6V series are limited to 2 videos, GLM4.5V + limit 1 video, and the format supports `mp4`，`mkv`，`mov`. + required: + - url + additionalProperties: false + required: + - type + - video_url + additionalProperties: false + - title: File + type: object + properties: + type: + type: string + enum: + - file_url + description: >- + Content type is file URL, not support passing both the + `file_url` and `image_url` or `video_url` parameters at the same + time. + default: file_url + file_url: + type: object + description: File information. + properties: + url: + type: string + description: >- + File URL address. Only GLM-5V-Turbo, GLM-4.6V, GLM-4.5V + supported. Supports formats such as + pdf、txt、word、jsonl、xlsx、pptx, with a maximum of 50. + required: + - url + additionalProperties: false + required: + - type + - file_url + additionalProperties: false + ChatCompletionResponseMessage: + type: object + properties: + role: + type: string + description: Current conversation role, default is ‘assistant’ (model) + example: assistant + content: + type: string + description: >- + Current conversation content. Hits function is null, otherwise + returns model inference result. + + For the GLM-4.5V series models, the output may contain the reasoning + process tags ` ` or the text boundary tags + `<|begin_of_box|> <|end_of_box|>`. + reasoning_content: + type: string + description: Reasoning content, supports by GLM-4.5 series. + tool_calls: + type: array + description: >- + Function names and parameters generated by the model that should be + called. + items: + $ref: '#/components/schemas/ChatCompletionResponseMessageToolCall' + WebSearchObjectResponse: + type: object + properties: + title: + type: string + description: Title. + content: + type: string + description: Content summary. + link: + type: string + description: Result URL. + media: + type: string + description: Website name. + icon: + type: string + description: Website icon. + refer: + type: string + description: Index number. + publish_date: + type: string + description: Website publication date. + FunctionObject: + type: object + properties: + name: + type: string + description: >- + The name of the function to be called. Must be a-z, A-Z, 0-9, or + contain underscores and dashes, with a maximum length of 64. + minLength: 1 + maxLength: 64 + pattern: ^[a-zA-Z0-9_-]+$ + description: + type: string + description: >- + A description of what the function does, used by the model to choose + when and how to call the function. + parameters: + $ref: '#/components/schemas/FunctionParameters' + required: + - name + - description + - parameters + RetrievalObject: + type: object + properties: + knowledge_id: + type: string + description: Knowledge base ID, created or obtained from the platform + prompt_template: + type: string + description: >- + Prompt template for requesting the model, a custom request template + containing placeholders `{{ knowledge }}` and `{{ question }}`. + Default template: Search for the answer to the question + `{{question}}` in the document `{{ knowledge }}`. If an answer is + found, respond only using statements from the document; if no answer + is found, use your own knowledge to answer and inform the user that + the information is not from the document. Do not repeat the + question, start the answer directly. + required: + - knowledge_id + WebSearchObject: + type: object + properties: + enable: + type: boolean + description: |- + Whether to enable search functionality. + Default is `false`. Set to true to `enable`. + search_engine: + type: string + description: |- + Type of search engine. + Default is `search_pro_jina`. Supports: `search_pro_jina`. + enum: + - search_pro_jina + search_query: + type: string + description: Force trigger a search + count: + type: integer + description: | + Number of returned results + Range: `1-50`, max `50` results per search + Default is `10` + Supported engines: `search_pro_jina` + minimum: 1 + maximum: 50 + search_domain_filter: + type: string + description: >- + Limits search results to specified whitelisted domains. Whitelist: + input domains directly (e.g., www.example.com) + + Supported engines: `search_pro_jina` + search_recency_filter: + type: string + description: |- + Limits search to a specific time range. + Default is `noLimit` + Values: + `oneDay`, within a day + `oneWeek`, within a week + `oneMonth`, within a month + `oneYear`, within a year + `noLimit`, no limit (default) + Supported engines: `search_pro_jina` + enum: + - oneDay + - oneWeek + - oneMonth + - oneYear + - noLimit + content_size: + type: string + description: >- + Number of characters for webpage summaries. + + Default is `medium` + + `medium`: Balanced mode for most queries. 400-600 characters + + `high`: Maximizes context for comprehensive answers, 2500 + characters. + enum: + - medium + - high + result_sequence: + type: string + description: >- + Specifies whether search results are shown before or after model + response. Options: `before`, `after`. Default is `after` + enum: + - before + - after + search_result: + type: boolean + description: |- + Whether to return search results in the response. + Default is `false` + require_search: + type: boolean + description: |- + Whether to force model response based on search result. + Default is `false` + search_prompt: + type: string + description: >- + Prompt to customize how search results are processed. + + Default Prompt: + + `You are an intelligent Q&A expert with the ability to synthesize + information, recognize time, understand semantics, and clean + contradictory data. The current date is {{current_date}}. Use this + as the only time reference. Based on the following information, + provide a comprehensive and accurate answer to the user's + question.Only extract valuable content for the answer. Ensure the + answer is timely and authoritative. State the answer directly + without citing data sources or internal processes.` + required: + - search_engine + ChatCompletionResponseMessageToolCall: + type: object + properties: + function: + type: object + description: >- + Contains the function name and JSON format parameters generated by + the model. + properties: + name: + type: string + description: Model-generated function name. + arguments: + type: object + description: >- + JSON format of the function call parameters generated by the + model. Validate the parameters before calling the function. + required: + - name + - arguments + id: + type: string + description: Unique identifier for the hit function. + type: + type: string + description: Tool type called by the model, currently only supports ‘function’. + FunctionParameters: + type: object + description: >- + Parameters defined using JSON Schema. Must pass a JSON Schema object to + accurately define accepted parameters. Omit if no parameters are needed + when calling the function. + additionalProperties: true + securitySchemes: + bearerAuth: + type: http + scheme: bearer + description: >- + Use the following format for authentication: Bearer [](https://z.ai/manage-apikey/apikey-list) + +```` \ No newline at end of file diff --git a/llmsdk_docs/glm5_2/docs/concept-param.md b/llmsdk_docs/glm5_2/docs/concept-param.md new file mode 100644 index 00000000..fffc8ab1 --- /dev/null +++ b/llmsdk_docs/glm5_2/docs/concept-param.md @@ -0,0 +1,182 @@ +> ## Documentation Index +> Fetch the complete documentation index at: https://docs.z.ai/llms.txt +> Use this file to discover all available pages before exploring further. + +# Core Parameters + + + When interacting with models, you can control the model's output by adjusting different parameters to meet the needs of various scenarios. Understanding these core parameters will help you better utilize the model's capabilities. + + +## Quick Reference + +| Parameter | Type | Default Value | Description | +| :------------------------------------- | :------ | :-------------------- | :-------------------------------------------------------------------------------------------- | +| [do\_sample](#do_sample) | Boolean | `true` | Whether to sample the output to increase diversity. | +| [temperature](#temperature) | Float | (Model dependent) | Controls the randomness of output, higher values are more random. | +| [top\_p](#top_p) | Float | (Model dependent) | Controls diversity through nucleus sampling, recommended to use either this or `temperature`. | +| [max\_tokens](#max_tokens) | Integer | (Model dependent) | Limits the maximum number of tokens generated in a single call. | +| [stream](#stream) | Boolean | `false` | Whether to return responses in streaming mode. | +| [thinking](#thinking) | Object | `{"type": "enabled"}` | Whether to enable chain-of-thought deep thinking, only supported by `GLM-4.5` and above. | +| [reasoning\_effort](#reasoning_effort) | String | `max` | Controls the model's reasoning effort level, only supported by `GLM-5.2` and above. | + +*** + +## Parameter Details + +### do\_sample + +`do_sample` is a boolean value (`true` or `false`) that determines whether to sample the model's output. + +* `true` (default): Performs random sampling based on the probability distribution of each token, increasing text diversity and creativity. Suitable for content creation, dialogue, and other scenarios. +* `false`: Uses a greedy strategy, always selecting the token with the highest probability. Provides high deterministic output, suitable for scenarios requiring precise, factual answers. + +Best Practices: + +* Set to `false` when you need reproducible, deterministic output. +* Set to `true` when you want the model to generate more diverse and interesting content, and use it in combination with `temperature` or `top_p`. + +### temperature + +The `temperature` parameter controls the randomness of the model's output. + +* Lower values (e.g., 0.2): Make the probability distribution more "sharp", resulting in more deterministic and conservative output. +* Higher values (e.g., 0.8): Make the probability distribution more "flat", resulting in more random and diverse output. + +Best Practices: + +* For scenarios requiring rigor and factual accuracy (such as knowledge Q\&A), it's recommended to use lower `temperature`. +* For scenarios requiring creativity (such as content creation), you can try higher `temperature`. +* It's recommended to use only one of `temperature` and `top_p`. + +### top\_p + +`top_p` (nucleus sampling) controls diversity by sampling from the smallest set of tokens whose cumulative probability exceeds the threshold. + +* Lower values (e.g., 0.2): Limit the sampling range, resulting in more deterministic output. +* Higher values (e.g., 0.9): Expand the sampling range, resulting in more diverse output. + +Best Practices: + +* If you want to achieve some diversity while ensuring content quality, `top_p` is a good choice (recommended values 0.8-0.95). +* It's generally not recommended to modify both `temperature` and `top_p` simultaneously. + +### max\_tokens + +`max_tokens` is used to limit the maximum number of tokens the model can generate in a single call. GLM-4.6 supports a maximum output length of 128K, GLM-4.5 supports a maximum output length of 96K, and it's recommended to set it to no less than 1024. Tokens are the basic units of text, typically 1 token equals approximately 0.75 English words or 1.5 Chinese characters. Setting an appropriate max\_tokens can control response length and cost, avoiding overly long outputs. If the model completes its answer before reaching the max\_tokens limit, it will naturally end; if it reaches the limit, the output may be truncated. + +* Purpose: Prevents generating overly long text and controls API call costs. +* Note: `max_tokens` limits the length of generated content, not including input. + +Best Practices: + +* Set `max_tokens` reasonably according to your application scenario. If you need short answers, you can set it to a smaller value (e.g., 50). + +Default `max_tokens` and maximum supported `max_tokens` for each model: + +| Model Code | Default max\_tokens | Maximum max\_tokens | +| :------------------ | :-----------------: | :-----------------: | +| glm-5.2 | 65536 | 131072 | +| glm-5.1 | 65536 | 131072 | +| glm-5v-turbo | 65536 | 131072 | +| glm-5-turbo | 65536 | 131072 | +| glm-5 | 65536 | 131072 | +| glm-4.7 | 65536 | 131072 | +| glm-4.6 | 65536 | 131072 | +| glm-4.6v | 16384 | 32768 | +| glm-4.6v-flash | 16384 | 32768 | +| glm-4.6v-flashx | 16384 | 32768 | +| glm-4.5 | 65536 | 98304 | +| glm-4.5-air | 65536 | 98304 | +| glm-4.5-x | 65536 | 98304 | +| glm-4.5-airx | 65536 | 98304 | +| glm-4.5-flash | 65536 | 98304 | +| glm-4.5v | 16384 | 16384 | +| glm-4-32b-0414-128k | 16384 | 16384 | + +### stream + +`stream` is a boolean value used to control the API's response method. + +* `false` (default): Returns the complete response at once, simple to implement but with long waiting times. +* `true`: Returns content in streaming (SSE) mode, significantly improving the experience of real-time interactive applications. + +Best Practices: + +* For chatbots, real-time code generation, and other applications, it's strongly recommended to set this to `true`. + +### thinking + +The `thinking` parameter controls whether the model enables "Chain of Thought" for deeper thinking and reasoning. + +* Type: Object +* Supported Models: `GLM-4.5` and above + +Properties: + +* `type` (string): + * `enabled` (default): Enable chain of thought. `GLM-5.2`, `GLM-5.1`, `GLM-5`, `GLM-5-Turbo`, `GLM-5V-Turbo`, `GLM-4.6`, and `GLM-4.5` auto-determine whether to think, while `GLM-4.7` and `GLM-4.5V` use forced thinking. + * `disabled`: Disable chain of thought. + +Best Practices: + +* It's recommended to enable this when you need the model to perform complex reasoning and planning. +* For simple tasks, you can disable it to get faster responses. + +### reasoning\_effort + +The `reasoning_effort` parameter is used to control the model's reasoning effort level when chain-of-thought thinking is enabled. + +* Type: String +* Supported Models: `GLM-5.2` and above +* Allowed values: `max`, `xhigh`, `high`, `medium`, `low`, `minimal`, `none` +* Default: `max` + +Parameter values: + +* `max`: Deep reasoning (default) +* `high`: Enhanced reasoning +* `xhigh`, `medium`, `low`, `minimal`, `none`: Compatibility mappings to maintain compatibility with other protocols + +Notes: + +* For compatibility with other protocols, passing `none` or `minimal` will cause the model to skip thinking; `low` and `medium` will be mapped to `high`; `xhigh` will be mapped to `max`. +* This parameter only takes effect when `thinking.type` is set to `enabled`. + +*** + +## Related Concepts + + + + Tokens are the basic units for model text processing. Usage calculation includes both input and output parts. + + * **Input Token Count:** The number of tokens contained in the text you send to the model. + * **Output Token Count:** The number of tokens contained in the text generated by the model. + * **Total Token Count:** The sum of input and output, usually used as the billing basis. + + You can call the `tokenizer` API to estimate the token count of text. + + + + Maximum Output Tokens refers to the maximum number of tokens a model can generate in a single request. It's different from the `max_tokens` parameter - `max_tokens` is the upper limit you set in your request, while Maximum Output Tokens is the architectural limitation of the model itself. + + For example, a model's context window might be 8k tokens, but its maximum output capability might be limited to 4k tokens. + + + + The Context Window refers to the total number of tokens a model can process in a single interaction, including all tokens from both **input text** and **generated text**. + + * **Importance:** The context window determines how much historical information the model can "remember". If the total length of input and expected output exceeds the model's context window, the model will be unable to process it. + * **Note:** Different models have different context window sizes. When conducting long conversations or processing long documents, special attention should be paid to context window limitations. + + + + Concurrency refers to the number of API requests you can initiate simultaneously. This is set by the platform to ensure service stability and fair resource allocation. + + * **Limits:** Different users or subscription plans may have different concurrency quotas. + * **Overages:** If you exceed the concurrency limit, new requests may fail or need to wait in queue. + + If your application requires high concurrency processing, please check your account limits or contact platform support. + + diff --git a/llmsdk_docs/glm5_2/docs/glm-5.2.md b/llmsdk_docs/glm5_2/docs/glm-5.2.md new file mode 100644 index 00000000..9a28c92c --- /dev/null +++ b/llmsdk_docs/glm5_2/docs/glm-5.2.md @@ -0,0 +1,482 @@ +> ## Documentation Index +> Fetch the complete documentation index at: https://docs.z.ai/llms.txt +> Use this file to discover all available pages before exploring further. + +# GLM-5.2 + +## Overview + +**GLM-5.2** is a flagship model built for the era of long-horizon tasks. With truly usable 1M-token context, it has been tested to handle project-scale engineering context, delivering more stable long-task execution, more reliable adherence to engineering standards, and higher success rates in development scenarios. A single task can complete the full development workflow—from requirements to deployable products across multiple platforms. + + + + Flagship Foundation Model + + + + Text + + + + Text + + + + 1M + + + + 128K + + + +## Capability + + + + Offering multiple thinking modes for different scenarios + + + + Support real-time streaming responses to enhance user interaction experience + + + + Powerful tool invocation capabilities, enabling integration with various external toolsets + + + + Intelligent caching mechanism to optimize performance in long conversations + + + + Support for structured output formats like JSON, facilitating system integration + + + + Flexibly integrate external MCP tools and data sources to expand application scenarios + + + +## Usage + + + + This is the best starting point to experience the generational leap of GLM-5.2. It can continuously retain module boundaries, architectural constraints, API contracts, directory structures, and historical decisions, significantly reducing the sense of context fragmentation in the later stages of long-running tasks. For complex projects, the key experience is that the model does not merely read more context—it can carry forward the engineering judgments formed earlier into subsequent execution. + + **Recommended way to try it**: Choose a real business codebase, preferably one that includes backend, frontend or client-side code, configuration files, tests, documentation, and engineering conventions. First, ask the model to perform a technical audit: + + > Please read the current project and output a system architecture map, core module responsibilities, key API contracts, major data flows, core call chains, potential technical debt, and the engineering constraints that must be followed in future refactoring. + + + + GLM-5.2 is more stable in cross-file, multi-step, long-chain tasks. It first breaks down the goal, identifies dependencies and risks, then implements, verifies, and closes the task in stages. This makes it suitable for tasks that require continuous progress, such as module decoupling, API migration, directory restructuring, SDK adaptation, and cross-language refactoring. + + **Recommended way to try it**: Choose a medium-sized refactoring task, define clear boundaries, and enable `/goal` mode: + + > Please complete the decoupling and refactoring of the current module without changing the business logic, API signatures, or runtime behavior. First provide the execution plan, impact scope, risk boundaries, and verification method. After completion, run the necessary tests and output the verification results. + + + + GLM-5.2 shows stronger consistency in following engineering standards, especially in long-context and multi-round execution. It is better at adhering to code style, architectural boundaries, dependency constraints, build processes, testing requirements, and commit boundaries, reducing risks such as out-of-scope changes, invalid dependencies, skipped verification, or unauthorized commits. + + **Recommended way to try it**: Provide the model with your team’s real engineering standards, such as lint rules, build commands, testing requirements, commit conventions, and prohibited actions in `CLAUDE.md` or `Agent.md`. Then give it a real modification task: + + > Please strictly follow the engineering standards of the current repository. Do not introduce new dependencies, do not modify API contracts, and do not commit changes proactively. After completing the modification, run the build, lint, and tests, then report the verification results and any uncovered risks. + + + + In mobile development scenarios, GLM-5.2 can cover client-side architecture, streaming messages, long-connection states, local state management, keyboard behavior, scrolling logic, system notifications, permission mechanisms, and background recovery. More importantly, it can use ADB, logcat, screenshots, and runtime logs to locate real-device issues, making it closely aligned with practical mobile engineering workflows. + + **Recommended way to try it**: Choose a real Android or Mini Program task and let the model go from implementation to validation: + + > Please implement a native Android client in Kotlin that connects to the existing server-side API and supports multi-session conversations, streaming messages, voice input, notifications, and reconnection after disconnection. After completion, install it on a real device using ADB, and debug it with logcat and screenshots. + + + + GLM-5.2 can handle page subpackages, custom components, page-level components, page stack management, `wx.request` wrapping and API-layer adaptation, authentication and login state maintenance (`wx.login` + custom login state), app/page/component lifecycle management, and exception handling in Mini Program development. It is suitable for testing whether the model can reorganize an existing Web page, official website, or backend capability into a runnable project that complies with Mini Program platform requirements. + + **Recommended way to try it**: Choose an existing Web project, specify the target technology stack — native Mini Program, Taro, or uni-app — and migrate all Web features into a Mini Program version: + + > Please migrate all features of the current Web project into a WeChat Mini Program. Use the \[native/Taro/uni-app] technology stack. First analyze the page structure, core user paths, backend API contracts, and platform constraints, including package size limits, domain allowlists, and HTTPS requirements. Then complete the implementation of pages, components, page navigation, and data flows. After completion, explain how to run the project, which APIs have been integrated, which features remain uncovered, and what can be optimized next. + + + + GLM-5.2 is well suited for testing rule understanding, state machine design, level structure, scoring logic, resource loading, interaction feedback, and settlement flows in mini game development. Compared with static pages, this type of task better demonstrates the model’s understanding of complex states, user paths, and product completeness. + + **Recommended way to try it**: Provide a complete but not overly detailed gameplay goal, and let the model first design the rules, then implement a runnable version: + + > Please develop a lightweight level-based mini game. First design the core gameplay loop, state machine, level structure, scoring rules, failure and settlement logic, then implement basic features including start, pause, resume, settlement, restart, and local save. After completion, explain the project structure, verified features, and possible next-step extensions. + + + + GLM-5.2 can turn the model architecture, loss functions, data pipelines, and training/inference scripts described in a paper into runnable code that aligns with the paper. It can correctly set up the model structure in one pass, maintain consistency across multiple files, and autonomously run, debug, and fix code and environment issues. What it delivers is not just code snippets, but an engineering project that can truly reproduce the paper’s reported results. + + **Recommended way to try it**: Pick a paper with a model and experiments, preferably one with open-source code or public metrics, and provide the paper and data to the model. See whether it can implement the model, run it successfully, and align the results with the paper: + + > Please reproduce the experiments based on this paper and dataset. Fill in implementation details not explicitly described in the paper. Use PyTorch to build the model architecture and loss functions, construct the data pipeline and training/inference scripts, and ensure the project runs successfully with consistency across multiple files. Autonomously identify and fix runtime issues, verify the paper’s metrics item by item until they are aligned, and explain the reproduction path, key changes, and any remaining gaps. + + + + In Code-to-Video scenarios, GLM-5.2 can use the Remotion framework to create videos programmatically with React code, including components, parameters, and animation logic, and then render them into MP4. In simple terms, it treats video creation as writing code. It covers the full workflow from translating natural-language ideas into Remotion React code to rendering video output, enabling code-driven generation of a runnable, demo-ready video. + + **Recommended way to try it**: Choose a real video creation task and let the model start from a single natural-language idea, then gradually produce a renderable, playable, and iterable video: + + > Please create a new composition in Remotion and add a map. Start from Los Angeles, zoom the camera out while keeping LA in focus. Then draw an animated route from Los Angeles to New York and have the camera follow the route. Add one more stop to the journey — this time, we are going to Paris. + + + +## Introducing GLM-5.2 + + + + The foundation of long-horizon tasks is not having a 1M context, but making 1M context truly usable. GLM-5.2 delivers a Solid 1M lossless context and has undergone months of specialized training for long-horizon Coding Agent scenarios, covering high-value tasks such as large-scale implementation, automated research, and performance optimization. + + Compared to solutions that merely extend context length, GLM-5.2 maintains more stable performance at ultra-long context, even surpassing Opus in select real-world benchmarks. + + GLM-5.2 delivers state-of-the-art long-horizon coding performance among open-source models. Across FrontierSWE, PostTrainBench, and SWE-Marathon, it consistently ranks among the top models overall—trailing Opus 4.8 by just 1% on FrontierSWE, outperforming GPT-5.5 and Opus 4.7 on multiple benchmarks, and remaining the highest-ranked open-source model across all three. These results demonstrate that GLM-5.2's 1M context window translates into practical long-horizon engineering capability. + + ![Description](https://cdn.bigmodel.cn/markdown/17816319661261.png?attname=1.png) + + + + On standard coding benchmarks, GLM-5.2 is the strongest open-source model, improving on GLM-5.1 by a wide margin: 81.0 vs. 62.0 on Terminal-Bench 2.1 and 62.1 vs. 58.4 on SWE-bench Pro. It also closes much of the gap to the closed-source frontier — on Terminal-Bench 2.1 (81.0) it lands within a few points of Claude Opus 4.8 (85.0) — while staying ahead of Gemini 3.1 Pro. + ![Description](https://cdn.bigmodel.cn/markdown/1781632244480plan2.png?attname=plan2.png) + + Before its official release, GLM-5.2 was made available in advance to GLM Coding Plan users. Developers reported improvements mainly in the following areas: + + * Stronger project-level context capacity, enabling an entire codebase to be placed within a single reasoning workflow; + * More stable long-horizon task execution, allowing complex tasks to progress continuously without easily going off track; + * More reliable adherence to production-grade engineering standards, helping enforce hard constraints in team development workflows; + * Stronger client-side and mobile engineering capabilities, going beyond app generation to support a complete on-device debugging loop. + + [↗ Blog](https://z.ai/blog/glm-5.2) + + + +## Resources + +* [API Documentation](/api-reference/llm/chat-completion): Learn how to call the API. + +## Quick Start + +The following is a full sample code to help you onboard GLM-5.2 with ease. + + + + **Basic Call** + + ```bash theme={null} + curl -X POST "https://api.z.ai/api/paas/v4/chat/completions" \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer your-api-key" \ + -d '{ + "model": "glm-5.2", + "messages": [ + { + "role": "system", + "content": "You are a senior full-stack software engineer, proficient in frontend development, backend architecture design, and modern web technology stacks." + }, + { + "role": "user", + "content": "Design and build a personal blog website for me, including a homepage, article list page, and article detail page, using React + Node.js technology stack." + } + ], + "thinking": { + "type": "enabled" + }, + "reasoning_effort": "max", + "max_tokens": 4096, + "temperature": 1.0 + }' + ``` + + **Streaming Call** + + ```bash theme={null} + curl -X POST "https://api.z.ai/api/paas/v4/chat/completions" \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer your-api-key" \ + -d '{ + "model": "glm-5.2", + "messages": [ + { + "role": "system", + "content": "You are a senior full-stack software engineer, proficient in frontend development, backend architecture design, and modern web technology stacks." + }, + { + "role": "user", + "content": "Design and build a personal blog website for me, including a homepage, article list page, and article detail page, using React + Node.js technology stack." + } + ], + "thinking": { + "type": "enabled" + }, + "reasoning_effort": "max", + "stream": true, + "max_tokens": 4096, + "temperature": 1.0 + }' + ``` + + + + **Install SDK** + + ```bash theme={null} + # Install latest version + pip install zai-sdk + + # Or specify version + pip install zai-sdk==0.2.3 + ``` + + **Verify Installation** + + ```python theme={null} + import zai + + print(zai.__version__) + ``` + + **Basic Call** + + ```python theme={null} + from zai import ZaiClient + + client = ZaiClient(api_key="your-api-key") # Your API Key + + response = client.chat.completions.create( + model="glm-5.2", + messages=[ + { + "role": "system", + "content": "You are a senior full-stack software engineer, proficient in frontend development, backend architecture design, and modern web technology stacks.", + }, + { + "role": "user", + "content": "Design and build a personal blog website for me, including a homepage, article list page, and article detail page, using React + Node.js technology stack.", + }, + ], + thinking={ + "type": "enabled", + }, + reasoning_effort="max", + max_tokens=4096, + temperature=1.0, + ) + + # Get complete response + print(response.choices[0].message) + ``` + + **Streaming Call** + + ```python theme={null} + from zai import ZaiClient + + client = ZaiClient(api_key="your-api-key") # Your API Key + + response = client.chat.completions.create( + model="glm-5.2", + messages=[ + { + "role": "system", + "content": "You are a senior full-stack software engineer, proficient in frontend development, backend architecture design, and modern web technology stacks.", + }, + { + "role": "user", + "content": "Design and build a personal blog website for me, including a homepage, article list page, and article detail page, using React + Node.js technology stack.", + }, + ], + thinking={ + "type": "enabled", # Optional: "disabled" or "enabled", default is "enabled" + }, + reasoning_effort="max", + stream=True, + max_tokens=4096, + temperature=0.6, + ) + + # Stream response + for chunk in response: + if chunk.choices[0].delta.reasoning_content: + print(chunk.choices[0].delta.reasoning_content, end="", flush=True) + + if chunk.choices[0].delta.content: + print(chunk.choices[0].delta.content, end="", flush=True) + ``` + + + + **Install SDK** + + **Maven** + + ```xml theme={null} + + ai.z.openapi + zai-sdk + 0.3.5 + + ``` + + **Gradle (Groovy)** + + ```groovy theme={null} + implementation 'ai.z.openapi:zai-sdk:0.3.5' + ``` + + **Basic Call** + + ```java theme={null} + import ai.z.openapi.ZaiClient; + import ai.z.openapi.service.model.ChatCompletionCreateParams; + import ai.z.openapi.service.model.ChatCompletionResponse; + import ai.z.openapi.service.model.ChatMessage; + import ai.z.openapi.service.model.ChatMessageRole; + import ai.z.openapi.service.model.ChatThinking; + import java.util.Arrays; + + public class BasicChat { + public static void main(String[] args) { + // Initialize client + ZaiClient client = ZaiClient.builder().ofZAI().apiKey("your-api-key").build(); + + // Create chat completion request + ChatCompletionCreateParams request = ChatCompletionCreateParams.builder() + .model("glm-5.2") + .messages( + Arrays.asList( + ChatMessage.builder() + .role(ChatMessageRole.SYSTEM.value()) + .content( + "You are a senior full-stack software engineer, proficient in frontend development, backend architecture design, and modern web technology stacks.") + .build(), + ChatMessage.builder() + .role(ChatMessageRole.USER.value()) + .content( + "Design and build a personal blog website for me, including a homepage, article list page, and article detail page, using React + Node.js technology stack.") + .build())) + .thinking(ChatThinking.builder().type("enabled").build()) + .reasoningEffort("max") + .maxTokens(4096) + .temperature(1.0f) + .build(); + + // Send request + ChatCompletionResponse response = client.chat().createChatCompletion(request); + + // Get response + if (response.isSuccess()) { + Object reply = response.getData().getChoices().get(0).getMessage(); + System.out.println("AI Response: " + reply); + } else { + System.err.println("Error: " + response.getMsg()); + } + } + } + ``` + + **Streaming Call** + + ```java theme={null} + import ai.z.openapi.ZaiClient; + import ai.z.openapi.service.model.ChatCompletionCreateParams; + import ai.z.openapi.service.model.ChatCompletionResponse; + import ai.z.openapi.service.model.ChatMessage; + import ai.z.openapi.service.model.ChatMessageRole; + import ai.z.openapi.service.model.ChatThinking; + import ai.z.openapi.service.model.Delta; + import java.util.Arrays; + + public class StreamingChat { + public static void main(String[] args) { + // Initialize client + ZaiClient client = ZaiClient.builder().ofZAI().apiKey("your-api-key").build(); + + // Create streaming chat completion request + ChatCompletionCreateParams request = ChatCompletionCreateParams.builder() + .model("glm-5.2") + .messages( + Arrays.asList( + ChatMessage.builder() + .role(ChatMessageRole.SYSTEM.value()) + .content( + "You are a senior full-stack software engineer, proficient in frontend development, backend architecture design, and modern web technology stacks.") + .build(), + ChatMessage.builder() + .role(ChatMessageRole.USER.value()) + .content( + "Design and build a personal blog website for me, including a homepage, article list page, and article detail page, using React + Node.js technology stack.") + .build())) + .thinking(ChatThinking.builder().type("enabled").build()) + .reasoningEffort("max") + .stream(true) // Enable streaming output + .maxTokens(4096) + .temperature(1.0f) + .build(); + + ChatCompletionResponse response = client.chat().createChatCompletion(request); + + if (response.isSuccess()) { + response.getFlowable() + .subscribe( + // Process streaming message data + data -> { + if (data.getChoices() != null && !data.getChoices().isEmpty()) { + Delta delta = data.getChoices().get(0).getDelta(); + System.out.print(delta + "\n"); + } + }, + // Process streaming response error + error -> System.err.println("\nStream error: " + error.getMessage()), + // Process streaming response completion event + () -> System.out.println("\nStreaming response completed")); + } else { + System.err.println("Error: " + response.getMsg()); + } + } + } + ``` + + + + **Install SDK** + + ```bash theme={null} + # Install or upgrade to latest version + pip install --upgrade 'openai>=1.0' + ``` + + **Verify Installation** + + ```python theme={null} + python -c "import openai; print(openai.__version__)" + ``` + + **Usage Example** + + ```python theme={null} + from openai import OpenAI + + client = OpenAI( + api_key="your-Z.AI-api-key", + base_url="https://api.z.ai/api/paas/v4/", + ) + + completion = client.chat.completions.create( + model="glm-5.2", + messages=[ + {"role": "system", "content": "You are a senior full-stack software engineer, proficient in frontend development, backend architecture design, and modern web technology stacks."}, + { + "role": "user", + "content": "Design and build a personal blog website for me, including a homepage, article list page, and article detail page, using React + Node.js technology stack.", + }, + ], + ) + + print(completion.choices[0].message.content) + ``` + + diff --git a/llmsdk_docs/glm5_2/docs/migrate-to-glm-new.md b/llmsdk_docs/glm5_2/docs/migrate-to-glm-new.md new file mode 100644 index 00000000..466fc429 --- /dev/null +++ b/llmsdk_docs/glm5_2/docs/migrate-to-glm-new.md @@ -0,0 +1,196 @@ +> ## Documentation Index +> Fetch the complete documentation index at: https://docs.z.ai/llms.txt +> Use this file to discover all available pages before exploring further. + +# Migrate to GLM-5.2 + + + This guide explains how to migrate your calls from GLM-5.1, GLM-5, GLM-4.7, GLM-4.6, GLM-4.5 or other earlier models to Z.AI GLM-5.2, our strongest coding model to date, covering sampling parameter differences, streaming tool calls, reasoning\_effort parameter, and other key points. + + +## GLM-5.2 Features + +* Support for larger context and output: Maximum context 1M, maximum output 128K. +* New support for streaming output during tool calling process (`tool_stream=true`), real-time retrieval of tool call parameters. +* Supports deep thinking (`thinking={ type: "enabled" }`): when enabled, the model automatically determines whether to think (unlike GLM-4.7, which uses forced thinking). +* New `reasoning_effort` parameter for controlling the model's reasoning effort level when chain-of-thought thinking is enabled. +* Superior code performance and advanced reasoning capabilities. + +## Migration Checklist + +* [ ] Update model identifier to `glm-5.2` +* [ ] Sampling parameters: `temperature` default value `1.0`, `top_p` default value `0.95`, recommend choosing only one for tuning +* [ ] Deep thinking: Enable or disable `thinking={ type: "enabled" }` as needed for complex reasoning/coding +* [ ] Control reasoning effort: Configure `reasoning_effort` to decide between `high` (enhanced reasoning) or `max` (deep reasoning, default) +* [ ] Streaming response: Enable `stream=true` and properly handle `delta.reasoning_content` and `delta.content` +* [ ] Streaming tool calls: Enable `stream=true` and `tool_stream=true` and stream-concatenate `delta.tool_calls[*].function.arguments` +* [ ] Maximum output and context: Set `max_tokens` appropriately (GLM-5.2 maximum output 128K, context 1M) +* [ ] Prompt optimization: Work with deep thinking, use clearer instructions and constraints +* [ ] Development environment verification: Conduct use case testing and regression, focus on randomness, latency, parameter completeness in tool streams + +## Start Migration + +### 1. Update Model Identifier + +* Update `model` to `glm-5.2`. + +```python theme={null} +resp = client.chat.completions.create( + model="glm-5.2", + messages=[{"role": "user", "content": "Briefly describe the advantages of GLM-5.2"}] +) +``` + +### 2. Update Sampling Parameters + +* `temperature`: Controls randomness; higher values are more divergent, lower values are more stable. +* `top_p`: Controls nucleus sampling; higher values expand candidate set, lower values converge candidate set. +* `temperature` defaults to `1.0`, `top_p` defaults to `0.95`, not recommended to adjust both simultaneously. + +```python theme={null} +# Plan A: Use temperature (recommended) +resp = client.chat.completions.create( + model="glm-5.2", + messages=[{"role": "user", "content": "Write a more creative brand introduction"}], + temperature=1.0 +) + +# Plan B: Use top_p +resp = client.chat.completions.create( + model="glm-5.2", + messages=[{"role": "user", "content": "Generate more stable technical documentation"}], + top_p=0.8 +) +``` + +### 3. Deep Thinking (Optional) + +* GLM-5.2 continues to support deep thinking capability, enabled by default. +* Recommended to enable for complex reasoning and coding tasks: + +```python theme={null} +resp = client.chat.completions.create( + model="glm-5.2", + messages=[{"role": "user", "content": "Design a three-tier microservice architecture for me"}], + thinking={"type": "enabled"} +) +``` + +* Control reasoning effort, default `max` for deep reasoning: + +```python theme={null} +resp = client.chat.completions.create( + model="glm-5.2", + messages=[{"role": "user", "content": "Design a three-tier microservice architecture for me"}], + thinking={"type": "enabled"}, + reasoning_effort="max" +) +``` + +### 4. Streaming Output and Tool Calls (Optional) + +* GLM-5.2 supports real-time streaming construction and output during tool calling process, disabled by default (`False`), requires enabling both: + * `stream=True`: Enable streaming output for responses + * `tool_stream=True`: Enable streaming output for tool call parameters + +```python theme={null} +response = client.chat.completions.create( + model="glm-5.2", + messages=[{"role": "user", "content": "How's the weather in Beijing"}], + tools=[ + { + "type": "function", + "function": { + "name": "get_weather", + "description": "Get current weather conditions for a specified location", + "parameters": { + "type": "object", + "properties": { + "location": {"type": "string", "description": "City, eg: Beijing, Shanghai"}, + "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]} + }, + "required": ["location"] + } + } + } + ], + stream=True, + tool_stream=True, +) + +# Initialize streaming collection variables +reasoning_content = "" +content = "" +final_tool_calls = {} +reasoning_started = False +content_started = False + +# Process streaming response +for chunk in response: + if not chunk.choices: + continue + + delta = chunk.choices[0].delta + + # Streaming reasoning process output + if hasattr(delta, 'reasoning_content') and delta.reasoning_content: + if not reasoning_started and delta.reasoning_content.strip(): + print("\n🧠 Thinking Process:") + reasoning_started = True + reasoning_content += delta.reasoning_content + print(delta.reasoning_content, end="", flush=True) + + # Streaming answer content output + if hasattr(delta, 'content') and delta.content: + if not content_started and delta.content.strip(): + print("\n\n💬 Answer Content:") + content_started = True + content += delta.content + print(delta.content, end="", flush=True) + + # Streaming tool call information (parameter concatenation) + if delta.tool_calls: + for tool_call in delta.tool_calls: + idx = tool_call.index + if idx not in final_tool_calls: + final_tool_calls[idx] = tool_call + final_tool_calls[idx].function.arguments = tool_call.function.arguments + else: + final_tool_calls[idx].function.arguments += tool_call.function.arguments + +# Output final tool call information +if final_tool_calls: + print("\n📋 Function Calls Triggered:") + for idx, tool_call in final_tool_calls.items(): + print(f" {idx}: Function Name: {tool_call.function.name}, Parameters: {tool_call.function.arguments}") +``` + +See: [Tool Streaming Output Documentation](/guides/tools/stream-tool) + +### 5. Testing and Regression + +> First verify in development environment that post-migration calls are stable, focus on: + +* Whether responses meet expectations, whether there's excessive randomness or excessive conservatism in output +* Whether tool streaming construction and output work normally +* Latency and cost in long context and deep thinking scenarios + +## More Resources + + + + Common model parameter concepts and sampling recommendations + + + + View tool streaming output usage details + + + + View complete API documentation + + + + Get technical support and help + + diff --git a/llmsdk_docs/glm5_2/docs/stream-tool.md b/llmsdk_docs/glm5_2/docs/stream-tool.md new file mode 100644 index 00000000..089300b9 --- /dev/null +++ b/llmsdk_docs/glm5_2/docs/stream-tool.md @@ -0,0 +1,148 @@ +> ## Documentation Index +> Fetch the complete documentation index at: https://docs.z.ai/llms.txt +> Use this file to discover all available pages before exploring further. + +# Stream Tool Call + +Stream Tool Call is a unique feature of Z.ai's latest GLM-4.6 model, allowing real-time access to reasoning processes, response content, and tool call information during tool invocation, providing better user experience and real-time feedback. + +## Features + +Tool calling in the latest GLM model now supports streaming output for responses. This allows developers to stream tool usage parameters without buffering or JSON validation when calling `chat.completions`, thereby reducing call latency and providing a better user experience. + +### Core Parameter Description + +* **`stream=True`**: Enable streaming output, must be set to `True` +* **`tool_stream=True`**: Enable tool call streaming output +* **`model`**: Use a model that supports tool calling, limited to `glm-4.6` `glm-4.7` `glm-5` + +### Response Parameter Description + +The `delta` object in streaming responses contains the following fields: + +* **`reasoning_content`**: Text content of the model's reasoning process +* **`content`**: Text content of the model's response +* **`tool_calls`**: Tool call information, including function names and parameters + +## Code Example + +By setting the `tool_stream=True` parameter, you can enable streaming tool call functionality: + + + + **Install SDK** + + ```bash theme={null} + # Install latest version + pip install zai-sdk + + # Or specify version + pip install zai-sdk==0.2.3 + ``` + + **Verify Installation** + + ```python theme={null} + import zai + print(zai.__version__) + ``` + + **Complete Example** + + ```python theme={null} + from zai import ZaiClient + + # Initialize client + client = ZaiClient(api_key='Your API key') + + # Create streaming tool call request + response = client.chat.completions.create( + model="glm-4.6", # Use model that supports tool calling + messages=[ + {"role": "user", "content": "How's the weather in Beijing?"}, + ], + tools=[ + { + "type": "function", + "function": { + "name": "get_weather", + "description": "Get current weather conditions for a specified location", + "parameters": { + "type": "object", + "properties": { + "location": {"type": "string", "description": "City, e.g.: Beijing, Shanghai"}, + "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]} + }, + "required": ["location"] + } + } + } + ], + stream=True, # Enable streaming output + tool_stream=True # Enable tool call streaming output + ) + + # Initialize variables to collect streaming data + reasoning_content = "" # Reasoning process content + content = "" # Response content + final_tool_calls = {} # Tool call information + reasoning_started = False # Reasoning process start flag + content_started = False # Content output start flag + + # Process streaming response + for chunk in response: + if not chunk.choices: + continue + + delta = chunk.choices[0].delta + + # Handle streaming reasoning process output + if hasattr(delta, 'reasoning_content') and delta.reasoning_content: + if not reasoning_started and delta.reasoning_content.strip(): + print("\n🧠 Thinking Process:") + reasoning_started = True + reasoning_content += delta.reasoning_content + print(delta.reasoning_content, end="", flush=True) + + # Handle streaming response content output + if hasattr(delta, 'content') and delta.content: + if not content_started and delta.content.strip(): + print("\n\n💬 Response Content:") + content_started = True + content += delta.content + print(delta.content, end="", flush=True) + + # Handle streaming tool call information + if delta.tool_calls: + for tool_call in delta.tool_calls: + index = tool_call.index + if index not in final_tool_calls: + # New tool call + final_tool_calls[index] = tool_call + final_tool_calls[index].function.arguments = tool_call.function.arguments + else: + # Append tool call parameters (streaming construction) + final_tool_calls[index].function.arguments += tool_call.function.arguments + + # Output final tool call information + if final_tool_calls: + print("\n📋 Function Calls Triggered:") + for index, tool_call in final_tool_calls.items(): + print(f" {index}: Function Name: {tool_call.function.name}, Parameters: {tool_call.function.arguments}") + ``` + + + +## Use Cases + + + + * Real-time display of query progress + * Improve waiting experience + + + + * Real-time code analysis process + * Display tool call chain + + diff --git a/llmsdk_docs/glm5_2/docs/thinking-mode.md b/llmsdk_docs/glm5_2/docs/thinking-mode.md new file mode 100644 index 00000000..a85793b5 --- /dev/null +++ b/llmsdk_docs/glm5_2/docs/thinking-mode.md @@ -0,0 +1,126 @@ +> ## Documentation Index +> Fetch the complete documentation index at: https://docs.z.ai/llms.txt +> Use this file to discover all available pages before exploring further. + +# Thinking Mode + +GLM offers multiple thinking modes for different scenarios. The sections below explain how to enable each mode, key considerations, and example usage. + +## **Default Thinking Behaviour** + +Thinking is activated by default in GLM-5.2 GLM-5.1 GLM-5 GLM-4.7 series, different from the default hybrid thinking in GLM-4.6. + +> If you want to disable thinking, use: + +```bash theme={null} +"thinking": { + "type": "disabled" +} +``` + +## **Interleaved thinking** + +We support **interleaved thinking** by default (supported since GLM-4.5), allowing GLM to think between tool calls and after receiving tool results. This enables more complex, step-by-step reasoning: interpreting each tool output before deciding what to do next, chaining multiple tool calls with reasoning steps, and making finer-grained decisions based on intermediate results. + + + When using interleaved thinking with tools, **thinking blocks should be explicitly preserved and returned together with the tool results.** + + +The detailed interleaved thinking process is as follows. + +![Description](https://cdn.bigmodel.cn/markdown/1766025484368img_v3_02t3_4677ac48-b748-44d8-a56f-8cbd599b51ag.jpg?attname=img_v3_02t3_4677ac48-b748-44d8-a56f-8cbd599b51ag.jpg) + +## **Preserved thinking** + +**We introduce a new capability** in coding scenarios: the model can retain **reasoning content from previous assistant turns** in the context. This helps preserve reasoning continuity and conversation integrity, improves model performance, and increases cache hit rates—saving tokens in real tasks. + + + This capability is **enabled by default** on the **Coding Plan endpoint** and **disabled by default** on the **standard API endpoint**. If you want to enable **Preserved Thinking** in your product (primarily recommended for coding/agent scenarios), you can turn it on for the API endpoint by setting **"clear\_thinking": false**, and **you must return the complete**, unmodified reasoning\_content back to the API. + + All consecutive reasoning\_content blocks must **exactly match the original sequence** generated by the model during the initial request. Do not reorder or edit these blocks; otherwise, performance may degrade and cache hit rates may be affected. + + +The detailed Preserved thinking process is as follows. + +![Description](https://cdn.bigmodel.cn/markdown/176641919972020251222-235942.jpeg?attname=20251222-235942.jpeg) + +## Turn-level Thinking + +“Turn-level Thinking” is a capability that **lets you control reasoning computation on a per-turn basis**: within the same session, each request can independently choose to enable or disable thinking. This is a new capability introduced in GLM-4.7, with the following advantages: + +* **More flexible cost/latency control:** For lightweight turns like “asking a fact” or “tweaking wording,” you can disable thinking to get faster responses; for heavier tasks like “complex planning,” “multi-constraint reasoning,” or “code debugging,” you can enable thinking to improve accuracy and stability. +* **Smoother multi-turn experience:** The thinking switch can be toggled at any point within a session. The model stays coherent across turns and keeps a consistent output style, making it feel “smarter when things are hard, faster when things are simple.” +* **Better for agent/tool-use scenarios:** On turns that require quick tool execution, you can reduce reasoning overhead; on turns that require making decisions based on tool results, you can turn on deeper thinking—dynamically balancing efficiency and quality. + +## Example Usage + +This applies to both **Interleaved Thinking** and **Preserved Thinking**—no manual differentiation is required. **Remember to return the historical** `reasoning_content`**to keep the reasoning coherent.** + +```python theme={null} +""""Interleaved Thinking + Tool Calling Example""" + +import json +from openai import OpenAI + +client = OpenAI( + api_key="YOUR_API_KEY", + base_url="https://api.z.ai/api/paas/v4/", +) + +tools = [{"type": "function", "function": { + "name": "get_weather", + "description": "Get weather information", + "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}, +}}] + +messages = [ + {"role": "system", "content": "You are an assistant"}, + {"role": "user", "content": "What's the weather like in Beijing?"}, +] + +# Round 1: the model reasons and then calls a tool +response = client.chat.completions.create(model="glm-4.7", messages=messages, tools=tools, stream=True, extra_body={ + "thinking":{ + "type":"enabled", + "clear_thinking": False # False for Preserved Thinking + }}) +reasoning, content, tool_calls = "", "", [] +for chunk in response: + delta = chunk.choices[0].delta + if hasattr(delta, "reasoning_content") and delta.reasoning_content: + reasoning += delta.reasoning_content + if hasattr(delta, "content") and delta.content: + content += delta.content + if hasattr(delta, "tool_calls") and delta.tool_calls: + for tc in delta.tool_calls: + if tc.index >= len(tool_calls): + tool_calls.append({"id": tc.id, "function": {"name": "", "arguments": ""}}) + if tc.function.name: + tool_calls[tc.index]["function"]["name"] = tc.function.name + if tc.function.arguments: + tool_calls[tc.index]["function"]["arguments"] += tc.function.arguments + +print(f"Reasoning: {reasoning}\nTool calls: {tool_calls}") + +# Key: return reasoning_content to keep the reasoning coherent +messages.append({"role": "assistant", "content": content, "reasoning_content": reasoning, + "tool_calls": [{"id": tc["id"], "type": "function", "function": tc["function"]} for tc in tool_calls]}) +messages.append({"role": "tool", "tool_call_id": tool_calls[0]["id"], + "content": json.dumps({"weather": "Sunny", "temp": "25°C"})}) + +# Round 2: the model continues reasoning based on the tool result and responds +response = client.chat.completions.create(model="glm-4.7", messages=messages, tools=tools, stream=True, extra_body={ + "thinking":{ + "type":"enabled", + "clear_thinking": False # False for Preserved Thinking + }}) +reasoning, content = "", "" +for chunk in response: + delta = chunk.choices[0].delta + if hasattr(delta, "reasoning_content") and delta.reasoning_content: + reasoning += delta.reasoning_content + if hasattr(delta, "content") and delta.content: + content += delta.content + +print(f"Reasoning: {reasoning}\nReply: {content}") +``` diff --git a/llmsdk_docs/glm5_2/docs/thinking.md b/llmsdk_docs/glm5_2/docs/thinking.md new file mode 100644 index 00000000..cd961cbe --- /dev/null +++ b/llmsdk_docs/glm5_2/docs/thinking.md @@ -0,0 +1,344 @@ +> ## Documentation Index +> Fetch the complete documentation index at: https://docs.z.ai/llms.txt +> Use this file to discover all available pages before exploring further. + +# Deep Thinking + +Deep Thinking is an advanced reasoning feature that enables Chain of Thought mechanisms, allowing the model to perform deep analysis and reasoning before answering questions. This approach significantly improves the model's accuracy and interpretability in complex tasks, particularly suitable for scenarios requiring multi-step reasoning, logical analysis, and problem-solving. + +## Features + +The Deep Thinking feature currently supports the latest models in the GLM-5.2 GLM-5.1 GLM-5 GLM-5-Turbo GLM-5V-Turbo GLM-4.5 GLM-4.6 GLM-4.7 series. By enabling deep thinking, the model can: + +* **Multi-step Reasoning**: Break down complex problems into multiple steps for gradual analysis and resolution +* **Logical Analysis**: Provide clear reasoning processes and logical chains +* **Improved Accuracy**: Reduce errors and improve answer quality through deep thinking +* **Enhanced Interpretability**: Display the thinking process to help users understand the model's reasoning logic +* **Intelligent Judgment**: The model automatically determines whether deep thinking is needed to optimize response efficiency + +### Core Parameters + +* **`thinking.type`**: Controls the deep thinking mode + * `enabled` (default): Enable dynamic thinking. The model automatically determines whether to think: `GLM-5.2`, `GLM-5.1`, `GLM-5`, `GLM-5-Turbo`, `GLM-5V-Turbo`, `GLM-4.6`, and `GLM-4.5` auto-decide whether to think, while `GLM-4.7` and `GLM-4.5V` use forced thinking + * `disabled`: Disable deep thinking, provide direct answers +* **`reasoning_effort`**: Controls the reasoning effort level when chain-of-thought thinking is enabled, only supported by `GLM-5.2` and above + * Allowed values: `max` (default and recommended, deep reasoning), `xhigh`, `high` (enhanced reasoning), `medium`, `low`, `minimal`, `none` + * `none` or `minimal`: Model skips thinking; `low` / `medium` are mapped to `high`; `xhigh` is mapped to `max` +* **`model`**: Models that support deep thinking, such as `glm-5.2`, `glm-5.1`, `glm-5`, `glm-4.7`, `glm-4.6`, `glm-4.5`, `glm-4.5v`, etc. + +## Code Examples + + + + **Basic Call (Enable Deep Thinking)** + + ```bash theme={null} + curl --location 'https://api.z.ai/api/paas/v4/chat/completions' \ + --header 'Authorization: Bearer YOUR_API_KEY' \ + --header 'Content-Type: application/json' \ + --data '{ + "model": "glm-5.2", + "messages": [ + { + "role": "user", + "content": "Explain in detail the basic principles of quantum computing and analyze its potential impact in the field of cryptography" + } + ], + "thinking": { + "type": "enabled" + }, + "max_tokens": 4096, + "temperature": 1.0 + }' + ``` + + **Streaming Call (Deep Thinking + Streaming Output)** + + ```bash theme={null} + curl --location 'https://api.z.ai/api/paas/v4/chat/completions' \ + --header 'Authorization: Bearer YOUR_API_KEY' \ + --header 'Content-Type: application/json' \ + --data '{ + "model": "glm-5.2", + "messages": [ + { + "role": "user", + "content": "Design a recommendation system architecture for an e-commerce website, considering user behavior, product features, and real-time requirements" + } + ], + "thinking": { + "type": "enabled" + }, + "stream": true, + "max_tokens": 4096, + "temperature": 1.0 + }' + ``` + + **Control Reasoning Effort (reasoning\_effort)** + + ```bash theme={null} + curl --location 'https://api.z.ai/api/paas/v4/chat/completions' \ + --header 'Authorization: Bearer YOUR_API_KEY' \ + --header 'Content-Type: application/json' \ + --data '{ + "model": "glm-5.2", + "messages": [ + { + "role": "user", + "content": "Analyze the solution approach for this math problem" + } + ], + "thinking": { + "type": "enabled" + }, + "reasoning_effort": "max" + }' + ``` + + **Disable Deep Thinking** + + ```bash theme={null} + curl --location 'https://api.z.ai/api/paas/v4/chat/completions' \ + --header 'Authorization: Bearer YOUR_API_KEY' \ + --header 'Content-Type: application/json' \ + --data '{ + "model": "glm-5.2", + "messages": [ + { + "role": "user", + "content": "How is the weather today?" + } + ], + "thinking": { + "type": "disabled" + } + }' + ``` + + + + **Install SDK** + + ```bash theme={null} + # Install latest version + pip install zai-sdk + + # Or specify version + pip install zai-sdk==0.2.3 + ``` + + **Verify Installation** + + ```python theme={null} + import zai + print(zai.__version__) + ``` + + **Basic Call (Enable Deep Thinking)** + + ```python theme={null} + from zai import ZaiClient + + # Initialize client + client = ZaiClient(api_key='your_api_key') + + # Create deep thinking request + response = client.chat.completions.create( + model="glm-5.2", + messages=[ + {"role": "user", "content": "Explain in detail the basic principles of quantum computing and analyze its potential impact in the field of cryptography"} + ], + thinking={ + "type": "enabled" # Enable deep thinking mode + }, + max_tokens=4096, + temperature=1.0 + ) + + print("Model response:") + print(response.choices[0].message.content) + print("\n---") + print(response.choices[0].message.reasoning_content) + ``` + + **Streaming Call (Deep Thinking + Streaming Output)** + + ```python theme={null} + from zai import ZaiClient + + # Initialize client + client = ZaiClient(api_key='your_api_key') + + # Create streaming deep thinking request + response = client.chat.completions.create( + model="glm-5.2", + messages=[ + {"role": "user", "content": "Design a recommendation system architecture for an e-commerce website, considering user behavior, product features, and real-time requirements"} + ], + thinking={ + "type": "enabled" # Enable deep thinking mode + }, + stream=True, # Enable streaming output + max_tokens=4096, + temperature=1.0 + ) + + # Process streaming response + reasoning_content = "" + thinking_phase = True + + for chunk in response: + if not chunk.choices: + continue + + delta = chunk.choices[0].delta + + # Process thinking process (if any) + if hasattr(delta, 'reasoning_content') and delta.reasoning_content: + reasoning_content += delta.reasoning_content + if thinking_phase: + print("🧠 Thinking...", end="", flush=True) + thinking_phase = False + print(delta.reasoning_content, end="", flush=True) + + # Process answer content + if hasattr(delta, 'content') and delta.content: + if thinking_phase: + print("\n\n💡 Answer:") + thinking_phase = False + print(delta.content, end="", flush=True) + + ``` + + **Control Reasoning Effort (reasoning\_effort)** + + ```python theme={null} + from zai import ZaiClient + + # Initialize client + client = ZaiClient(api_key='your_api_key') + + # Use reasoning_effort to control reasoning level + response = client.chat.completions.create( + model="glm-5.2", + messages=[ + {"role": "user", "content": "Analyze the solution approach for this math problem"} + ], + thinking={ + "type": "enabled" + }, + reasoning_effort="high" # Options: max, xhigh, high, medium, low, minimal, none + ) + + print(response.choices[0].message.content) + print(response.choices[0].message.reasoning_content) + ``` + + **Disable Deep Thinking** + + ```python theme={null} + from zai import ZaiClient + + # Initialize client + client = ZaiClient(api_key='your_api_key') + + # Disable deep thinking for quick response + response = client.chat.completions.create( + model="glm-5.2", + messages=[ + {"role": "user", "content": "How is the weather today?"} + ], + thinking={ + "type": "disabled" # Disable deep thinking mode + } + ) + + print(response.choices[0].message.content) + ``` + + + +### Response Example + +Response format with deep thinking enabled: + +```json theme={null} +{ + "created": 1677652288, + "model": "glm-5.2", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "Artificial intelligence has tremendous application prospects in medical diagnosis...", + "reasoning_content": "Let me analyze this question from multiple angles. First, I need to consider the technical advantages of AI in medical diagnosis..." + }, + "finish_reason": "stop" + } + ], + "usage": { + "completion_tokens": 239, + "prompt_tokens": 8, + "prompt_tokens_details": { + "cached_tokens": 0 + }, + "total_tokens": 247 + } +} +``` + +## Best Practices + +**Recommended scenarios to enable:** + +* Complex problem analysis and solving +* Multi-step reasoning tasks +* Technical solution design +* Strategy planning and decision +* Academic research and analysis +* Creative writing and content creation + +**Can be disabled scenarios:** + +* Simple fact query +* Basic translation tasks +* Simple classification judgment +* Quick question and answer requirements + +## Application scenarios + + + + * Research method design + * Data analysis and explanation + * Theory deduction and proof + + + + * System architecture design + * Technological scheme evaluation + * Problem diagnosis and solution + + + + * Market trends analysis + * Business model design + * Investment decision support + + + + * Complex concept explanation + * Learning path planning + * Knowledge system building + + + +## Notes + +1. **Response time**：Enable deep thinking will increase response time, particularly for complex tasks +2. **Token consumption**：Thinking process will consume extra tokens, please manage your tokens +3. **Model support**：Ensure you're using models that support deep thinking +4. **Task matching**：Choose whether to enable deep thinking according to the task complexity +5. **Streaming output**：Combine streaming output to see the thinking process, improving user experience diff --git a/llmsdk_docs/kimi_k3/README.md b/llmsdk_docs/kimi_k3/README.md new file mode 100644 index 00000000..999c29b1 --- /dev/null +++ b/llmsdk_docs/kimi_k3/README.md @@ -0,0 +1,34 @@ +# Kimi K3 SDK Documentation + +This directory contains documentation for Moonshot's Kimi K3 API, snapshotted from the +official platform documentation (https://platform.kimi.com/docs/). + +## Quick Start + +- See [quickstart.md](./quickstart.md) (OpenAI-compatible API, Python and cURL examples) + +## Documentation + +The `docs/` folder contains detailed guides on Kimi K3 features: + +- [thinking-effort.md](./docs/thinking-effort.md) - The `reasoning_effort` parameter (`low`/`high`/`max`, default `max`; reasoning cannot be disabled) +- [tool-choice.md](./docs/tool-choice.md) - `tool_choice` values (`auto`/`none`/`required`/specific function) +- [tool-calling-best-practice.md](./docs/tool-calling-best-practice.md) - K3 tool calling best practices +- [tool-calls.md](./docs/tool-calls.md) - Complete tool calling walkthrough +- [vision.md](./docs/vision.md) - Image and video input (base64 or `ms://`; public URLs are not supported) +- [context-caching.md](./docs/context-caching.md) - Automatic context caching (no extra request parameters) +- [streaming.md](./docs/streaming.md) - Streaming output (`reasoning_content` and `content` deltas) +- [chat-api.md](./docs/chat-api.md) - Chat Completion API reference +- [models-overview.md](./docs/models-overview.md) - Model overview +- [pricing.md](./docs/pricing.md) - K3 pricing and context window + +## Key protocol differences vs Kimi K2.x + +- Reasoning is configured with the top-level `reasoning_effort` parameter (`low`/`high`/`max`, + default `max`) instead of the K2.x `extra_body.thinking` object, and cannot be disabled. +- `tool_choice` additionally supports `required`; forcing a specific function is incompatible + with reasoning (which is always on). +- Context caching is fully automatic; no `prompt_cache_key` or other cache parameters are needed. +- Multi-turn conversations must replay the complete assistant message exactly as received, + including `reasoning_content` and `tool_calls`. +- Sampling parameters (temperature and friends) remain fixed, as in K2.x. diff --git a/llmsdk_docs/kimi_k3/docs/chat-api.md b/llmsdk_docs/kimi_k3/docs/chat-api.md new file mode 100644 index 00000000..33b02e1e --- /dev/null +++ b/llmsdk_docs/kimi_k3/docs/chat-api.md @@ -0,0 +1,1386 @@ +> Fetch the complete documentation index at: https://platform.kimi.com/docs/llms.txt +> Use this file to discover all available pages before exploring further. + +# 创建对话补全 + +> 为聊天消息创建补全结果。支持标准聊天、Partial Mode 和 Tool Use（函数调用）。 + +创建一个对话补全请求，模型将根据输入的消息列表生成回复。 + + + `content` 字段支持以下两种形式： + + **纯文本字符串** + + ```json theme={null} + { "content": "你好" } + ``` + + **对象数组**（用于多模态输入） + + 数组中每个元素通过 `type` 字段区分类型： + + ```json theme={null} + { + "content": [ + { "type": "text", "text": "描述这张图片" }, + { "type": "image_url", "image_url": { "url": "data:image/png;base64,..." } }, + { "type": "video_url", "video_url": { "url": "data:video/mp4;base64,..." } } + ] + } + ``` + + 其中 `image_url` 和 `video_url` 也支持直接传入字符串，效果等同于对象形式中的 `url` 字段： + + ```json theme={null} + { "type": "image_url", "image_url": "data:image/png;base64,..." } + ``` + + #### 参数说明 + + 数组中每个元素的字段说明如下： + + | 参数名称 | 是否必须 | 说明 | 类型 | + | ----------- | ---------------------- | -------------------------------------------- | ------------------------------------------ | + | `type` | required | 内容类型 | `"text"` \| `"image_url"` \| `"video_url"` | + | `text` | 当 `type=text` 时必填 | 文本内容 | string | + | `image_url` | 当 `type=image_url` 时必填 | 用于传输图片，支持对象形式 `{"url": "..."}` 或直接传入 URL 字符串 | object \| string | + | `video_url` | 当 `type=video_url` 时必填 | 用于传输视频，支持对象形式 `{"url": "..."}` 或直接传入 URL 字符串 | object \| string | + + 当 `image_url` 传入对象时，其字段说明如下： + + | 参数名称 | 是否必须 | 说明 | 类型 | + | ----- | -------- | ------------------------------- | ------ | + | `url` | required | 使用 base64 编码或通过 file id 指定的图片内容 | string | + + 当 `video_url` 传入对象时，其字段说明如下： + + | 参数名称 | 是否必须 | 说明 | 类型 | + | ----- | -------- | -------------------------------------------------------------- | ------ | + | `url` | required | 使用 base64 编码或通过 file id 指定的视频内容，例如 `data:video/mp4;base64,...` | string | + + + 无论使用对象形式（`url` 字段）还是字符串简写，均支持以下两种格式： + + * base64 编码：`data:image/png;base64,...` 或 `data:video/mp4;base64,...` + * 文件引用：`ms://` + + 详见[使用 Kimi 视觉模型](/docs/guide/use-kimi-vision-model)。 + + + #### 调用示例 + + + ```python python expandable theme={null} + import os + import base64 + + from openai import OpenAI + from openai.types.chat import ChatCompletion + + client: OpenAI = OpenAI( + api_key=os.environ.get("MOONSHOT_API_KEY"), + base_url="https://api.moonshot.cn/v1", + ) + + # 对图片进行 base64 编码 + with open("您的图片地址", "rb") as f: + img_base: str = base64.b64encode(f.read()).decode("utf-8") + + response: ChatCompletion = client.chat.completions.create( + model="kimi-k2.6", + messages=[ + { + "role": "user", + "content": [ + { + "type": "image_url", + "image_url": { + "url": f"data:image/jpeg;base64,{img_base}", + }, + }, + { + "type": "text", + "text": "请描述这个图片", + }, + ], + } + ], + ) + print(response.choices[0].message.content) + ``` + + ```bash curl expandable theme={null} + curl https://api.moonshot.cn/v1/chat/completions \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $MOONSHOT_API_KEY" \ + -d '{ + "model": "kimi-k2.6", + "messages": [ + { + "role": "user", + "content": [ + { + "type": "image_url", + "image_url": { + "url": "data:image/jpeg;base64,/9j/4AAQ..." + } + }, + { + "type": "text", + "text": "请描述这个图片" + } + ] + } + ] + }' + ``` + + ```javascript node.js expandable theme={null} + const fs = require("fs"); + const OpenAI = require("openai"); + + const client = new OpenAI({ + apiKey: process.env.MOONSHOT_API_KEY, + baseURL: "https://api.moonshot.cn/v1", + }); + + async function main() { + // 对图片进行 base64 编码 + const imgBase = fs.readFileSync("您的图片地址").toString("base64"); + + const response = await client.chat.completions.create({ + model: "kimi-k2.6", + messages: [ + { + role: "user", + content: [ + { + type: "image_url", + image_url: { + url: `data:image/jpeg;base64,${imgBase}`, + }, + }, + { + type: "text", + text: "请描述这个图片", + }, + ], + }, + ], + }); + console.log(response.choices[0].message.content); + } + + main(); + ``` + + + + + ### 非流式响应 + + ```json theme={null} + { + "id": "cmpl-04ea926191a14749b7f2c7a48a68abc6", + "object": "chat.completion", + "created": 1698999496, + "model": "kimi-k2.6", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "你好，李雷！1+1等于2。如果你有其他问题，请随时提问！" + }, + "finish_reason": "stop" + } + ], + "usage": { + "prompt_tokens": 19, + "completion_tokens": 21, + "total_tokens": 40, + "cached_tokens": 10 + } + } + ``` + + ### 流式响应 + + ```text theme={null} + data: {"id":"cmpl-xxx","object":"chat.completion.chunk","created":1698999575,"model":"kimi-k2.6","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]} + + data: {"id":"cmpl-xxx","object":"chat.completion.chunk","created":1698999575,"model":"kimi-k2.6","choices":[{"index":0,"delta":{"content":"你好"},"finish_reason":null}]} + + ... + + data: {"id":"cmpl-xxx","object":"chat.completion.chunk","created":1698999575,"model":"kimi-k2.6","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":19,"completion_tokens":13,"total_tokens":32,"cached_tokens":12}} + + data: [DONE] + ``` + + + 响应示例中的模型名称会根据请求中的 model 参数返回。当使用 `kimi-k2.6` 模型时，响应中的 `"model"` 字段将显示为 `"kimi-k2.6"`。 + + + + + Kimi API 是无状态的，本身不具有记忆功能。要实现多轮对话，需在每次请求时把前一轮的 assistant 回复（以及工具执行结果，如适用）原样追加到 `messages` 数组中再发送。 + + ```python theme={null} + messages = [ + {"role": "system", "content": "你是 Kimi。"}, + {"role": "user", "content": "你好，我叫李雷。"} + ] + + completion = client.chat.completions.create(model="kimi-k2.6", messages=messages) + reply = completion.choices[0].message + + # 将 assistant 回复追加回 messages，供下一轮使用 + messages.append({"role": "assistant", "content": reply.content}) + messages.append({"role": "user", "content": "1+1 等于多少？"}) + ``` + + 当对话历史过长时，建议只保留最近的若干条消息，或做消息压缩，以避免超出模型的上下文长度限制。 + + + 详见 [配置多轮对话参数](/docs/guide/engage-in-multi-turn-conversations-using-kimi-api)。 + + + + + 通过 `response_format` 参数可约束模型输出格式： + + * `{"type": "text"}`（默认）：普通文本输出 + * `{"type": "json_object"}`：强制输出合法 JSON Object + * `{"type": "json_schema", "json_schema": {...}}`：按给定 JSON Schema 输出结构化数据（Structured Output） + + 使用 `json_object` 时，**必须在 system prompt 或 user prompt 中明确描述期望的 JSON 字段和类型**，否则模型可能输出不符合预期的结果。 + + ```json theme={null} + { + "model": "kimi-k2.6", + "messages": [ + {"role": "system", "content": "请输出 JSON，包含 title、author、summary 字段。"}, + {"role": "user", "content": "总结这篇文章..."} + ], + "response_format": {"type": "json_object"} + } + ``` + + + 详见 [使用 Kimi API 的 JSON Mode](/docs/guide/use-json-mode-feature-of-kimi-api)。 + + + + + 通过 `tools` 参数传入 JSON Schema 定义的外部工具，模型可决定在适当时机调用它们。 + + **请求示例** + + ```json theme={null} + { + "model": "kimi-k2.6", + "messages": [{"role": "user", "content": "北京今天天气怎么样？"}], + "tools": [ + { + "type": "function", + "function": { + "name": "get_weather", + "description": "获取指定城市的天气", + "parameters": { + "type": "object", + "properties": { + "city": {"type": "string", "description": "城市名称"} + }, + "required": ["city"] + } + } + } + ] + } + ``` + + **响应中的 `tool_calls`** + + 当 `finish_reason` 为 `"tool_calls"` 时，模型返回 `tool_calls` 数组，包含 `id`、`function.name` 和 `function.arguments`： + + ```json theme={null} + { + "choices": [{ + "message": { + "role": "assistant", + "content": "", + "tool_calls": [{ + "id": "call_xxx", + "type": "function", + "function": { + "name": "get_weather", + "arguments": "{\"city\":\"北京\"}" + } + }] + }, + "finish_reason": "tool_calls" + }] + } + ``` + + **提交工具执行结果** + + 在本地执行工具后，将结果通过 `role="tool"` 消息追加到 `messages` 中（`tool_call_id` 必须与请求中的 `id` 对应）： + + ```json theme={null} + {"role": "tool", "tool_call_id": "call_xxx", "content": "晴，25°C"} + ``` + + + 详见 [使用 Kimi API 完成工具调用](/docs/guide/use-kimi-api-to-complete-tool-calls)。 + + + + + `kimi-k2.6` 和 `kimi-k2.7-code` 支持思考模式，模型在输出最终答案前会先输出推理过程（`reasoning_content`）。K3 始终进行推理，使用顶层 `reasoning_effort`（当前唯一值为 `"max"`）。 + + **K2.x 请求参数** + + | 字段 | 类型 | 说明 | + | --------------- | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | + | `thinking.type` | `"enabled"` \| `"disabled"` | 思考开关（`kimi-k2.7-code` 始终为 `enabled`，不可关闭） | + | `thinking.keep` | `null` \| `"all"` | Preserved Thinking：是否将历史轮次的 `reasoning_content` 保留在上下文。`kimi-k2.6` 默认 `null`（不保留），可选 `"all"`；`kimi-k2.7-code` 固定为 `"all"`（保留），传入其他值报错 | + + **响应字段** + + 非流式响应中，`choices[0].message` 包含： + + | 字段 | 说明 | + | ------------------- | ----------------- | + | `content` | 最终答案 | + | `reasoning_content` | 推理过程（仅在思考模式启用时返回） | + + ```json theme={null} + { + "choices": [{ + "message": { + "role": "assistant", + "content": "1+1 等于 2。", + "reasoning_content": "用户问的是基础数学问题，直接相加即可。" + } + }] + } + ``` + + + 多轮对话中若使用思考模式，请务必将每一轮 assistant 消息的 `reasoning_content` 原样保留在 `messages` 中，否则模型可能丢失推理上下文。 + + + + 详见 [配置思考模式](/docs/guide/use-kimi-k2-thinking-model)。 + + + + + 设置 `stream: true` 可启用流式输出，模型会以 Server-Sent Events (SSE) 格式逐段返回生成的内容。推荐在聊天、代码生成、长文本输出等实时性要求高的场景中使用。 + + ```python theme={null} + completion = client.chat.completions.create( + model="kimi-k2.6", + messages=[{"role": "user", "content": "请解释什么是递归。"}], + stream=True + ) + + for chunk in completion: + if chunk.choices[0].delta.content: + print(chunk.choices[0].delta.content, end="") + ``` + + **SSE 响应格式** + + 每一行以 `data:` 开头，内容为 JSON 对象。当 `finish_reason` 为 `null` 时，内容在 `delta.content` 中累加；当 `finish_reason` 不为 `null` 时，表示输出结束： + + ```text theme={null} + data: {"id":"cmpl-xxx","object":"chat.completion.chunk","created":1698999575,"model":"kimi-k2.6","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]} + + data: {"id":"cmpl-xxx","object":"chat.completion.chunk","created":1698999575,"model":"kimi-k2.6","choices":[{"index":0,"delta":{"content":"你好"},"finish_reason":null}]} + + data: {"id":"cmpl-xxx","object":"chat.completion.chunk","created":1698999575,"model":"kimi-k2.6","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":19,"completion_tokens":13,"total_tokens":32,"cached_tokens":12}} + + data: [DONE] + ``` + + **`stream_options`** + + 通过 `stream_options: {"include_usage": true}` 可在最后一个 chunk（`data: [DONE]` 之前）额外获取 `usage` 字段，显示本次请求的 Token 消耗： + + ```python theme={null} + stream=True, + stream_options={"include_usage": True} + ``` + + + 详见 [利用 Kimi API 的流式输出功能](/docs/guide/utilize-the-streaming-output-feature-of-kimi-api)。 + + + + + Partial Mode（Prefill）允许你在 `messages` 的最后一条 assistant 消息中预填输出前缀，从而引导模型按照你期望的格式或方向继续生成。 + + **开启方式** + + 在 `messages` 数组末尾添加一条 `role="assistant"` 的消息，并设置 `partial: true`： + + ````python theme={null} + completion = client.chat.completions.create( + model="kimi-k2.6", + messages=[ + {"role": "user", "content": "用 Python 实现快速排序。"}, + {"role": "assistant", "content": "```python\n", "partial": True} + ] + ) + ```` + + 模型会从 \`\`\`\`python\n\` 之后继续生成代码，而不是先输出解释文字再写代码。 + + **常见用途** + + * 强制模型以特定格式开头（如 JSON 的 `{`、代码块的 \`\`\`\`python\`） + * 角色扮演中保持角色名称前缀（配合 `name` 字段） + * 在 `finish_reason="length"` 时，用相同的前缀续写被截断的内容 + + + 请勿将 Partial Mode 与 `response_format={"type": "json_object"}` 混用，否则可能获得预期外的模型回复。如需引导 JSON 输出，建议直接使用 [Structured Output](/docs/guide/response_format) 或单独设置 `partial: true` 并预填 `{`。 + + + + 详见 [使用 Kimi API 的 Partial Mode](/docs/guide/use-partial-mode-feature-of-kimi-api)。 + + + + +## OpenAPI + +````yaml POST /v1/chat/completions +openapi: 3.1.0 +info: + title: Moonshot AI API + version: 1.0.0 + description: Moonshot AI / Kimi 大语言模型服务 API +servers: + - url: https://api.moonshot.cn + description: 生产环境 +security: [] +paths: + /v1/chat/completions: + post: + tags: + - Chat + summary: 创建聊天补全 + description: 为聊天消息创建补全结果。支持标准聊天、Partial Mode 和 Tool Use（函数调用）。 + requestBody: + required: true + content: + application/json: + schema: + oneOf: + - $ref: '#/components/schemas/KimiK3ChatRequest' + - $ref: '#/components/schemas/KimiK27CodeChatRequest' + - $ref: '#/components/schemas/KimiK26ChatRequest' + - $ref: '#/components/schemas/KimiK25ChatRequest' + - $ref: '#/components/schemas/MoonshotV1ChatRequest' + discriminator: + propertyName: model + mapping: + kimi-k3: + $ref: '#/components/schemas/KimiK3ChatRequest' + kimi-k2.7-code: + $ref: '#/components/schemas/KimiK27CodeChatRequest' + kimi-k2.7-code-highspeed: + $ref: '#/components/schemas/KimiK27CodeChatRequest' + kimi-k2.6: + $ref: '#/components/schemas/KimiK26ChatRequest' + kimi-k2.5: + $ref: '#/components/schemas/KimiK25ChatRequest' + moonshot-v1-8k: + $ref: '#/components/schemas/MoonshotV1ChatRequest' + moonshot-v1-32k: + $ref: '#/components/schemas/MoonshotV1ChatRequest' + moonshot-v1-128k: + $ref: '#/components/schemas/MoonshotV1ChatRequest' + moonshot-v1-auto: + $ref: '#/components/schemas/MoonshotV1ChatRequest' + moonshot-v1-8k-vision-preview: + $ref: '#/components/schemas/MoonshotV1ChatRequest' + moonshot-v1-32k-vision-preview: + $ref: '#/components/schemas/MoonshotV1ChatRequest' + moonshot-v1-128k-vision-preview: + $ref: '#/components/schemas/MoonshotV1ChatRequest' + responses: + '200': + description: 聊天补全响应 + content: + application/json: + schema: + $ref: '#/components/schemas/ChatCompletionResponse' + text/event-stream: + schema: + $ref: '#/components/schemas/ChatCompletionChunk' + '400': + description: 请求错误 - 参数无效 + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + '401': + description: 未授权 - API 密钥无效或缺失 + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + '500': + description: 服务器错误 + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + security: + - bearerAuth: [] +components: + schemas: + KimiK3ChatRequest: + title: kimi-k3 + allOf: + - $ref: '#/components/schemas/ChatRequestCommon' + - type: object + properties: + model: + type: string + description: 模型 ID + enum: + - kimi-k3 + default: kimi-k3 + messages: + type: array + description: >- + Kimi K3 对话消息列表。除标准消息外，还可在任意对话位置插入 {"role": "system", "tools": + [...]} 消息动态加载工具；该动态工具消息不包含 content 字段，并且只影响后续对话。 + items: + $ref: '#/components/schemas/KimiK3Message' + reasoning_effort: + type: string + enum: + - low + - high + - max + default: max + description: >- + Kimi K3 始终启用思考，并开启 Preserved Thinking。思考强度支持 low、high 和 max，默认为 + max。 + required: + - model + - messages + KimiK27CodeChatRequest: + title: kimi-k2.7-code + allOf: + - $ref: '#/components/schemas/ChatRequestBase' + - type: object + properties: + model: + type: string + description: >- + 模型 ID。可选 `kimi-k2.7-code` 或其高速版 + `kimi-k2.7-code-highspeed`；两者为同一模型、参数完全一致，高速版输出速度约 180 + Tokens/s（短上下文场景可达 260 Tokens/s）。 + enum: + - kimi-k2.7-code + - kimi-k2.7-code-highspeed + default: kimi-k2.7-code + thinking: + type: object + description: >- + 控制 kimi-k2.7-code 模型是否启用思考能力，以及是否完整保留多轮对话中的 + reasoning_content。可选参数，默认值为 {"type": "enabled", "keep": "all"}。 + + + 与 kimi-k2.6 的差异： + + - `type` 仅支持 `"enabled"`。与 kimi-k2.6 不同，不支持 `"disabled"` — + 传入会报错。该模型始终开启思考。 + + - `keep` 仅接受合法值 `"all"`；不传或传 `"all"` 时服务端均按 `"all"` + 处理，传入其他非法值会报错。因此该模型始终启用 Preserved Thinking。 + properties: + type: + type: string + enum: + - enabled + description: >- + 启用思考能力。对 kimi-k2.7-code 仅接受 `"enabled"`；传入 `"disabled"` + 会报错。这与同时支持 `"disabled"` 的 kimi-k2.6 不同。 + keep: + type: + - string + - 'null' + enum: + - all + - null + description: >- + 控制是否保留历史对话轮次（previous turns）的 reasoning_content，从而启用 + Preserved Thinking。 + + + - 对 kimi-k2.7-code，该参数仅接受合法值 `"all"`：传 `"all"`、传 `null` + 或不传表现完全一致，服务端均按 `"all"` 处理 — 始终保留历史轮次的 + reasoning_content；传入其他非法值会报错。这与 kimi-k2.6 不同，k2.6 默认为 + `null`（除非显式设为 `"all"`，否则不保留历史思考）。 + + - 由于 Preserved Thinking 始终开启，请把每一轮历史 assistant 消息中的 + reasoning_content 原样保留在 messages 中。 + + - 注意：该参数只影响历史轮次的 reasoning_content；不改变模型在当前 turn + 内是否产生/输出思考内容（由 `type` 控制）。关于使用方式的最佳实践，详见 [Preserved + Thinking](/guide/use-kimi-k2-thinking-model#preserved-thinking)。 + required: + - type + additionalProperties: false + required: + - model + KimiK26ChatRequest: + title: kimi-k2.6 + allOf: + - $ref: '#/components/schemas/ChatRequestBase' + - type: object + properties: + model: + type: string + description: 模型 ID + enum: + - kimi-k2.6 + default: kimi-k2.6 + thinking: + type: object + description: >- + 控制 kimi-k2.6 模型是否启用思考能力，以及是否完整保留多轮对话中的 + reasoning_content。可选参数，默认值为 {"type": "enabled"}。 + properties: + type: + type: string + enum: + - enabled + - disabled + description: 启用或禁用思考能力 + keep: + type: + - string + - 'null' + enum: + - all + - null + description: >- + 控制是否保留历史对话轮次（previous turns）的 reasoning_content，从而启用 + Preserved Thinking。默认为 `null`，即不保留历史轮次的思考内容。 + + + - `null`（默认）或不传：服务端会忽略历史 turns 的 reasoning_content。 + + - `"all"`：保留历史 turns 的 reasoning_content 并随上下文一同提供给模型，启用 + Preserved Thinking。使用时需把每一轮历史 assistant 消息中的 + reasoning_content 原样保留在 messages 中。推荐与 `type: "enabled"` + 搭配使用。 + + - 注意：该参数只影响历史轮次的 reasoning_content；不改变模型在当前 turn + 内是否产生/输出思考内容（由 `type` 控制）。关于使用方式的最佳实践，详见 [Preserved + Thinking](/guide/use-kimi-k2-thinking-model#preserved-thinking)。 + required: + - type + additionalProperties: false + required: + - model + KimiK25ChatRequest: + title: kimi-k2.5 + allOf: + - $ref: '#/components/schemas/ChatRequestBase' + - type: object + properties: + model: + type: string + description: 模型 ID + enum: + - kimi-k2.5 + default: kimi-k2.5 + thinking: + type: object + description: '控制模型是否启用思考能力。可选参数，默认值为 {"type": "enabled"}。' + properties: + type: + type: string + enum: + - enabled + - disabled + description: 启用或禁用思考能力 + required: + - type + additionalProperties: false + required: + - model + MoonshotV1ChatRequest: + title: moonshot-v1 + allOf: + - $ref: '#/components/schemas/ChatRequestBase' + - type: object + properties: + model: + type: string + description: 模型 ID + enum: + - moonshot-v1-8k + - moonshot-v1-32k + - moonshot-v1-128k + - moonshot-v1-auto + - moonshot-v1-8k-vision-preview + - moonshot-v1-32k-vision-preview + - moonshot-v1-128k-vision-preview + default: moonshot-v1-128k + temperature: + type: number + format: float + description: 采样温度，范围 0 到 1。较高的值（如 0.7）使输出更随机，较低的值（如 0.2）使输出更集中和确定。默认值为 0.0。 + default: 0 + minimum: 0 + maximum: 1 + top_p: + type: number + format: float + description: >- + 另一种采样方法，模型考虑累积概率质量为 top_p 的 Token 结果。例如 0.1 表示仅考虑概率质量前 10% 的 + Token。通常建议只修改此参数或 temperature 其中之一。默认值为 1.0。 + default: 1 + minimum: 0 + maximum: 1 + 'n': + type: integer + description: 每条输入消息生成的结果数量。默认为 1，不超过 5。当温度非常接近 0 时，只能返回 1 个结果。 + default: 1 + minimum: 1 + maximum: 5 + presence_penalty: + type: number + format: float + description: 存在惩罚，范围 -2.0 到 2.0。正值会根据 Token 是否出现在文本中进行惩罚，增加模型讨论新话题的可能性 + default: 0 + minimum: -2 + maximum: 2 + frequency_penalty: + type: number + format: float + description: 频率惩罚，范围 -2.0 到 2.0。正值会根据 Token 在文本中的现有频率进行惩罚，降低模型逐字重复相同短语的可能性 + default: 0 + minimum: -2 + maximum: 2 + required: + - model + ChatCompletionResponse: + type: object + properties: + id: + type: string + description: 补全结果的唯一标识符 + object: + type: string + description: 对象类型 + example: chat.completion + created: + type: integer + description: 补全创建时的 Unix 时间戳 + model: + type: string + description: 用于补全的模型 + choices: + type: array + description: 补全选项列表 + items: + type: object + properties: + index: + type: integer + message: + type: object + properties: + role: + type: string + enum: + - assistant + content: + type: + - string + - 'null' + description: 助手的消息内容 + tool_calls: + type: array + description: 模型发起的工具调用 + items: + type: object + properties: + id: + type: string + type: + type: string + enum: + - function + function: + type: object + properties: + name: + type: string + arguments: + type: string + description: 函数参数的 JSON 字符串 + reasoning_content: + type: + - string + - 'null' + description: 推理过程（仅在思考模式启用时返回） + finish_reason: + type: string + enum: + - stop + - length + - tool_calls + usage: + type: object + properties: + prompt_tokens: + type: integer + description: 提示中的 Token 数量 + completion_tokens: + type: integer + description: 补全中的 Token 数量 + total_tokens: + type: integer + description: 使用的总 Token 数量 + cached_tokens: + type: integer + description: 命中缓存的 Token 数量 + ChatCompletionChunk: + type: object + properties: + id: + type: string + description: 补全结果的唯一标识符 + object: + type: string + description: 对象类型 + example: chat.completion.chunk + created: + type: integer + description: 补全创建时的 Unix 时间戳 + model: + type: string + description: 用于补全的模型 + choices: + type: array + description: 补全选项列表 + items: + $ref: '#/components/schemas/ChoiceDelta' + usage: + type: + - object + - 'null' + description: Token 使用统计。包含 usage 的最终 chunk 中为对象，其他 chunk 中为 null + properties: + prompt_tokens: + type: integer + description: 提示中的 Token 数量 + completion_tokens: + type: integer + description: 补全中的 Token 数量 + total_tokens: + type: integer + description: 使用的总 Token 数量 + cached_tokens: + type: integer + description: 命中缓存的 Token 数量 + ErrorResponse: + type: object + properties: + error: + type: object + properties: + message: + type: string + description: 描述错误原因的错误消息 + type: + type: string + description: 错误类型 + code: + type: string + description: 错误码 + required: + - message + required: + - error + ChatRequestCommon: + type: object + properties: + max_tokens: + type: integer + deprecated: true + description: 已弃用，请使用 max_completion_tokens + max_completion_tokens: + type: integer + description: >- + 聊天补全生成的最大 Token 数量。默认值因模型而异：Kimi K3 默认为 131072，最大可设置为 + 1048576。如果结果达到最大 Token 数而未结束，finish reason 将为 "length"；否则为 + "stop"。此值为期望返回的 Token 长度，而非输入加输出的总长度。如果输入加 max_completion_tokens + 超出模型上下文窗口，将返回 invalid_request_error。 + response_format: + type: object + description: >- + 控制模型输出格式。默认值为 {"type": "text"}，即纯文本输出。设置为 {"type": "json_object"} + 可启用 JSON 模式，确保输出为合法 JSON 对象（需在 prompt 中引导模型输出 JSON 并指定格式）。设置为 + {"type": "json_schema"} 可启用 Structured Output，按指定的 JSON Schema + 约束输出结构（推荐，需配合 json_schema 字段使用）。如果您在使用 JSON Schema 时遇到校验问题，欢迎到 walle + GitHub Issues (https://github.com/MoonshotAI/walle/issues) 提交反馈。 + properties: + type: + type: string + enum: + - text + - json_object + - json_schema + description: >- + 输出格式类型。text：默认，纯文本输出；json_object：保证输出为合法 JSON 对象；json_schema：按指定 + JSON Schema 约束输出（推荐，需配合 json_schema 字段使用） + json_schema: + type: object + description: 当 type 为 json_schema 时使用，定义输出应遵循的 JSON Schema + properties: + name: + type: string + description: Schema 名称，用于标识 + strict: + type: boolean + default: true + description: >- + 是否严格按 schema 约束输出。默认为 true。为 true 时 schema 需符合 MFJS + 规范，不符合会返回错误或 warning；为 false 时仅保证输出为合法 JSON 对象，不强制约束内部结构。 + schema: + type: object + description: >- + JSON Schema 对象，定义输出应遵循的结构。需符合 MFJS（Moonshot Flavored JSON + Schema）规范。可使用 walle CLI 工具自检：go install + github.com/moonshotai/walle/cmd/walle@latest && walle + -schema '你的schema' -level strict + additionalProperties: true + required: + - name + - schema + stop: + oneOf: + - type: string + - type: array + items: + type: string + maxItems: 5 + default: null + description: 停用词，完全匹配时将停止输出。匹配到的词本身不会被输出。最多允许 5 个字符串，每个不超过 32 字节 + stream: + type: boolean + default: false + description: 是否以流式方式返回响应，默认 false + stream_options: + type: object + description: 流式响应选项 + properties: + include_usage: + type: boolean + default: false + description: >- + 如果设置，将在 data: [DONE] 消息之前额外发送一个 chunk。该 chunk 的 usage 字段显示整个请求的 + Token 使用统计，choices 字段为空数组。其他所有 chunk 也会包含 usage 字段，但值为 + null。注意：如果流中断，可能无法收到包含总 Token 用量的最终 chunk + tools: + type: array + description: 模型可调用的工具列表 + items: + $ref: '#/components/schemas/ToolDefinition' + prompt_cache_key: + type: string + default: null + description: >- + 用于缓存相似请求的响应以优化缓存命中率。对于 Coding Agent，通常是代表单个会话的 session id 或 task + id；退出并恢复会话时应保持不变。对于 Kimi Code Plan，此字段为必填以提高缓存命中率。对于其他多轮对话 + Agent，也建议使用此字段 + safety_identifier: + type: string + description: 用于检测可能违反使用政策的用户的稳定标识符。应为唯一标识每个用户的字符串。建议对用户名或邮箱进行哈希处理以避免发送可识别信息 + tool_choice: + oneOf: + - type: string + enum: + - auto + - none + - required + description: auto：模型自行决定是否调用工具；none：不调用工具；required：强制调用工具 + - type: object + description: 强制调用指定工具 + properties: + type: + type: string + enum: + - function + function: + type: object + properties: + name: + type: string + description: 要调用的函数名称 + required: + - name + required: + - type + - function + description: >- + 控制模型是否调用工具。`auto`（默认）：模型自行决定是否调用工具；`none`：不调用工具；`required`：强制调用工具；也可传入特定函数对象强制调用指定工具。 + KimiK3Message: + oneOf: + - $ref: '#/components/schemas/Message' + title: 标准消息 + - $ref: '#/components/schemas/KimiK3DynamicToolMessage' + title: 动态工具消息 + description: Kimi K3 对话消息。既支持标准消息，也支持不含 content、通过 tools 声明动态工具的 system 消息。 + ChatRequestBase: + type: object + properties: + messages: + type: array + description: >- + 包含迄今为止对话的消息列表。每个元素格式为 {"role": "user", "content": "你好"}。role 支持 + system、user、assistant、tool 其一，content 不得为空。content 字段可以是 string，也可以是 + array[object]（用于多模态输入） + items: + $ref: '#/components/schemas/Message' + max_tokens: + type: integer + deprecated: true + description: 已弃用，请使用 max_completion_tokens + max_completion_tokens: + type: integer + description: >- + 聊天补全生成的最大 Token 数量。默认值因模型而异：Kimi K3 默认为 131072，最大可设置为 + 1048576。如果结果达到最大 Token 数而未结束，finish reason 将为 "length"；否则为 + "stop"。此值为期望返回的 Token 长度，而非输入加输出的总长度。如果输入加 max_completion_tokens + 超出模型上下文窗口，将返回 invalid_request_error。 + response_format: + type: object + description: >- + 控制模型输出格式。默认值为 {"type": "text"}，即纯文本输出。设置为 {"type": "json_object"} + 可启用 JSON 模式，确保输出为合法 JSON 对象（需在 prompt 中引导模型输出 JSON 并指定格式）。设置为 + {"type": "json_schema"} 可启用 Structured Output，按指定的 JSON Schema + 约束输出结构（推荐，需配合 json_schema 字段使用）。如果您在使用 JSON Schema 时遇到校验问题，欢迎到 walle + GitHub Issues (https://github.com/MoonshotAI/walle/issues) 提交反馈。 + properties: + type: + type: string + enum: + - text + - json_object + - json_schema + description: >- + 输出格式类型。text：默认，纯文本输出；json_object：保证输出为合法 JSON 对象；json_schema：按指定 + JSON Schema 约束输出（推荐，需配合 json_schema 字段使用） + json_schema: + type: object + description: 当 type 为 json_schema 时使用，定义输出应遵循的 JSON Schema + properties: + name: + type: string + description: Schema 名称，用于标识 + strict: + type: boolean + default: true + description: >- + 是否严格按 schema 约束输出。默认为 true。为 true 时 schema 需符合 MFJS + 规范，不符合会返回错误或 warning；为 false 时仅保证输出为合法 JSON 对象，不强制约束内部结构。 + schema: + type: object + description: >- + JSON Schema 对象，定义输出应遵循的结构。需符合 MFJS（Moonshot Flavored JSON + Schema）规范。可使用 walle CLI 工具自检：go install + github.com/moonshotai/walle/cmd/walle@latest && walle + -schema '你的schema' -level strict + additionalProperties: true + required: + - name + - schema + stop: + oneOf: + - type: string + - type: array + items: + type: string + maxItems: 5 + default: null + description: 停用词，完全匹配时将停止输出。匹配到的词本身不会被输出。最多允许 5 个字符串，每个不超过 32 字节 + stream: + type: boolean + default: false + description: 是否以流式方式返回响应，默认 false + stream_options: + type: object + description: 流式响应选项 + properties: + include_usage: + type: boolean + default: false + description: >- + 如果设置，将在 data: [DONE] 消息之前额外发送一个 chunk。该 chunk 的 usage 字段显示整个请求的 + Token 使用统计，choices 字段为空数组。其他所有 chunk 也会包含 usage 字段，但值为 + null。注意：如果流中断，可能无法收到包含总 Token 用量的最终 chunk + tools: + type: array + description: 模型可调用的工具列表 + items: + $ref: '#/components/schemas/ToolDefinition' + prompt_cache_key: + type: string + default: null + description: >- + 用于缓存相似请求的响应以优化缓存命中率。对于 Coding Agent，通常是代表单个会话的 session id 或 task + id；退出并恢复会话时应保持不变。对于 Kimi Code Plan，此字段为必填以提高缓存命中率。对于其他多轮对话 + Agent，也建议使用此字段 + safety_identifier: + type: string + description: 用于检测可能违反使用政策的用户的稳定标识符。应为唯一标识每个用户的字符串。建议对用户名或邮箱进行哈希处理以避免发送可识别信息 + tool_choice: + oneOf: + - type: string + enum: + - auto + - none + - required + description: auto：模型自行决定是否调用工具；none：不调用工具；required：强制调用工具 + - type: object + description: 强制调用指定工具 + properties: + type: + type: string + enum: + - function + function: + type: object + properties: + name: + type: string + description: 要调用的函数名称 + required: + - name + required: + - type + - function + description: >- + 控制模型是否调用工具。`auto`（默认）：模型自行决定是否调用工具；`none`：不调用工具；`required`：强制调用工具；也可传入特定函数对象强制调用指定工具。 + required: + - messages + ChoiceDelta: + type: object + properties: + index: + type: integer + description: 选项索引 + delta: + type: object + description: 增量内容对象 + properties: + role: + type: string + description: 消息角色（仅在首个 chunk 中出现） + content: + type: string + description: 消息内容片段 + tool_calls: + type: array + description: 模型发起的工具调用片段 + items: + type: object + properties: + id: + type: string + type: + type: string + enum: + - function + function: + type: object + properties: + name: + type: string + arguments: + type: string + description: 函数参数的 JSON 字符串 + finish_reason: + type: + - string + - 'null' + enum: + - stop + - length + - tool_calls + - null + description: 停止原因，仅在最后一个 chunk 中出现 + usage: + type: object + description: Token 使用统计（可选） + properties: + prompt_tokens: + type: integer + description: 提示中的 Token 数量 + completion_tokens: + type: integer + description: 补全中的 Token 数量 + total_tokens: + type: integer + description: 使用的总 Token 数量 + ToolDefinition: + type: object + properties: + type: + type: string + enum: + - function + function: + type: object + properties: + name: + type: string + description: 函数名称。必须符合正则表达式：^[a-zA-Z_][a-zA-Z0-9-_]{2,63}$ + pattern: ^[a-zA-Z_][a-zA-Z0-9-_]{2,63}$ + description: + type: string + description: 函数功能描述 + parameters: + type: object + description: >- + 函数参数，JSON Schema 格式。需符合 [MFJS（Moonshot Flavored JSON + Schema）规范](https://github.com/MoonshotAI/walle/blob/main/docs/mfjs-spec.zh.md)。 + additionalProperties: true + strict: + type: boolean + default: true + description: >- + 是否严格按 parameters schema 约束工具调用参数的输出。默认为 true。设为 false 时仅保证输出为合法 + JSON 对象，不强制约束内部结构。 + required: + - name + - parameters + required: + - type + - function + Message: + type: object + properties: + role: + type: string + enum: + - system + - user + - assistant + - tool + example: user + description: 消息发送者的角色，支持 system、user、assistant、tool + content: + oneOf: + - type: string + - type: array + items: + oneOf: + - title: text + type: object + properties: + type: + type: string + enum: + - text + text: + type: string + required: + - type + - text + - title: image_url + type: object + properties: + type: + type: string + enum: + - image_url + image_url: + oneOf: + - type: object + properties: + url: + type: string + required: + - url + - type: string + required: + - type + - image_url + - title: video_url + type: object + properties: + type: + type: string + enum: + - video_url + video_url: + oneOf: + - type: object + properties: + url: + type: string + required: + - url + - type: string + required: + - type + - video_url + example: 你好 + description: 消息内容。可以是纯文本字符串，也可以是包含 text/image_url/video_url 类型的对象数组（用于多模态输入） + name: + type: string + default: null + description: 消息发送者的名称（可选） + partial: + type: boolean + default: false + description: 在最后一条 assistant 消息中设置为 true 以启用 Partial Mode + required: + - role + - content + KimiK3DynamicToolMessage: + type: object + description: >- + Kimi K3 动态加载工具消息。该消息只能使用 system 角色，通过 tools 字段在当前对话位置声明后续对话可用的工具，且不包含 + content 字段。 + properties: + role: + type: string + enum: + - system + description: 动态加载工具消息的角色，固定为 system + tools: + type: array + description: 从该消息位置开始可供模型调用的工具列表 + items: + $ref: '#/components/schemas/ToolDefinition' + required: + - role + - tools + additionalProperties: false + securitySchemes: + bearerAuth: + type: http + scheme: bearer + description: >- + Authorization 请求头需要一个 Bearer 令牌。使用 MOONSHOT_API_KEY 作为令牌。这是一个服务端密钥，请在 + [API 密钥页面](https://platform.kimi.com/console/api-keys) 生成。 + +```` \ No newline at end of file diff --git a/llmsdk_docs/kimi_k3/docs/context-caching.md b/llmsdk_docs/kimi_k3/docs/context-caching.md new file mode 100644 index 00000000..a87edecd --- /dev/null +++ b/llmsdk_docs/kimi_k3/docs/context-caching.md @@ -0,0 +1,52 @@ +> Fetch the complete documentation index at: https://platform.kimi.com/docs/llms.txt +> Use this file to discover all available pages before exploring further. + +# 使用 Kimi API 的 Context Caching 功能 + +Context Caching（上下文缓存）会预先存储可能被频繁请求的大量数据；再次请求相同信息时，系统直接从缓存提供，无需重新计算或从原始数据源检索，从而节省时间和资源。在 Kimi API 中，Context Caching 对所有模型请求自动启用：当系统检测到重复的初始上下文（如 system prompt、知识文档、工具定义等）时，会自动复用已缓存的内容，为你带来成本优化和响应加速，无需手动创建或管理缓存。 + +## 频繁请求固定长上下文时使用 + +Context Caching 特别适合频繁请求、重复引用大量初始上下文的场景，例如： + +* 提供大量预设内容的 QA Bot，例如产品文档问答助手。 +* 针对固定文档集合的频繁查询，例如上市公司信息披露问答工具。 +* 对静态代码库或知识库的周期性分析，例如各类 Copilot Agent。 +* 瞬时流量巨大的爆款 AI 应用。 +* 交互规则复杂的 Agent 类应用。 + +## Context Caching 与 RAG 怎么选 + +业界广泛采用 RAG（检索增强生成）方案进行长文本业务的降本。Context Caching 的降本幅度与业务特性高度相关，RAG 则与业务特性无关；两者的主要区别如下： + +| 维度 | Context Caching | RAG | +| ---- | -------------------------- | -------------------------------------- | +| 业务成本 | 特定场景下成本压缩程度极高，最高可降本 90% | 任何业务均可降本，但召回精度问题可能导致回答准确率下降 | +| 研发成本 | 相对较低，系统自动处理缓存，无需额外接入或调优 | 相对较高，需 RAG 与 Embedding 结合，并持续进行业务定制化调优 | +| 额外优势 | 长文本场景下首 Token 延迟平均可降至 5s 内 | 原始文本长度可扩展到非常长，适合一次性数百万字上下文的场景 | + +> **建议**：频繁查询固定内容（如 FAQ、文档问答）时优先使用 Context Caching；内容极长且查询方向不固定时，可考虑 RAG 方案。 + +## 无需配置，缓存自动命中 + +Context Caching 采用全自动缓存机制，你只需像平常一样调用 API： + +* **无需手动创建**：系统会自动识别并缓存高频使用的初始上下文。 +* **无需引用缓存 ID**：调用 `/v1/chat/completions` 时按正常方式传入 messages 即可，系统会在后台自动匹配缓存。 +* **无需管理 TTL**：缓存的生命周期由系统自动管理，无需人工干预。 + +系统会在合适的时机自动触发缓存优化。 + + + 当前一个请求的 prompt tokens 大于 256 时，新的请求才能命中前缀缓存；当前一个请求的 prompt tokens 小于 256 时，请求不会被缓存而是被丢弃。 + + +## 计费 + +Context Caching 的计费方式与具体价格，请参阅[产品定价页面的计费说明](/docs/pricing/chat#计费逻辑)。 + +## 注意事项 + +* **缓存命中条件**：系统会自动对高频重复的初始上下文进行缓存优化。请确保你的知识内容、system prompt 和工具定义相对稳定，以获得更好的缓存命中率。 +* **多轮对话**：将固定的大段上下文（如知识文档）放在 `messages` 数组的最前面（system 消息之前），然后将用户问题和模型回复追加其后，系统会自动识别并缓存这些固定内容。 +* **无需额外配置**：Context Caching 对所有请求自动生效，无需修改 API 调用方式或添加额外参数，只需关注 prompt 设计和业务逻辑即可。 diff --git a/llmsdk_docs/kimi_k3/docs/models-overview.md b/llmsdk_docs/kimi_k3/docs/models-overview.md new file mode 100644 index 00000000..1716b934 --- /dev/null +++ b/llmsdk_docs/kimi_k3/docs/models-overview.md @@ -0,0 +1,160 @@ +> Fetch the complete documentation index at: https://platform.kimi.com/docs/llms.txt +> Use this file to discover all available pages before exploring further. + +# 模型参数参考 + +export const DocTable = ({columns = [], rows = []}) => { + return
+ + {columns.length > 0 ? + {columns.map((column, index) => )} + : null} + + + {columns.map((column, index) => )} + + + + {rows.map((row, rowIndex) => + {row.map((cell, cellIndex) => )} + )} + +
{column.title}
{cell}
+
; +}; + +不同模型系列对 Chat Completions API 参数有不同的默认值和约束。完整的模型列表请参阅[模型列表](/docs/models)。 + +## 参数对比 + +temperature, 不可修改, 不可修改, 不可修改, "0.0"], +[top_p, <>0.95 不可改, <>0.95 不可改, <>0.95 不可改, "1.0"], +[n, <>1 不可改, <>1 不可改, <>1 不可改, "1（最大 5）"], +[presence_penalty, <>0 不可改, <>0 不可改, <>0 不可改, "0（可修改）"], +[frequency_penalty, <>0 不可改, <>0 不可改, <>0 不可改, "0（可修改）"], +[<>推理配置, reasoning_effort, thinking, thinking, "—"], +]} +/> + + + 当 `temperature` 接近 0 时，`n` 只能为 1，否则将返回 `invalid_request_error`。 + + +## 模型参数配置差异 + +切换模型时，除了替换 `model` 字段，还需要注意各模型对请求参数的支持范围和默认值不同： + +| 参数 | `kimi-k3` | `kimi-k2.7-code` | `kimi-k2.6` | `kimi-k2.5` | +| ---------------------------------------- | ---------------------------------------- | ---------------------------------------------- | -------------------------------------------------------------------------------- | ---------------------------------------------- | +| 上下文窗口 | 1M tokens | 256K tokens | 256K tokens | 256K tokens | +| `thinking` | — | 可省略；显式设置时仅接受 `{"type":"enabled","keep":"all"}` | `{"type":"enabled"}`（默认）、`{"type":"disabled"}`、`{"type":"enabled","keep":"all"}` | `{"type":"enabled"}`（默认）、`{"type":"disabled"}` | +| `reasoning_effort` | `"low"` / `"high"` / `"max"`（默认 `"max"`） | 不支持 | 不支持 | 不支持 | +| `tool_choice` | `auto` / `none` / `required` | 不支持 `required` | 不支持 `required` | — | +| `temperature` | 固定 `1.0` | 固定 `1.0` | 思考 `1.0` / 非思考 `0.6` | 思考 `1.0` / 非思考 `0.6` | +| `top_p` | 固定 `0.95` | 固定 `0.95` | 固定 `0.95` | — | +| `n` | 固定 `1` | 固定 `1` | 固定 `1` | — | +| `presence_penalty` / `frequency_penalty` | 固定 `0` | 固定 `0` | 固定 `0` | — | + + + 表中"固定"表示该参数不可修改：传入其他值会报错，建议不要显式传入。 + + +### `thinking` + +`thinking` 是 K2.x 专属请求参数： + +* `kimi-k2.6`：支持 `{"type": "enabled"}`（默认）、`{"type": "disabled"}`、`{"type": "enabled", "keep": "all"}` 三种配置。 +* `kimi-k2.7-code`：思考默认开启，仅支持 `{"type": "enabled", "keep": "all"}`，传入其他配置会报错。从 `kimi-k2.6` 切换时，需要按 Preserved Thinking 的要求在 `messages` 中回传历史 `reasoning_content`。 + +详见[使用思考模式](/docs/guide/use-kimi-k2-thinking-model)。 + +### `reasoning_effort` + +K3 始终进行推理思考且保留式思考（Preserved Thinking）始终开启。通过请求顶层 `reasoning_effort` 配置推理力度，支持 `"low"` / `"high"` / `"max"` 三档，默认 `"max"`。详见[思考力度](/docs/guide/use-thinking-effort)。 + + + 切换档位会破坏前缀缓存命中，建议在会话开始前确定 `effort` 档位，避免中途切换。 + + +### `tool_choice` + +`kimi-k3` 支持 `auto` / `none` / `required` 三档；`kimi-k2.6` 与 `kimi-k2.7-code` 不支持 `required`，传入会报错。详见[工具调用约束](/docs/guide/use-tool-choice)。 + +### `temperature` + +* `kimi-k2.6` / `kimi-k2.5`：思考模式固定 `1.0`，非思考模式固定 `0.6`，传入其他值报错； +* `kimi-k2.7-code`：固定 `1.0`，传入其他值报错。 +* `kimi-k3`：固定 `1.0`，传入其他值报错。 + +建议调用以上模型时不要显式传入 `temperature`。 + +`kimi-k2.7-code-highspeed` 与 `kimi-k2.7-code` 为同一模型、参数约束完全一致，仅输出速度不同。 + +### 常见问题 + +**从 `kimi-k2.6` 切换到 `kimi-k3`，需要改代码吗？** + +将 `model` 替换为 `kimi-k3`，并移除 K2.x 的 `thinking` 配置；如需显式设置推理力度，使用顶层 `reasoning_effort`。K3 的多轮对话和工具调用需要把 API 返回的完整 assistant message 原样回传到 `messages`，包括可能返回的 `reasoning_content`。 + +**从 `kimi-k2.7-code` 切换到 `kimi-k3`，需要改代码吗？** + +替换 `model` 即可，并继续原样回传完整 assistant message；如需显式设置推理力度，使用顶层 `reasoning_effort`。 + +**原来代码里用的是 OpenAI 的 `reasoning_effort`，切到 `kimi-k3` 需要改吗？** + +不需要。K3 支持顶层 `reasoning_effort`，可选值为 `"low"` / `"high"` / `"max"`，默认 `"max"`。 + +**`tool_choice: "required"` 在 `kimi-k2.6` / `kimi-k2.7-code` 上能用吗？** + +不能。这两个模型不支持 `required`，传入会报错；该档位仅 `kimi-k3` 支持。 + +## Kimi K2.7 Code 系列 — thinking 参数 + +`kimi-k2.7-code` 系列包含 `kimi-k2.7-code` 及其高速版 `kimi-k2.7-code-highspeed`，二者为同一模型、参数约束完全一致（含上方表格与 `thinking` 行为），仅输出速度不同，下文统称 `kimi-k2.7-code`。 + +`kimi-k2.7-code` 面向代码场景，除 `thinking` 外的参数约束与 `kimi-k2.6` 完全一致。与 `kimi-k2.6` 不同的是，它 **始终开启思考、不可禁用**（传入 `{"type": "disabled"}` 会报错），且 **Preserved Thinking 始终开启**（`thinking.keep` 不传或传 `"all"` 都按 `"all"` 处理，传入其他非法值会报错）。因此调用时无需传入 `thinking` 参数，只需切换 `model` 即可，模型始终输出 `reasoning_content`。详细用法见[使用思考模式](/docs/guide/use-kimi-k2-thinking-model)。 + +## Kimi K2.6 — thinking 参数 + +Kimi K2.6 支持通过 `thinking` 参数控制是否启用深度思考。接受 `{"type": "enabled"}` 或 `{"type": "disabled"}`。 + +由于 OpenAI SDK 没有原生的 `thinking` 参数，需要使用 `extra_body` 传递： + + + ```python Python theme={null} + completion = client.chat.completions.create( + model="kimi-k2.6", + messages=[ + {"role": "user", "content": "你好"} + ], + extra_body={ + "thinking": {"type": "disabled"} + }, + max_tokens=1024*32, + ) + ``` + + ```bash cURL theme={null} + curl https://api.moonshot.cn/v1/chat/completions \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $MOONSHOT_API_KEY" \ + -d '{ + "model": "kimi-k2.6", + "messages": [ + {"role": "user", "content": "你好"} + ], + "thinking": {"type": "disabled"} + }' + ``` + diff --git a/llmsdk_docs/kimi_k3/docs/pricing.md b/llmsdk_docs/kimi_k3/docs/pricing.md new file mode 100644 index 00000000..2a60bafc --- /dev/null +++ b/llmsdk_docs/kimi_k3/docs/pricing.md @@ -0,0 +1,55 @@ +> Fetch the complete documentation index at: https://platform.kimi.com/docs/llms.txt +> Use this file to discover all available pages before exploring further. + +# 旗舰模型 Kimi K3 定价 + +export const DocTable = ({columns = [], rows = []}) => { + return
+ + {columns.length > 0 ? + {columns.map((column, index) => )} + : null} + + + {columns.map((column, index) => )} + + + + {rows.map((row, rowIndex) => + {row.map((cell, cellIndex) => )} + )} + +
{column.title}
{cell}
+
; +}; + +## 产品定价 + + + +此处 1M = 1,000,000，表格中的价格代表每消耗 1M tokens 的价格。 + +## 模型说明 + + + 联网搜索（`web_search`）正在更新升级中，近期不建议使用该功能，当前文档已经过时，请关注后续内容更新。 + + +* Kimi K3 是 Kimi 的旗舰模型，面向长程编程与端到端知识工作，1M token 上下文，综合智能达到领先水平，详见 [Kimi K3 模型介绍](/docs/guide/kimi-k3-quickstart) +* 始终进行推理，支持通过请求顶层 `reasoning_effort` 配置推理力度（`low` / `high` / `max`，默认 `max`），详见[思考力度](/docs/guide/use-thinking-effort) +* 支持[自动上下文缓存](/docs/guide/use-context-caching-feature-of-kimi-api)、[工具调用（ToolCalls）](/docs/guide/use-kimi-api-to-complete-tool-calls)、[JSON Mode](/docs/guide/use-json-mode-feature-of-kimi-api)、[结构化输出（`response_format` / JSON Schema）](/docs/guide/response_format)、[Partial Mode](/docs/guide/use-partial-mode-feature-of-kimi-api)、[联网搜索](/docs/guide/use-web-search)等能力 +* K3 新增 API 能力：[工具调用约束（`tool_choice`）](/docs/guide/use-tool-choice)、[动态加载工具](/docs/guide/use-dynamic-tool-loading)，组合用法见 [K3 工具调用最佳实践](/docs/guide/kimi-k3-tool-calling-best-practice) diff --git a/llmsdk_docs/kimi_k3/docs/streaming.md b/llmsdk_docs/kimi_k3/docs/streaming.md new file mode 100644 index 00000000..9fd55032 --- /dev/null +++ b/llmsdk_docs/kimi_k3/docs/streaming.md @@ -0,0 +1,389 @@ +> Fetch the complete documentation index at: https://platform.kimi.com/docs/llms.txt +> Use this file to discover all available pages before exploring further. + +# 使用 Kimi API 的流式输出功能 + +Kimi 大模型收到问题后会先进行推理，再逐个 Token 生成回答；流式输出（Streaming）让模型每生成一定数量的 Tokens（通常是 1 个 Token）就立即发送给客户端，而不是等全部生成完毕再一次性返回。等待完整回复通常要数秒，问题复杂、回复较长时可能拉长到 10 秒甚至 20 秒；开启流式输出后，用户能第一时间看到第一个 Token，显著减少等待时间。当你与 [Kimi 智能助手](https://kimi.com) 对话时，回复逐字“跳”出来，就是流式输出的效果。 + +## 开启流式输出 + +在请求中设置 `stream=True` 即可开启流式输出。此时 SDK 返回一个可迭代对象，用循环逐个读取数据块（chunk）：每个 chunk 的结构与 completion 相似，但 `message` 字段被替换为 `delta` 字段。 + + + 本页示例默认使用最新模型 `kimi-k3`。K3 使用请求顶层 `reasoning_effort` 配置思考力度（支持 `"low"` / `"high"` / `"max"`，默认 `"max"`）。换用 `kimi-k2.6`、`kimi-k2.5` 等其他模型时，只需替换 `model` 字段，但各模型的参数配置存在差异，详见[模型参数参考](/docs/api/models-overview)。 + + + + + ```python theme={null} + import os + from openai import OpenAI + + client = OpenAI( + api_key = os.environ["MOONSHOT_API_KEY"], # 运行前请设置 MOONSHOT_API_KEY 环境变量 + base_url = "https://api.moonshot.cn/v1", + ) + + stream = client.chat.completions.create( + model = "kimi-k3", + messages = [ + {"role": "system", "content": "你是 Kimi，由 Moonshot AI 提供的人工智能助手，你更擅长中文和英文的对话。你会为用户提供安全，有帮助，准确的回答。同时，你会拒绝一切涉及恐怖主义，种族歧视，黄色暴力等问题的回答。Moonshot AI 为专有名词，不可翻译成其他语言。"}, + {"role": "user", "content": "你好，我叫李雷，1+1等于多少？"} + ], + stream=True, # <-- 注意这里，我们通过设置 stream=True 开启流式输出模式 + ) + + # 当启用流式输出模式（stream=True），SDK 返回的内容也发生了变化，我们不再直接访问返回值中的 choice + # 而是通过 for 循环逐个访问返回值中每个单独的块（chunk） + + for chunk in stream: + # 在这里，每个 chunk 的结构都与之前的 completion 相似，但 message 字段被替换成了 delta 字段 + delta = chunk.choices[0].delta # <-- message 字段被替换成了 delta 字段 + + if delta.content: + # 我们在打印内容时，由于是流式输出，为了保证句子的连贯性，我们不人为地添加 + # 换行符，因此通过设置 end="" 来取消 print 自带的换行符。 + print(delta.content, end="") + ``` + + + + ```js theme={null} + const OpenAI = require('openai') + + const client = new OpenAI({ + apiKey: process.env.MOONSHOT_API_KEY, // 运行前请设置 MOONSHOT_API_KEY 环境变量 + baseURL: "https://api.moonshot.cn/v1", + }) + + async function main() { + const stream = await client.chat.completions.create({ + model: "kimi-k3", + messages: [ + {role: "system", content: "你是 Kimi，由 Moonshot AI 提供的人工智能助手，你更擅长中文和英文的对话。你会为用户提供安全，有帮助，准确的回答。同时，你会拒绝一切涉及恐怖主义，种族歧视，黄色暴力等问题的回答。Moonshot AI 为专有名词，不可翻译成其他语言。"}, + {role: "user", content: "你好，我叫李雷，1+1等于多少？"} + ], + stream: true, // <-- 注意这里，我们通过设置 stream=True 开启流式输出模式 + }) + + // 当启用流式输出模式（stream=True），SDK 返回的内容也发生了变化，我们不再直接访问返回值中的 choice + // 而是通过 for 循环逐个访问返回值中每个单独的块（chunk） + + for await (chunk of stream) { + // 在这里，每个 chunk 的结构都与之前的 completion 相似，但 message 字段被替换成了 delta 字段 + delta = chunk.choices[0].delta // <-- message 字段被替换成了 delta 字段 + + if (delta.content) { + // 我们在打印内容时，由于是流式输出，为了保证句子的连贯性，我们不人为地添加 + // 换行符，因此通过设置 end="" 来取消 print 自带的换行符。 + console.log(delta.content, end="") + } + } + } + + main() + ``` + + + +## 解析 SSE 响应体 + +开启流式输出后，接口不再返回 JSON 格式的响应（`Content-Type: application/json`），而是返回 `Content-Type: text/event-stream`（SSE），服务端得以源源不断地向客户端传输 Tokens。[SSE](https://kimi.com/share/cr7boh3dqn37a5q9tds0) 的响应体如下所示： + +```text theme={null} +data: {"id":"cmpl-1305b94c570f447fbde3180560736287","object":"chat.completion.chunk","created":1698999575,"model":"kimi-k3","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]} + +data: {"id":"cmpl-1305b94c570f447fbde3180560736287","object":"chat.completion.chunk","created":1698999575,"model":"kimi-k3","choices":[{"index":0,"delta":{"content":"你好"},"finish_reason":null}]} + +... + +data: {"id":"cmpl-1305b94c570f447fbde3180560736287","object":"chat.completion.chunk","created":1698999575,"model":"kimi-k3","choices":[{"index":0,"delta":{"content":"。"},"finish_reason":null}]} + +data: {"id":"cmpl-1305b94c570f447fbde3180560736287","object":"chat.completion.chunk","created":1698999575,"model":"kimi-k3","choices":[{"index":0,"delta":{},"finish_reason":"stop","usage":{"prompt_tokens":19,"completion_tokens":13,"total_tokens":32}}]} + +data: [DONE] +``` + +响应体中的每个数据块均以 `data: ` 为前缀，紧跟一个合法的 JSON 对象，并以两个换行符 `\n\n` 结束。所有数据块传输完成后，服务端发送 `data: [DONE]` 标识传输结束，此时可断开网络连接。 + +*注意：请始终使用 `data: [DONE]` 判断数据是否传输完成，而不是使用 `finish_reason` 或其他方式。如果未收到 `data: [DONE]`，即使已经获取了 `finish_reason=stop`，也不应视作传输完成；换句话说，在收到 `data: [DONE]` 之前，都应视作 **消息是不完整的**。* + +流式输出过程中会有 `content` 字段会逐块下发；`role` 和 `usage` 不会在每个数据块中重复出现——`role` 仅出现在第一个数据块，`usage` 仅出现在最后一个数据块。 + +## 统计 Tokens 用量 + +计算 Tokens 有两种方式。最直接、最准确的一种，是等所有数据块传输完毕后，读取最后一个数据块中的 `usage` 字段，查看本次请求产生的 `prompt_tokens`/`completion_tokens`/`total_tokens`： + +```text theme={null} +... + +data: {"id":"cmpl-1305b94c570f447fbde3180560736287","object":"chat.completion.chunk","created":1698999575,"model":"kimi-k3","choices":[{"index":0,"delta":{},"finish_reason":"stop","usage":{"prompt_tokens":19,"completion_tokens":13,"total_tokens":32}}]} + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + 通过访问最后一个数据块中的 usage 字段来查看当前请求产生的 Tokens 数量 +data: [DONE] +``` + + + 注意 `usage` 嵌套在最后一个数据块的 `choices[0]` 内（即 `choices[0].usage`），而非数据块顶层。使用 OpenAI SDK 时 `chunk.usage` 为 `None`，请读取 `chunk.choices[0].usage`，或自行解析原始 SSE 数据块。 + + +但流式输出可能因网络连接中断、客户端程序错误等不可控因素被打断，此时最后一个数据块尚未到达，也就无从得知本次请求消耗的 Tokens。为避免统计失败，建议保存已收到的每个数据块的内容，并在请求结束后（无论是否成功结束）调用 Tokens 计算接口统计实际消耗量： + + + + ```python theme={null} + import os + import httpx + from openai import OpenAI + + client = OpenAI( + api_key = os.environ["MOONSHOT_API_KEY"], # 运行前请设置 MOONSHOT_API_KEY 环境变量 + base_url = "https://api.moonshot.cn/v1", + ) + + stream = client.chat.completions.create( + model = "kimi-k3", + messages = [ + {"role": "system", "content": "你是 Kimi，由 Moonshot AI 提供的人工智能助手，你更擅长中文和英文的对话。你会为用户提供安全，有帮助，准确的回答。同时，你会拒绝一切涉及恐怖主义，种族歧视，黄色暴力等问题的回答。Moonshot AI 为专有名词，不可翻译成其他语言。"}, + {"role": "user", "content": "你好，我叫李雷，1+1等于多少？"} + ], + stream=True, # <-- 注意这里，我们通过设置 stream=True 开启流式输出模式 + ) + + + def estimate_token_count(input: str) -> int: + """ + 在这里实现你的 Tokens 计算逻辑，或是直接调用我们的 Tokens 计算接口计算 Tokens + + https://api.moonshot.cn/v1/tokenizers/estimate-token-count + """ + header = { + "Authorization": f"Bearer {os.environ['MOONSHOT_API_KEY']}", + } + data = { + "model": "kimi-k3", + "messages": [ + {"role": "user", "content": input}, + ] + } + r = httpx.post("https://api.moonshot.cn/v1/tokenizers/estimate-token-count", headers=header, json=data) + r.raise_for_status() + return r.json()["data"]["total_tokens"] + + + completion = [] + for chunk in stream: + delta = chunk.choices[0].delta + if delta.content: + completion.append(delta.content) + + + print("completion_tokens:", estimate_token_count("".join(completion))) + ``` + + + + ```js theme={null} + const axios = require('axios'); + const OpenAI = require('openai'); + + client = new OpenAI({ + apiKey: process.env.MOONSHOT_API_KEY, + baseURL: "https://api.moonshot.cn/v1", + }) + + + async function estimate_token_count(input_messages) { + /* + 在这里实现你的 Tokens 计算逻辑，或是直接调用我们的 Tokens 计算接口计算 Tokens + + https://api.moonshot.cn/v1/tokenizers/estimate-token-count + */ + header = { + "Authorization": `Bearer ${process.env.MOONSHOT_API_KEY}`, + } + data = { + "model": "kimi-k3", + "messages": input_messages, + } + r = await axios.post("https://api.moonshot.cn/v1/tokenizers/estimate-token-count", data, {headers: header}) + .catch(function (error) { + console.log(error) + }) + return r.data.data.total_tokens + } + + async function main() { + + const stream = await client.chat.completions.create({ + model: "kimi-k3", + messages: [ + {role: "system", content: "你是 Kimi，由 Moonshot AI 提供的人工智能助手，你更擅长中文和英文的对话。你会为用户提供安全，有帮助，准确的回答。同时，你会拒绝一切涉及恐怖主义，种族歧视，黄色暴力等问题的回答。Moonshot AI 为专有名词，不可翻译成其他语言。"}, + {role: "user", content: "你好，我叫李雷，1+1等于多少？"} + ], + stream: true, // <-- 注意这里，我们通过设置 stream=True 开启流式输出模式 + }) + + const completion = []; + for await (chunk of stream) { + const delta = chunk.choices[0].delta + if (delta.content) { + completion.push(delta.content) + } + } + + console.log("completion_tokens:", await estimate_token_count(completion.join(""))) + } + + main() + ``` + + + +## 终止流式输出 + +需要提前终止输出时，直接关闭 HTTP 网络连接或丢弃后续数据块即可，例如在循环中 `break`： + +```python theme={null} +for chunk in stream: + if condition: + break +``` + +## 不用 SDK 直接处理 SSE + +在没有 SDK 的语言环境，或 SDK 无法满足你的业务逻辑时，可以直接对接 HTTP 接口来处理流式输出。以下示例演示如何逐行读取并解析 [SSE](https://kimi.com/share/cr7boh3dqn37a5q9tds0) 响应体，详细说明见代码注释： + + + + ```python theme={null} + import os + import json + import httpx # 我们使用 httpx 库来执行我们的 HTTP 请求 + + + data = { + "model": "kimi-k3", + "messages": [ + # 具体的 messages + ], + "stream": True, + } + + + # 使用 httpx 向 Kimi 大模型发出 chat 请求，并获得响应 r + r = httpx.post("https://api.moonshot.cn/v1/chat/completions", headers={"Authorization": f"Bearer {os.environ['MOONSHOT_API_KEY']}"}, json=data) + if r.status_code != 200: + raise Exception(r.text) + + + data: str + + # 在这里，我们使用了 iter_lines 方法来逐行读取响应体 + for line in r.iter_lines(): + # 去除每一行收尾的空格，以便更好地处理数据块 + line = line.strip() + + # 接下来我们要处理三种不同的情况： + # 1. 如果当前行是空行，则表明前一个数据块已接收完毕（即前文提到的，通过两个换行符结束数据块传输），我们可以对该数据块进行反序列化，并打印出对应的 content 内容； + # 2. 如果当前行为非空行，且以 data: 开头，则表明这是一个数据块传输的开始，我们去除 data: 前缀后，首先判断是否是结束符 [DONE]，如果不是，将数据内容保存到 data 变量； + # 3. 如果当前行为非空行，但不以 data: 开头，则表明当前行仍然归属上一个正在传输的数据块，我们将当前行的内容追加到 data 变量尾部； + + if len(line) == 0: + chunk = json.loads(data) + + # 这里的处理逻辑可以替换成你的业务逻辑，打印仅是为了展示处理流程 + choice = chunk["choices"][0] + usage = choice.get("usage") + if usage: + print("total_tokens:", usage["total_tokens"]) + delta = choice["delta"] + role = delta.get("role") + if role: + print("role:", role) + content = delta.get("content") + if content: + print(content, end="") + + data = "" # 重置 data + elif line.startswith("data: "): + data = line.lstrip("data: ") + + # 当数据块内容为 [DONE] 时，则表明所有数据块已发送完毕，可断开网络连接 + if data == "[DONE]": + break + else: + data = data + "\n" + line # 我们仍然在追加内容时，为其添加一个换行符，因为这可能是该数据块有意将数据分行展示 + ``` + + + + ```js theme={null} + const axios = require('axios'); // 使用 axios 库来执行 HTTP 请求 + + let data = { + "model": "kimi-k3", + "messages": [ + // 具体的 messages + ], + "stream": true, + }; + + // 使用 axios 向 Kimi 大模型发出 chat 请求，并获得响应 r + axios.post("https://api.moonshot.cn/v1/chat/completions", data, { + responseType: 'stream' + }).then(response => { + let data = ''; + response.data.on('data', chunk => { + // 去除每一行收尾的空格，以便更好地处理数据块 + let line = chunk.toString().trim(); + + if (line === '') { + try { + let chunk = JSON.parse(data); + let choice = chunk.choices[0]; + let usage = choice.usage; + if (usage) { + console.log("total_tokens:", usage.total_tokens); + } + let delta = choice.delta; + let role = delta.role; + if (role) { + console.log("role:", role); + } + let content = delta.content; + if (content) { + console.log(content); + } + } catch (error) { + console.error("Error parsing JSON:", error); + } + data = ''; // 重置 data + } else if (line.startsWith('data: ')) { + data = line.substring(6); + if (data === '[DONE]') { + response.data.destroy(); + } + } else { + data += '\n' + line; + } + }); + }).catch(error => { + console.error("Error in request:", error); + }); + ``` + + + +无论使用哪种语言，处理流式输出的基本步骤相同： + +1. 发起 HTTP 请求，并在请求体中将 `stream` 参数设置为 `true`； +2. 检查响应 `Headers` 中的 `Content-Type`，为 `text/event-stream` 即表示当前响应是流式输出； +3. 逐行读取响应内容并解析数据块（JSON 格式），通过 `data: ` 前缀和换行符 `\n` 判断数据块的起止位置； +4. 数据块内容为 `[DONE]` 时表示传输完成。 + +## 多个回复（`n` 参数） + + + 当前模型（`kimi-k3`、`kimi-k2.7-code`、`kimi-k2.6`）的 `n` 固定为 `1`，暂不支持一次请求返回多个回复；传入大于 1 的 `n` 会返回 400 错误（`invalid n: only 1 is allowed for this model`），流式与非流式请求均如此。各模型的参数约束详见[模型参数参考](/docs/api/models-overview)。 + diff --git a/llmsdk_docs/kimi_k3/docs/thinking-effort.md b/llmsdk_docs/kimi_k3/docs/thinking-effort.md new file mode 100644 index 00000000..d838268a --- /dev/null +++ b/llmsdk_docs/kimi_k3/docs/thinking-effort.md @@ -0,0 +1,79 @@ +> Fetch the complete documentation index at: https://platform.kimi.com/docs/llms.txt +> Use this file to discover all available pages before exploring further. + +# 思考力度 + +Kimi K3 始终进行推理，并通过请求顶层 `reasoning_effort` 配置 **推理力度**。该字段支持 `"low"` / `"high"` / `"max"` 三档，默认 `"max"`。 + +## 设置推理力度 + +在 Chat Completions 请求顶层设置 `reasoning_effort`： + +```json theme={null} +{ + "model": "kimi-k3", + "messages": [{"role": "user", "content": "请推导一下这个数列的通项公式：1, 4, 9, 25, 64, ..."}], + "reasoning_effort": "high" +} +``` + +从 K2.x 迁移到 K3 时，移除 K2.x 的 `thinking` 配置，并按需使用顶层 `reasoning_effort`。 + + + + ```bash theme={null} + $ curl https://api.moonshot.cn/v1/chat/completions \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $MOONSHOT_API_KEY" \ + -d '{ + "model": "kimi-k3", + "messages": [ + { + "role": "user", + "content": "请推导一下这个数列的通项公式：1, 4, 9, 25, 64, ..." + } + ], + "reasoning_effort": "high" + }' + ``` + + + + ```python theme={null} + import os + from openai import OpenAI + + client = OpenAI( + api_key=os.environ["MOONSHOT_API_KEY"], + base_url="https://api.moonshot.cn/v1", + ) + + completion = client.chat.completions.create( + model="kimi-k3", + messages=[ + {"role": "user", "content": "请推导一下这个数列的通项公式：1, 4, 9, 25, 64, ..."}, + ], + reasoning_effort="high", + ) + + message = completion.choices[0].message + if hasattr(message, "reasoning_content"): + print(getattr(message, "reasoning_content")) + print(message.content) + ``` + + + +## 字段说明 + +| 字段 | 类型 | 必填 | 说明 | +| ------------------ | ------ | -- | -------------------------------------------------------- | +| `reasoning_effort` | string | 否 | K3 的顶层推理力度字段，支持 `"low"` / `"high"` / `"max"`，默认 `"max"`。 | + +K3 的多轮对话和工具调用必须将 API 返回的完整 assistant message 原样回传到 `messages`，包括 `reasoning_content` 和 `tool_calls`。 + +## 相关阅读 + +* [Kimi K3 API 工具调用最佳实践](/docs/guide/kimi-k3-tool-calling-best-practice)：工具调用场景中的思考力度配置建议 +* [使用思考模式](/docs/guide/use-kimi-k2-thinking-model)：各模型的思考行为与保留式思考（Preserved Thinking） +* [模型参数参考](/docs/api/models-overview)：各模型的参数配置差异 diff --git a/llmsdk_docs/kimi_k3/docs/tool-calling-best-practice.md b/llmsdk_docs/kimi_k3/docs/tool-calling-best-practice.md new file mode 100644 index 00000000..0213ba5f --- /dev/null +++ b/llmsdk_docs/kimi_k3/docs/tool-calling-best-practice.md @@ -0,0 +1,104 @@ +> Fetch the complete documentation index at: https://platform.kimi.com/docs/llms.txt +> Use this file to discover all available pages before exploring further. + +# Kimi K3 API 工具调用最佳实践 + +> 工具数量较多时，结合动态加载、tool_choice 与思考力度设计工具调用流程。 + +当 Agent 可用的工具达到几十上百个时，不要把所有工具定义一次性放进请求——它们会占掉大量上下文，还会让模型更容易选错工具。本页介绍一套在 Kimi K3 上的工具编排方式：先用一个搜索工具检索候选工具，再按需把工具定义动态注入对话。 + +## 先声明一个搜索工具，而不是全部工具 + +会话开始时，在请求顶层 `tools` 中只声明一个由你后端实现的 `search_tools` 工具，以及少量每轮都可能用到的核心工具： + +```json theme={null} +{ + "tools": [ + { + "type": "function", + "function": { + "name": "search_tools", + "description": "按关键词搜索可用工具，返回工具名称和简介", + "parameters": { + "type": "object", + "properties": { + "query": { + "type": "string", + "description": "搜索关键词，例如 github、database" + } + }, + "required": ["query"] + } + } + } + ] +} +``` + +在 system prompt 中告知模型可搜索的领域标签（例如工具目录、业务域），引导它在需要工具时先调用 `search_tools`。这样无论工具总量多大，每轮请求里的工具声明都只有少量几个。 + +## 用 tool\_choice 强制首轮检索 + +模型可以选择不调用任何工具、直接凭记忆作答。为了确保它先检索再回答，首轮请求设置 `tool_choice: "required"`： + +```json theme={null} +{ + "model": "kimi-k3", + "messages": [{"role": "user", "content": "帮我创建一个 GitHub PR"}], + "tools": ["..."], + "tool_choice": "required" +} +``` + +检索完成后，后续请求把 `tool_choice` 恢复为 `"auto"`。修改 `tool_choice` 不会破坏前缀缓存，可以按请求粒度调整。各取值含义见[工具调用约束](/docs/guide/use-tool-choice)。 + +## 按需注入工具定义 + +`search_tools` 返回候选工具后，由你的应用把对应工具的完整声明，通过一条携带 `tools` 的 `system` 消息插入 `messages`。工具从该消息所在的位置开始对模型可见： + +```json theme={null} +{ + "role": "system", + "tools": [ + { + "type": "function", + "function": { + "name": "create_github_pr", + "description": "在指定仓库创建 Pull Request", + "parameters": { + "type": "object", + "properties": {} + } + } + } + ] +} +``` + +动态声明的格式与顶层 `tools` 完全一致，不需要维护两套 schema；注入的工具与顶层声明的全局工具并存。动态工具声明按请求生效，不会被服务端记住。下一轮可以继续携带原声明，让工具保持可用并复用前缀缓存；也可以移除该声明。如果工具未在其他位置声明，模型将无法调用这个工具，同时后续前缀可能无法命中缓存。完整用法见[动态加载工具](/docs/guide/use-dynamic-tool-loading)。 + + + 当前一个请求的 prompt tokens 大于 256 时，新的请求才能命中前缀缓存；当前一个请求的 prompt tokens 小于 256 时，请求不会被缓存而是被丢弃。详见[上下文缓存](/docs/guide/use-context-caching-feature-of-kimi-api)。 + + +## 按任务复杂度确定思考力度 + +请求顶层 `reasoning_effort` 支持 `low` / `high` / `max` 三档，默认 `max`。 + +建议在会话开始前确定该配置。在 `messages` 末尾追加动态工具声明，不会影响已有前缀的缓存；删除或修改之前的工具声明，可能影响变更位置之后的缓存命中。修改 `tool_choice` 不会破坏前缀缓存。配置说明见[思考力度](/docs/guide/use-thinking-effort)。 + +## 完整流程 + +1. 会话开始：顶层 `tools` 只放 `search_tools` 和少量核心工具； +2. 首轮检索：`tool_choice: "required"` 强制模型调用 `search_tools`； +3. 按需注入：按检索结果用 `system` 消息动态插入工具定义； +4. 直接调用：模型在后续生成中调用已加载的工具； +5. 思考力度：会话开始前确定顶层 `reasoning_effort` 配置。 + +## 相关阅读 + +* [动态加载工具](/docs/guide/use-dynamic-tool-loading) +* [工具调用约束](/docs/guide/use-tool-choice) +* [思考力度](/docs/guide/use-thinking-effort) +* [使用 Kimi API 完成工具调用](/docs/guide/use-kimi-api-to-complete-tool-calls) +* [模型参数参考](/docs/api/models-overview) diff --git a/llmsdk_docs/kimi_k3/docs/tool-calls.md b/llmsdk_docs/kimi_k3/docs/tool-calls.md new file mode 100644 index 00000000..ec7e23d7 --- /dev/null +++ b/llmsdk_docs/kimi_k3/docs/tool-calls.md @@ -0,0 +1,1217 @@ +> Fetch the complete documentation index at: https://platform.kimi.com/docs/llms.txt +> Use this file to discover all available pages before exploring further. + +# 使用 Kimi API 完成工具调用（tool_calls） + +工具调用 `tool_calls` 让 Kimi 大模型从“说”进化到“做”：模型根据对话上下文决定是否调用工具、以 JSON 格式生成调用参数，由你的应用执行工具并回传结果，模型再基于结果生成最终回复。借助 `tool_calls`，Kimi 大模型能帮你搜索互联网内容、查询数据库，甚至操作智能家居。本页用一个联网搜索案例走通从定义、注册到执行的完整流程，并覆盖流式输出等场景的注意事项。 + +## 一次工具调用的完整流程 + +一次工具调用 `tool_calls` 包含以下步骤： + +1. 使用 JSON Schema 格式定义工具； +2. 通过 `tools` 参数将定义好的工具提交给 Kimi 大模型，你可以一次性提交多个工具； +3. Kimi 大模型会根据当前聊天的上下文，决定使用哪个或哪几个工具，Kimi 大模型也可以选择不使用工具； +4. Kimi 大模型会将调用工具所需要的参数和信息通过 JSON 格式输出； +5. 使用 Kimi 大模型输出的参数，执行对应的工具，并将工具执行结果提交给 Kimi 大模型； +6. Kimi 大模型根据工具执行结果，给予用户回复； + + + 如果你的应用需要挂载大量工具（几十上百个），建议使用[动态加载工具](/docs/guide/use-dynamic-tool-loading)按需注入工具定义，而不是一次性全部提交——可以显著降低 token 消耗并提升工具选择的准确率。 + + +## 用工具调用让模型学会联网搜索 + +Kimi 大模型的知识来源于训练数据，无法回答时效性强的问题。下面用“搜索引擎”和“网页浏览器”两个工具，演示如何让模型自己搜索最新知识并据此作答。 + +### 用 JSON Schema 定义工具 + +人在网上查资料时，通常先打开搜索引擎（例如百度或必应）搜索内容、浏览搜索结果，再打开一个或多个结果网页获取需要的知识。把这两个动作抽象成工具，就是“搜索引擎”和“网页浏览器”——用 JSON Schema 描述后提交给 Kimi 大模型，它就能和人一样搜索并浏览网页。 + +工具定义使用 JSON Schema 格式编写： + +> [JSON Schema](https://json-schema.org/) is a vocabulary that you can use to annotate and validate JSON documents. +> +> [JSON Schema](https://json-schema.org/) 是一种用于描述 JSON 数据格式的 JSON 文档。 + +我们定义以下 JSON Schema： + +```json theme={null} +{ + "type": "object", + "properties": { + "name": { + "type": "string" + } + } +} +``` + +这个 JSON Schema 定义了一个 JSON Object，这个 JSON Object 中包含了一个名为 `name` 的字段，并且该字段的类型为 `string`，例如： + +```json theme={null} +{ + "name": "Hei" +} +``` + +通过 JSON Schema 来描述我们的工具定义，能让 Kimi 大模型更清晰和直观地知道我们的工具需要哪些参数，以及每个参数的类型和介绍。接下来让我们来定义前文提到的“搜索引擎”和“网页浏览器”这两个工具： + + + + ```python theme={null} + tools = [ + { + "type": "function", # 约定的字段 type，目前支持 function 作为值 + "function": { # 当 type 为 function 时，使用 function 字段定义具体的函数内容 + "name": "search", # 函数的名称，请使用英文大小写字母、数据加上减号和下划线作为函数名称 + "description": """ + 通过搜索引擎搜索互联网上的内容。 + + 当你的知识无法回答用户提出的问题，或用户请求你进行联网搜索时，调用此工具。请从与用户的对话中提取用户想要搜索的内容作为 query 参数的值。 + 搜索结果包含网站的标题、网站的地址（URL）以及网站简介。 + """, # 函数的介绍，在这里写上函数的具体作用以及使用场景，以便 Kimi 大模型能正确地选择使用哪些函数 + "parameters": { # 使用 parameters 字段来定义函数接收的参数 + "type": "object", # 固定使用 type: object 来使 Kimi 大模型生成一个 JSON Object 参数 + "required": ["query"], # 使用 required 字段告诉 Kimi 大模型哪些参数是必填项 + "properties": { # properties 中是具体的参数定义，你可以定义多个参数 + "query": { # 在这里，key 是参数名称，value 是参数的具体定义 + "type": "string", # 使用 type 定义参数类型 + "description": """ + 用户搜索的内容，请从用户的提问或聊天上下文中提取。 + """ # 使用 description 描述参数以便 Kimi 大模型更好地生成参数 + } + } + } + } + }, + { + "type": "function", # 约定的字段 type，目前支持 function 作为值 + "function": { # 当 type 为 function 时，使用 function 字段定义具体的函数内容 + "name": "crawl", # 函数的名称，请使用英文大小写字母、数据加上减号和下划线作为函数名称 + "description": """ + 根据网站地址（URL）获取网页内容。 + """, # 函数的介绍，在这里写上函数的具体作用以及使用场景，以便 Kimi 大模型能正确地选择使用哪些函数 + "parameters": { # 使用 parameters 字段来定义函数接收的参数 + "type": "object", # 固定使用 type: object 来使 Kimi 大模型生成一个 JSON Object 参数 + "required": ["url"], # 使用 required 字段告诉 Kimi 大模型哪些参数是必填项 + "properties": { # properties 中是具体的参数定义，你可以定义多个参数 + "url": { # 在这里，key 是参数名称，value 是参数的具体定义 + "type": "string", # 使用 type 定义参数类型 + "description": """ + 需要获取内容的网站地址（URL），通常情况下从搜索结果中可以获取网站的地址。 + """ # 使用 description 描述参数以便 Kimi 大模型更好地生成参数 + } + } + } + } + } + ] + ``` + + + + ```js theme={null} + const tools = [ + { + "type": "function", // 约定的字段 type，目前支持 function 作为值 + "function": { // 当 type 为 function 时，使用 function 字段定义具体的函数内容 + "name": "search", // 函数的名称，请使用英文大小写字母、数据加上减号和下划线作为函数名称 + "description": ""/* + 通过搜索引擎搜索互联网上的内容。 + + 当你的知识无法回答用户提出的问题，或用户请求你进行联网搜索时，调用此工具。请从与用户的对话中提取用户想要搜索的内容作为 query 参数的值。 + 搜索结果包含网站的标题、网站的地址（URL）以及网站简介。 + */, // 函数的介绍，在这里写上函数的具体作用以及使用场景，以便 Kimi 大模型能正确地选择使用哪些函数 + "parameters": { // 使用 parameters 字段来定义函数接收的参数 + "type": "object", // 固定使用 type: object 来使 Kimi 大模型生成一个 JSON Object 参数 + "required": ["query"], // 使用 required 字段告诉 Kimi 大模型哪些参数是必填项 + "properties": { // properties 中是具体的参数定义，你可以定义多个参数 + "query": { // 在这里，key 是参数名称，value 是参数的具体定义 + "type": "string", // 使用 type 定义参数类型 + "description": ""/* + 用户搜索的内容，请从用户的提问或聊天上下文中提取。 + */ // 使用 description 描述参数以便 Kimi 大模型更好地生成参数 + } + } + } + } + }, + { + "type": "function", // 约定的字段 type，目前支持 function 作为值 + "function": { // 当 type 为 function 时，使用 function 字段定义具体的函数内容 + "name": "crawl", // 函数的名称，请使用英文大小写字母、数据加上减号和下划线作为函数名称 + "description": ""/* + 根据网站地址（URL）获取网页内容。 + */, // 函数的介绍，在这里写上函数的具体作用以及使用场景，以便 Kimi 大模型能正确地选择使用哪些函数 + "parameters": { // 使用 parameters 字段来定义函数接收的参数 + "type": "object", // 固定使用 type: object 来使 Kimi 大模型生成一个 JSON Object 参数 + "required": ["url"], // 使用 required 字段告诉 Kimi 大模型哪些参数是必填项 + "properties": { // properties 中是具体的参数定义，你可以定义多个参数 + "url": { // 在这里，key 是参数名称，value 是参数的具体定义 + "type": "string", // 使用 type 定义参数类型 + "description": ""/* + 需要获取内容的网站地址（URL），通常情况下从搜索结果中可以获取网站的地址。 + */ // 使用 description 描述参数以便 Kimi 大模型更好地生成参数 + } + } + } + } + } + ] + ``` + + + +在使用 JSON Schema 定义工具时，我们使用以下固定的格式来定义一个工具： + +```json theme={null} +{ + "type": "function", + "function": { + "name": "NAME", + "description": "DESCRIPTION", + "parameters": { + "type": "object", + "properties": { + + } + } + } +} +``` + +其中，`name`、`description`、`parameters.properties` 由工具提供方定义，其中 `description` 描述了工具的具体作用、以及在什么场合需要使用工具，`parameters` 描述了成功调用工具所需要的具体参数，包括参数类型、参数介绍等；**最终，Kimi 大模型会根据 JSON Schema 的定义，生成一个满足定义要求的 JSON Object 作为工具调用的参数（arguments）。** + +### 把工具注册给模型 + +把 `search` 工具提交给 Kimi 大模型，看看它能否正确调用工具： + + + 本页示例默认使用最新模型 `kimi-k3`。K3 使用请求顶层 `reasoning_effort` 配置思考力度（支持 `"low"` / `"high"` / `"max"`，默认 `"max"`）。换用 `kimi-k2.6`、`kimi-k2.5` 等其他模型时，只需替换 `model` 字段，但各模型的参数配置存在差异，详见[模型参数参考](/docs/api/models-overview)。 + + + + + ```python theme={null} + import os + + from openai import OpenAI + + + client = OpenAI( + api_key=os.environ["MOONSHOT_API_KEY"], # 运行前请设置 MOONSHOT_API_KEY 环境变量 + base_url="https://api.moonshot.cn/v1", + ) + + tools = [ + { + "type": "function", # 约定的字段 type，目前支持 function 作为值 + "function": { # 当 type 为 function 时，使用 function 字段定义具体的函数内容 + "name": "search", # 函数的名称，请使用英文大小写字母、数据加上减号和下划线作为函数名称 + "description": """ + 通过搜索引擎搜索互联网上的内容。 + + 当你的知识无法回答用户提出的问题，或用户请求你进行联网搜索时，调用此工具。请从与用户的对话中提取用户想要搜索的内容作为 query 参数的值。 + 搜索结果包含网站的标题、网站的地址（URL）以及网站简介。 + """, # 函数的介绍，在这里写上函数的具体作用以及使用场景，以便 Kimi 大模型能正确地选择使用哪些函数 + "parameters": { # 使用 parameters 字段来定义函数接收的参数 + "type": "object", # 固定使用 type: object 来使 Kimi 大模型生成一个 JSON Object 参数 + "required": ["query"], # 使用 required 字段告诉 Kimi 大模型哪些参数是必填项 + "properties": { # properties 中是具体的参数定义，你可以定义多个参数 + "query": { # 在这里，key 是参数名称，value 是参数的具体定义 + "type": "string", # 使用 type 定义参数类型 + "description": """ + 用户搜索的内容，请从用户的提问或聊天上下文中提取。 + """ # 使用 description 描述参数以便 Kimi 大模型更好地生成参数 + } + } + } + } + }, + # { + # "type": "function", # 约定的字段 type，目前支持 function 作为值 + # "function": { # 当 type 为 function 时，使用 function 字段定义具体的函数内容 + # "name": "crawl", # 函数的名称，请使用英文大小写字母、数据加上减号和下划线作为函数名称 + # "description": """ + # 根据网站地址（URL）获取网页内容。 + # """, # 函数的介绍，在这里写上函数的具体作用以及使用场景，以便 Kimi 大模型能正确地选择使用哪些函数 + # "parameters": { # 使用 parameters 字段来定义函数接收的参数 + # "type": "object", # 固定使用 type: object 来使 Kimi 大模型生成一个 JSON Object 参数 + # "required": ["url"], # 使用 required 字段告诉 Kimi 大模型哪些参数是必填项 + # "properties": { # properties 中是具体的参数定义，你可以定义多个参数 + # "url": { # 在这里，key 是参数名称，value 是参数的具体定义 + # "type": "string", # 使用 type 定义参数类型 + # "description": """ + # 需要获取内容的网站地址（URL），通常情况下从搜索结果中可以获取网站的地址。 + # """ # 使用 description 描述参数以便 Kimi 大模型更好地生成参数 + # } + # } + # } + # } + # } + ] + + completion = client.chat.completions.create( + model="kimi-k3", + messages=[ + {"role": "system", "content": "你是 Kimi，由 Moonshot AI 提供的人工智能助手，你更擅长中文和英文的对话。你会为用户提供安全，有帮助，准确的回答。同时，你会拒绝一切涉及恐怖主义，种族歧视，黄色暴力等问题的回答。Moonshot AI 为专有名词，不可翻译成其他语言。"}, + {"role": "user", "content": "请联网搜索 Context Caching，并告诉我它是什么。"} # 在提问中要求 Kimi 大模型联网搜索 + ], + tools=tools, # <-- 我们通过 tools 参数，将定义好的 tools 提交给 Kimi 大模型 + ) + + print(completion.choices[0].model_dump_json(indent=4)) + ``` + + + + ```js theme={null} + const OpenAI = require("openai") + + + const client = new OpenAI({ + apiKey: process.env.MOONSHOT_API_KEY, // 运行前请设置 MOONSHOT_API_KEY 环境变量 + baseURL: "https://api.moonshot.cn/v1", + }) + + const tools = [ + { + "type": "function", // 约定的字段 type，目前支持 function 作为值 + "function": { // 当 type 为 function 时，使用 function 字段定义具体的函数内容 + "name": "search", // 函数的名称，请使用英文大小写字母、数据加上减号和下划线作为函数名称 + "description": ""/* + 通过搜索引擎搜索互联网上的内容。 + + 当你的知识无法回答用户提出的问题，或用户请求你进行联网搜索时，调用此工具。请从与用户的对话中提取用户想要搜索的内容作为 query 参数的值。 + 搜索结果包含网站的标题、网站的地址（URL）以及网站简介。 + */, // 函数的介绍，在这里写上函数的具体作用以及使用场景，以便 Kimi 大模型能正确地选择使用哪些函数 + "parameters": { // 使用 parameters 字段来定义函数接收的参数 + "type": "object", // 固定使用 type: object 来使 Kimi 大模型生成一个 JSON Object 参数 + "required": ["query"], // 使用 required 字段告诉 Kimi 大模型哪些参数是必填项 + "properties": { // properties 中是具体的参数定义，你可以定义多个参数 + "query": { // 在这里，key 是参数名称，value 是参数的具体定义 + "type": "string", // 使用 type 定义参数类型 + "description": ""/* + 用户搜索的内容，请从用户的提问或聊天上下文中提取。 + */ // 使用 description 描述参数以便 Kimi 大模型更好地生成参数 + } + } + } + } + }, + // { + // "type": "function", // 约定的字段 type，目前支持 function 作为值 + // "function": { // 当 type 为 function 时，使用 function 字段定义具体的函数内容 + // "name": "crawl", // 函数的名称，请使用英文大小写字母、数据加上减号和下划线作为函数名称 + // "description": """ + // 根据网站地址（URL）获取网页内容。 + // """, // 函数的介绍，在这里写上函数的具体作用以及使用场景，以便 Kimi 大模型能正确地选择使用哪些函数 + // "parameters": { // 使用 parameters 字段来定义函数接收的参数 + // "type": "object", // 固定使用 type: object 来使 Kimi 大模型生成一个 JSON Object 参数 + // "required": ["url"], // 使用 required 字段告诉 Kimi 大模型哪些参数是必填项 + // "properties": { // properties 中是具体的参数定义，你可以定义多个参数 + // "url": { // 在这里，key 是参数名称，value 是参数的具体定义 + // "type": "string", // 使用 type 定义参数类型 + // "description": """ + // 需要获取内容的网站地址（URL），通常情况下从搜索结果中可以获取网站的地址。 + // """ // 使用 description 描述参数以便 Kimi 大模型更好地生成参数 + // } + // } + // } + // } + // } + ] + + async function main() { + const completion = await client.chat.completions.create({ + model: "kimi-k3", + messages: [ + {role: "system", content: "你是 Kimi，由 Moonshot AI 提供的人工智能助手，你更擅长中文和英文的对话。你会为用户提供安全，有帮助，准确的回答。同时，你会拒绝一切涉及恐怖主义，种族歧视，黄色暴力等问题的回答。Moonshot AI 为专有名词，不可翻译成其他语言。"}, + {role: "user", content: "请联网搜索 Context Caching，并告诉我它是什么。"} // 在提问中要求 Kimi 大模型联网搜索 + ], + tools: tools, // <-- 我们通过 tools 参数，将定义好的 tools 提交给 Kimi 大模型 + }) + + console.log(JSON.stringify(completion.choices[0], null, 4)) + } + + main() + ``` + + + +代码运行成功后，模型返回如下内容： + +```json theme={null} +{ + "finish_reason": "tool_calls", + "message": { + "content": "", + "role": "assistant", + "tool_calls": [ + { + "id": "search:0", + "function": { + "arguments": "{\n \"query\": \"Context Caching\"\n}", + "name": "search" + }, + "type": "function" + } + ] + } +} +``` + +`finish_reason` 为 `tool_calls` 表示本次返回的不是模型回复，而是模型选择执行工具——可以通过 `finish_reason` 的值判断当前回复是否是一次工具调用。 + +此时 `message` 中的 `content` 为空，因为模型还在执行 `tool_calls`，尚未生成面向用户的回复；新增的 `tool_calls` 字段是一个列表，包含本次需要调用的所有工具调用信息——这说明 **模型可以一次性选择多个工具进行调用，可以是多个不同的工具，也可以是相同工具使用不同参数进行调用** 。`tool_calls` 中每个元素都代表一次工具调用：模型为每次调用生成唯一的 `id`，用 `function.name` 表明工具函数名称，把执行参数放在 `function.arguments` 中（`arguments` 是合法的、被序列化的 JSON Object；`type` 目前是固定值 `function`）。 + +接下来，用模型生成的工具调用参数去执行具体的工具。 + +### 执行工具并回传结果 + +Kimi 大模型不会替你执行工具——收到模型生成的参数后，需要由你的应用自行执行。为什么模型不自己执行工具？设想一个典型场景： **你向用户提供一个基于 Kimi 大模型的智能机器人，在这个场景有三个角色：用户、机器人、Kimi 大模型。用户向机器人提问，机器人调用 Kimi 大模型 API，并将 API 的结果返回给用户。当使用 `tool_calls` 时，用户向机器人提问，机器人带着 `tools` 调用 Kimi API，Kimi 大模型返回 `tool_calls` 参数，机器人执行完 `tool_calls`，将结果再次提交给 Kimi API，Kimi 大模型生成返回给用户的消息（`finish_reason=stop`），此时机器人才会把消息返回给用户。** 整个 `tool_calls` 过程对用户而言是透明、隐式的：用户并不直接“看到”工具调用，只看到机器人返回的最终回复。 + +下面的完整示例以“机器人”的视角执行模型返回的 `tool_calls`，演示工具执行循环： + + + + ```python theme={null} + from typing import * + + import json + import httpx + import os + + from openai import OpenAI + + + client = OpenAI( + api_key=os.environ["MOONSHOT_API_KEY"], # 运行前请设置 MOONSHOT_API_KEY 环境变量 + base_url="https://api.moonshot.cn/v1", + ) + + tools = [ + { + "type": "function", # 约定的字段 type，目前支持 function 作为值 + "function": { # 当 type 为 function 时，使用 function 字段定义具体的函数内容 + "name": "search", # 函数的名称，请使用英文大小写字母、数据加上减号和下划线作为函数名称 + "description": """ + 通过搜索引擎搜索互联网上的内容。 + + 当你的知识无法回答用户提出的问题，或用户请求你进行联网搜索时，调用此工具。请从与用户的对话中提取用户想要搜索的内容作为 query 参数的值。 + 搜索结果包含网站的标题、网站的地址（URL）以及网站简介。 + """, # 函数的介绍，在这里写上函数的具体作用以及使用场景，以便 Kimi 大模型能正确地选择使用哪些函数 + "parameters": { # 使用 parameters 字段来定义函数接收的参数 + "type": "object", # 固定使用 type: object 来使 Kimi 大模型生成一个 JSON Object 参数 + "required": ["query"], # 使用 required 字段告诉 Kimi 大模型哪些参数是必填项 + "properties": { # properties 中是具体的参数定义，你可以定义多个参数 + "query": { # 在这里，key 是参数名称，value 是参数的具体定义 + "type": "string", # 使用 type 定义参数类型 + "description": """ + 用户搜索的内容，请从用户的提问或聊天上下文中提取。 + """ # 使用 description 描述参数以便 Kimi 大模型更好地生成参数 + } + } + } + } + }, + { + "type": "function", # 约定的字段 type，目前支持 function 作为值 + "function": { # 当 type 为 function 时，使用 function 字段定义具体的函数内容 + "name": "crawl", # 函数的名称，请使用英文大小写字母、数据加上减号和下划线作为函数名称 + "description": """ + 根据网站地址（URL）获取网页内容。 + """, # 函数的介绍，在这里写上函数的具体作用以及使用场景，以便 Kimi 大模型能正确地选择使用哪些函数 + "parameters": { # 使用 parameters 字段来定义函数接收的参数 + "type": "object", # 固定使用 type: object 来使 Kimi 大模型生成一个 JSON Object 参数 + "required": ["url"], # 使用 required 字段告诉 Kimi 大模型哪些参数是必填项 + "properties": { # properties 中是具体的参数定义，你可以定义多个参数 + "url": { # 在这里，key 是参数名称，value 是参数的具体定义 + "type": "string", # 使用 type 定义参数类型 + "description": """ + 需要获取内容的网站地址（URL），通常情况下从搜索结果中可以获取网站的地址。 + """ # 使用 description 描述参数以便 Kimi 大模型更好地生成参数 + } + } + } + } + } + ] + + + def search_impl(query: str) -> List[Dict[str, Any]]: + """ + search_impl 使用搜索引擎对 query 进行搜索，目前主流的搜索引擎（例如 Bing）都提供了 API 调用方式，你可以自行选择 + 你喜欢的搜索引擎 API 进行调用，并将返回结果中的网站标题、网站链接、网站简介信息放置在一个 dict 中返回。 + + 这里只是一个简单的示例，你可能需要编写一些鉴权、校验、解析的代码。 + """ + r = httpx.get("https://your.search.api", params={"query": query}) + return r.json() + + + def search(arguments: Dict[str, Any]) -> Any: + query = arguments["query"] + result = search_impl(query) + return {"result": result} + + + def crawl_impl(url: str) -> str: + """ + crawl_url 根据 url 获取网页上的内容。 + + 这里只是一个简单的示例，在实际的网页抓取过程中，你可能需要编写更多的代码来适配复杂的情况，例如异步加载的数据等；同时，在获取 + 网页内容后，你可以根据自己的需要对网页内容进行清洗，只保留文本或移除不必要的内容（例如广告信息等）。 + """ + r = httpx.get(url) + return r.text + + + def crawl(arguments: dict) -> str: + url = arguments["url"] + content = crawl_impl(url) + return {"content": content} + + + # 通过 tool_map 将每个工具名称及其对应的函数进行映射，以便在 Kimi 大模型返回 tool_calls 时能快速找到应该执行的函数 + tool_map = { + "search": search, + "crawl": crawl, + } + + messages = [ + {"role": "system", + "content": "你是 Kimi，由 Moonshot AI 提供的人工智能助手，你更擅长中文和英文的对话。你会为用户提供安全，有帮助，准确的回答。同时，你会拒绝一切涉及恐怖主义，种族歧视，黄色暴力等问题的回答。Moonshot AI 为专有名词，不可翻译成其他语言。"}, + {"role": "user", "content": "请联网搜索 Context Caching，并告诉我它是什么。"} # 在提问中要求 Kimi 大模型联网搜索 + ] + + finish_reason = None + + # 我们的基本流程是，带着用户的问题和 tools 向 Kimi 大模型提问，如果 Kimi 大模型返回了 finish_reason: tool_calls，则我们执行对应的 tool_calls， + # 将执行结果以 role=tool 的 message 的形式重新提交给 Kimi 大模型，Kimi 大模型根据 tool_calls 结果进行下一步内容的生成： + # + # 1. 如果 Kimi 大模型认为当前的工具调用结果已经可以回答用户问题，则返回 finish_reason: stop，我们会跳出循环，打印出 message.content； + # 2. 如果 Kimi 大模型认为当前的工具调用结果无法回答用户问题，需要再次调用工具，我们会继续在循环中执行接下来的 tool_calls，直到 finish_reason 不再是 tool_calls； + # + # 在这个过程中，只有当 finish_reason 为 stop 时，我们才会将结果返回给用户。 + + while finish_reason is None or finish_reason == "tool_calls": + completion = client.chat.completions.create( + model="kimi-k3", + messages=messages, + tools=tools, # <-- 我们通过 tools 参数，将定义好的 tools 提交给 Kimi 大模型 + ) + choice = completion.choices[0] + finish_reason = choice.finish_reason + if finish_reason == "tool_calls": # <-- 判断当前返回内容是否包含 tool_calls + messages.append(choice.message) # <-- 我们将 Kimi 大模型返回给我们的 assistant 消息也添加到上下文中，以便于下次请求时 Kimi 大模型能理解我们的诉求 + for tool_call in choice.message.tool_calls: # <-- tool_calls 可能是多个，因此我们使用循环逐个执行 + tool_call_name = tool_call.function.name + tool_call_arguments = json.loads(tool_call.function.arguments) # <-- arguments 是序列化后的 JSON Object，我们需要使用 json.loads 反序列化一下 + tool_function = tool_map[tool_call_name] # <-- 通过 tool_map 快速找到需要执行哪个函数 + tool_result = tool_function(tool_call_arguments) + + # 使用函数执行结果构造一个 role=tool 的 message，以此来向模型展示工具调用的结果； + # 注意，我们需要在 message 中提供 tool_call_id 和 name 字段，以便 Kimi 大模型 + # 能正确匹配到对应的 tool_call。 + messages.append({ + "role": "tool", + "tool_call_id": tool_call.id, + "name": tool_call_name, + "content": json.dumps(tool_result), # <-- 我们约定使用字符串格式向 Kimi 大模型提交工具调用结果，因此在这里使用 json.dumps 将执行结果序列化成字符串 + }) + + print(choice.message.content) # <-- 在这里，我们才将模型生成的回复返回给用户 + ``` + + + + ```js theme={null} + const axios = require('axios'); + const openai = require('openai'); // 需要安装 openai 库 + + const client = new openai.OpenAI({ + apiKey: process.env.MOONSHOT_API_KEY, // 运行前请设置 MOONSHOT_API_KEY 环境变量 + baseURL: "https://api.moonshot.cn/v1", + }); + + const tools = [ + { + "type": "function", + "function": { + "name": "search", + "description": "通过搜索引擎搜索互联网上的内容。\n\n当你的知识无法回答用户提出的问题，或用户请求你进行联网搜索时，调用此工具。请从与用户的对话中提取用户想要搜索的内容作为 query 参数的值。\n搜索结果包含网站的标题、网站的地址（URL）以及网站简介。", + "parameters": { + "type": "object", + "required": ["query"], + "properties": { + "query": { + "type": "string", + "description": "用户搜索的内容，请从用户的提问或聊天上下文中提取。" + } + } + } + } + }, + { + "type": "function", + "function": { + "name": "crawl", + "description": "根据网站地址（URL）获取网页内容。", + "parameters": { + "type": "object", + "required": ["url"], + "properties": { + "url": { + "type": "string", + "description": "需要获取内容的网站地址（URL），通常情况下从搜索结果中可以获取网站的地址。" + } + } + } + } + } + ]; + + async function searchImpl(query) { + const response = await axios.get("https://your.search.api", { params: { query } }); + return response.data; + } + + async function search(args) { + const query = args.query; + const result = await searchImpl(query); + return { "result": result }; + } + + async function crawlImpl(url) { + const response = await axios.get(url); + return response.data; + } + + async function crawl(args) { + const url = args.url; + const content = await crawlImpl(url); + return { "content": content }; + } + + const toolMap = { + "search": search, + "crawl": crawl, + }; + + const messages = [ + { "role": "system", "content": "你是 Kimi，由 Moonshot AI 提供的人工智能助手，你更擅长中文和英文的对话。你会为用户提供安全，有帮助，准确的回答。同时，你会拒绝一切涉及恐怖主义，种族歧视，黄色暴力等问题的回答。Moonshot AI 为专有名词，不可翻译成其他语言。" }, + { "role": "user", "content": "请联网搜索 Context Caching，并告诉我它是什么。" } // 在提问中要求 Kimi 大模型联网搜索 + ]; + + let finishReason = null; + let choice; + + async function main() { + while (finishReason === null || finishReason === "tool_calls") { + const completion = await client.chat.completions.create({ + model: "kimi-k3", + messages: messages, + tools: tools, // <-- 我们通过 tools 参数，将定义好的 tools 提交给 Kimi 大模型 + }); + choice = completion.choices[0]; + finishReason = choice.finish_reason; + if (finishReason === "tool_calls") { // <-- 判断当前返回内容是否包含 tool_calls + messages.push(choice.message); // <-- 我们将 Kimi 大模型返回给我们的 assistant 消息也添加到上下文中，以便于下次请求时 Kimi 大模型能理解我们的诉求 + for (const toolCall of choice.message.tool_calls) { // <-- tool_calls 可能是多个，因此我们使用循环逐个执行 + const toolCallName = toolCall.function.name; + const toolCallArguments = JSON.parse(toolCall.function.arguments); // <-- arguments 是序列化后的 JSON Object，我们需要使用 JSON.parse 反序列化一下 + const toolFunction = toolMap[toolCallName]; // <-- 通过 tool_map 快速找到需要执行哪个函数 + const toolResult = await toolFunction(toolCallArguments); + + // 使用函数执行结果构造一个 role=tool 的 message，以此来向模型展示工具调用的结果； + // 注意，我们需要在 message 中提供 tool_call_id 和 name 字段，以便 Kimi 大模型 + // 能正确匹配到对应的 tool_call。 + messages.push({ + "role": "tool", + "tool_call_id": toolCall.id, + "name": toolCallName, + "content": JSON.stringify(toolResult), // <-- 我们约定使用字符串格式向 Kimi 大模型提交工具调用结果，因此在这里使用 JSON.stringify 将执行结果序列化成字符串 + }); + } + } + } + console.log(choice.message.content); // <-- 在这里，我们才将模型生成的回复返回给用户 + } + + main(); + ``` + + + +我们使用 while 循环来执行包含工具调用在内的代码逻辑，这是因为 Kimi 大模型通常不会只执行一次工具调用，尤其是在联网搜索这个场景，通常，Kimi 大模型会先选择调用 `search` 工具，通过 `search` 工具获取搜索结果后，再调用 `crawl` 工具将搜索结果中的 `url` 转换为具体的网页内容，整体的 messages 结构如下所示： + +``` +system: prompt # 系统提示词 +user: prompt # 用户提问 +assistant: tool_call(name=search, arguments={query: query}) # Kimi 大模型返回 tool_call 调用（单个） +tool: search_result(tool_call_id=tool_call.id, name=search) # 提交 tool_call 执行结果 +assistant: tool_call_1(name=crawl, arguments={url: url_1}), tool_call_2(name=crawl, arguments={url: url_2}) # Kimi 大模型继续返回 tool_calls 调用（多个） +tool: crawl_content(tool_call_id=tool_call_1.id, name=crawl) # 提交 tool_call_1 执行结果 +tool: crawl_content(tool_call_id=tool_call_2.id, name=crawl) # 提交 tool_call_2 执行结果 +assistant: message_content(finish_reason=stop) # Kimi 大模型生成面向用户的回复消息，本轮对话结束 +``` + +至此，我们完成了“联网查询”工具调用的全过程，如果你实现了自己的 `search` 和 `crawl` 方法，那么当你向 Kimi 大模型要求联网查询时，它会调用 `search` 和 `crawl` 两个工具，并根据工具调用结果给予你正确的回复。 + +## 处理流式输出中的 tool\_calls + +流式输出模式（`stream`）下，`tool_calls` 同样适用，但有几点需要额外注意： + +* 在流式输出的过程中，由于 `finish_reason` 将会在最后的数据块中出现，因此建议使用 `delta.tool_calls` 字段是否存在来判断当前回复是否包含工具调用； +* 在流式输出的过程中，会先输出 `delta.content`，再输出 `delta.tool_calls`，因此你必须等待 `delta.content` 输出完成后，才能判断和识别 `tool_calls`； +* 在流式输出的过程中，我们会在最初的数据块中，指明当前调用 `tool_calls` 的 `tool_call.id` 和 `tool_call.function.name`，在后续的数据块中将只输出 `tool_call.function.arguments`； +* 在流式输出的过程中，如果 Kimi 大模型一次性返回多个 `tool_calls`，那么我们会额外使用一个名为 `index` 的字段来标识当前 `tool_call` 的索引，以便于你能正确拼接 `tool_call.function.arguments` 参数，我们使用流式输出章节中的代码例子（不使用 SDK 的场合）来说明如何操作： + + + + ```python theme={null} + import os + import json + import httpx + + tools = [ + { + "type": "function", # 约定的字段 type，目前支持 function 作为值 + "function": { # 当 type 为 function 时，使用 function 字段定义具体的函数内容 + "name": "search", # 函数的名称，请使用英文大小写字母、数据加上减号和下划线作为函数名称 + "description": """ + 通过搜索引擎搜索互联网上的内容。 + + 当你的知识无法回答用户提出的问题，或用户请求你进行联网搜索时，调用此工具。请从与用户的对话中提取用户想要搜索的内容作为 query 参数的值。 + 搜索结果包含网站的标题、网站的地址（URL）以及网站简介。 + """, # 函数的介绍，在这里写上函数的具体作用以及使用场景，以便 Kimi 大模型能正确地选择使用哪些函数 + "parameters": { # 使用 parameters 字段来定义函数接收的参数 + "type": "object", # 固定使用 type: object 来使 Kimi 大模型生成一个 JSON Object 参数 + "required": ["query"], # 使用 required 字段告诉 Kimi 大模型哪些参数是必填项 + "properties": { # properties 中是具体的参数定义，你可以定义多个参数 + "query": { # 在这里，key 是参数名称，value 是参数的具体定义 + "type": "string", # 使用 type 定义参数类型 + "description": """ + 用户搜索的内容，请从用户的提问或聊天上下文中提取。 + """ # 使用 description 描述参数以便 Kimi 大模型更好地生成参数 + } + } + } + } + }, + ] + + header = { + "Content-Type": "application/json", + "Authorization": f"Bearer {os.environ.get('MOONSHOT_API_KEY')}", + } + + data = { + "model": "kimi-k3", + "messages": [ + {"role": "user", "content": "请联网搜索 Context Caching 技术。"} + ], + "stream": True, + "tools": tools, # <-- 添加工具调用 + } + + # 使用 httpx 向 Kimi 大模型发出 chat 请求，并获得响应 r + r = httpx.post("https://api.moonshot.cn/v1/chat/completions", + headers=header, + json=data) + if r.status_code != 200: + raise Exception(r.text) + + data: str + + # 在这里，我们预先构建一个 List，用于存放不同的回复消息，由于我们设置了 n=2，因此我们将 List 初始化为 2 个元素 + messages = [{}, {}] + + # 在这里，我们使用了 iter_lines 方法来逐行读取响应体 + for line in r.iter_lines(): + # 去除每一行收尾的空格，以便更好地处理数据块 + line = line.strip() + + # 接下来我们要处理三种不同的情况： + # 1. 如果当前行是空行，则表明前一个数据块已接收完毕（即前文提到的，通过两个换行符结束数据块传输），我们可以对该数据块进行反序列化，并打印出对应的 content 内容； + # 2. 如果当前行为非空行，且以 data: 开头，则表明这是一个数据块传输的开始，我们去除 data: 前缀后，首先判断是否是结束符 [DONE]，如果不是，将数据内容保存到 data 变量； + # 3. 如果当前行为非空行，但不以 data: 开头，则表明当前行仍然归属上一个正在传输的数据块，我们将当前行的内容追加到 data 变量尾部； + + if len(line) == 0: + chunk = json.loads(data) + + # 通过循环获取每个数据块中所有的 choice，并获取 index 对应的 message 对象 + for choice in chunk["choices"]: + index = choice["index"] + message = messages[index] + usage = choice.get("usage") + if usage: + message["usage"] = usage + delta = choice["delta"] + role = delta.get("role") + if role: + message["role"] = role + content = delta.get("content") + if content: + if "content" not in message: + message["content"] = content + else: + message["content"] = message["content"] + content + + # 从这里，我们开始处理 tool_calls + tool_calls = delta.get("tool_calls") # <-- 先判断数据块中是否包含 tool_calls + if tool_calls: + if "tool_calls" not in message: + message["tool_calls"] = [] # <-- 如果包含 tool_calls，我们初始化一个列表来保存这些 tool_calls，注意此时的列表中没有任何元素，长度为 0 + for tool_call in tool_calls: + tool_call_index = tool_call["index"] # <-- 获取当前 tool_call 的 index 索引 + if len(message["tool_calls"]) < ( + tool_call_index + 1): # <-- 根据 index 索引扩充 tool_calls 列表，以便于我们能通过下标访问到对应的 tool_call + message["tool_calls"].extend([{}] * (tool_call_index + 1 - len(message["tool_calls"]))) + tool_call_object = message["tool_calls"][tool_call_index] # <-- 根据下标访问对应的 tool_call + tool_call_object["index"] = tool_call_index + + # 下面的步骤，是根据数据块中的信息填充每个 tool_call 的 id、type、function 字段 + # 在 function 字段中，又包括 name 和 arguments 字段，arguments 字段会由每个数据块 + # 依次补充，如同 delta.content 字段一般。 + + tool_call_id = tool_call.get("id") + if tool_call_id: + tool_call_object["id"] = tool_call_id + tool_call_type = tool_call.get("type") + if tool_call_type: + tool_call_object["type"] = tool_call_type + tool_call_function = tool_call.get("function") + if tool_call_function: + if "function" not in tool_call_object: + tool_call_object["function"] = {} + tool_call_function_name = tool_call_function.get("name") + if tool_call_function_name: + tool_call_object["function"]["name"] = tool_call_function_name + tool_call_function_arguments = tool_call_function.get("arguments") + if tool_call_function_arguments: + if "arguments" not in tool_call_object["function"]: + tool_call_object["function"]["arguments"] = tool_call_function_arguments + else: + tool_call_object["function"]["arguments"] = tool_call_object["function"][ + "arguments"] + tool_call_function_arguments # <-- 依次补充 function.arguments 字段的值 + message["tool_calls"][tool_call_index] = tool_call_object + + data = "" # 重置 data + elif line.startswith("data: "): + data = line[len("data: "):] + + # 当数据块内容为 [DONE] 时，则表明所有数据块已发送完毕，可断开网络连接 + if data == "[DONE]": + break + else: + data = data + "\n" + line # 我们仍然在追加内容时，为其添加一个换行符，因为这可能是该数据块有意将数据分行展示 + + # 在组装完所有 messages 后，我们分别打印其内容 + for index, message in enumerate(messages): + print("index:", index) + print("message:", json.dumps(message, ensure_ascii=False)) + print("") + ``` + + + + ```js theme={null} + const os = require('os'); + const axios = require('axios');// 使用 axios 库来执行 HTTP 请求 + + const tools = [ + { + "type": "function", + "function": { + "name": "search", + "description": "通过搜索引擎搜索互联网上的内容。\n\n当你的知识无法回答用户提出的问题，或用户请求你进行联网搜索时，调用此工具。请从与用户的对话中提取用户想要搜索的内容作为 query 参数的值。\n搜索结果包含网站的标题、网站的地址（URL）以及网站简介。", + "parameters": { + "type": "object", + "required": ["query"], + "properties": { + "query": { + "type": "string", + "description": "用户搜索的内容，请从用户的提问或聊天上下文中提取。" + } + } + } + } + }, + ]; + + const header = { + "Content-Type": "application/json", + "Authorization": `Bearer ${process.env.MOONSHOT_API_KEY}` + }; + + const data = { + "model": "kimi-k3", + "messages": [ + {"role": "user", "content": "请联网搜索 Context Caching 技术。"} + ], + "stream": true, + "tools": tools, + "tool_choice": "auto" + }; + + axios.post("https://api.moonshot.cn/v1/chat/completions", + data,{ + headers: header, + responseType: 'stream' + }).then(response => { + if (response.status !== 200) { + throw new Error(response.text); + } + + let data = ""; + let messages = [{}, {}]; + + response.data.on('data', chunk => { + let line = chunk.toString().trim(); + + if (line === "") { + let chunk = JSON.parse(data); + + for (let choice of chunk.choices) { + let index = choice.index; + let message = messages[index]; + let usage = choice.usage; + if (usage) message.usage = usage; + let delta = choice.delta; + let role = delta.role; + if (role) message.role = role; + let content = delta.content; + if (content) message.content = (message.content || "") + content; + + let tool_calls = delta.tool_calls; + if (tool_calls) { + if (!message.tool_calls) message.tool_calls = []; + for (let tool_call of tool_calls) { + let tool_call_index = tool_call.index; + while (message.tool_calls.length < tool_call_index + 1) { + message.tool_calls.push({}); + } + let tool_call_object = message.tool_calls[tool_call_index]; + tool_call_object.index = tool_call_index; + + let tool_call_id = tool_call.id; + if (tool_call_id) tool_call_object.id = tool_call_id; + let tool_call_type = tool_call.type; + if (tool_call_type) tool_call_object.type = tool_call_type; + let tool_call_function = tool_call.function; + if (tool_call_function) { + if (!tool_call_object.function) tool_call_object.function = {}; + let tool_call_function_name = tool_call_function.name; + if (tool_call_function_name) tool_call_object.function.name = tool_call_function_name; + let tool_call_function_arguments = tool_call_function.arguments; + if (tool_call_function_arguments) { + if (!tool_call_object.function.arguments) { + tool_call_object.function.arguments = tool_call_function_arguments; + } else { + tool_call_object.function.arguments = tool_call_object.function.arguments + tool_call_function_arguments; + } + } + } + message.tool_calls[tool_call_index] = tool_call_object; + } + } + } + data = ""; // 重置 data + } else if (line.startsWith("data: ")) { + data = line.substring(6); + } else { + data = data + "\n" + line; + } + }); + + response.data.on('end', () => { + for (let index = 0; index < messages.length; index++) { + console.log("index:", index); + console.log("message:", JSON.stringify(messages[index], null, 4)); + console.log(""); + } + }); + }).catch(error => { + console.error("请求失败:", error); + }); + ``` + + + +以下是使用 openai SDK 处理流式输出中的 `tool_calls` 的代码示例： + + + + ```python theme={null} + import os + import json + + from openai import OpenAI + + client = OpenAI( + api_key=os.environ.get("MOONSHOT_API_KEY"), + base_url="https://api.moonshot.cn/v1", + ) + + tools = [ + { + "type": "function", # 约定的字段 type，目前支持 function 作为值 + "function": { # 当 type 为 function 时，使用 function 字段定义具体的函数内容 + "name": "search", # 函数的名称，请使用英文大小写字母、数据加上减号和下划线作为函数名称 + "description": """ + 通过搜索引擎搜索互联网上的内容。 + + 当你的知识无法回答用户提出的问题，或用户请求你进行联网搜索时，调用此工具。请从与用户的对话中提取用户想要搜索的内容作为 query 参数的值。 + 搜索结果包含网站的标题、网站的地址（URL）以及网站简介。 + """, # 函数的介绍，在这里写上函数的具体作用以及使用场景，以便 Kimi 大模型能正确地选择使用哪些函数 + "parameters": { # 使用 parameters 字段来定义函数接收的参数 + "type": "object", # 固定使用 type: object 来使 Kimi 大模型生成一个 JSON Object 参数 + "required": ["query"], # 使用 required 字段告诉 Kimi 大模型哪些参数是必填项 + "properties": { # properties 中是具体的参数定义，你可以定义多个参数 + "query": { # 在这里，key 是参数名称，value 是参数的具体定义 + "type": "string", # 使用 type 定义参数类型 + "description": """ + 用户搜索的内容，请从用户的提问或聊天上下文中提取。 + """ # 使用 description 描述参数以便 Kimi 大模型更好地生成参数 + } + } + } + } + }, + ] + + completion = client.chat.completions.create( + model="kimi-k3", + messages=[ + {"role": "user", "content": "请联网搜索 Context Caching 技术。"} + ], + stream=True, + tools=tools, # <-- 添加工具调用 + ) + + # 在这里，我们预先构建一个 List，用于存放不同的回复消息，由于我们设置了 n=2，因此我们将 List 初始化为 2 个元素 + messages = [{}, {}] + + for chunk in completion: + # 通过循环获取每个数据块中所有的 choice，并获取 index 对应的 message 对象 + for choice in chunk.choices: + index = choice.index + message = messages[index] + delta = choice.delta + role = delta.role + if role: + message["role"] = role + content = delta.content + if content: + if "content" not in message: + message["content"] = content + else: + message["content"] = message["content"] + content + + # 从这里，我们开始处理 tool_calls + tool_calls = delta.tool_calls # <-- 先判断数据块中是否包含 tool_calls + if tool_calls: + if "tool_calls" not in message: + message["tool_calls"] = [] # <-- 如果包含 tool_calls，我们初始化一个列表来保存这些 tool_calls，注意此时的列表中没有任何元素，长度为 0 + for tool_call in tool_calls: + tool_call_index = tool_call.index # <-- 获取当前 tool_call 的 index 索引 + if len(message["tool_calls"]) < ( + tool_call_index + 1): # <-- 根据 index 索引扩充 tool_calls 列表，以便于我们能通过下标访问到对应的 tool_call + message["tool_calls"].extend([{}] * (tool_call_index + 1 - len(message["tool_calls"]))) + tool_call_object = message["tool_calls"][tool_call_index] # <-- 根据下标访问对应的 tool_call + tool_call_object["index"] = tool_call_index + + # 下面的步骤，是根据数据块中的信息填充每个 tool_call 的 id、type、function 字段 + # 在 function 字段中，又包括 name 和 arguments 字段，arguments 字段会由每个数据块 + # 依次补充，如同 delta.content 字段一般。 + + tool_call_id = tool_call.id + if tool_call_id: + tool_call_object["id"] = tool_call_id + tool_call_type = tool_call.type + if tool_call_type: + tool_call_object["type"] = tool_call_type + tool_call_function = tool_call.function + if tool_call_function: + if "function" not in tool_call_object: + tool_call_object["function"] = {} + tool_call_function_name = tool_call_function.name + if tool_call_function_name: + tool_call_object["function"]["name"] = tool_call_function_name + tool_call_function_arguments = tool_call_function.arguments + if tool_call_function_arguments: + if "arguments" not in tool_call_object["function"]: + tool_call_object["function"]["arguments"] = tool_call_function_arguments + else: + tool_call_object["function"]["arguments"] = tool_call_object["function"][ + "arguments"] + tool_call_function_arguments # <-- 依次补充 function.arguments 字段的值 + message["tool_calls"][tool_call_index] = tool_call_object + + # 在组装完所有 messages 后，我们分别打印其内容 + for index, message in enumerate(messages): + print("index:", index) + print("message:", json.dumps(message, ensure_ascii=False)) + print("") + ``` + + + + ```js theme={null} + const os = require('os'); + const openai = require('openai'); // 需要安装 openai 库 + + const client = new openai.OpenAI({ + apiKey: process.env.MOONSHOT_API_KEY, + baseURL: "https://api.moonshot.cn/v1" + }); + + const tools = [ + { + "type": "function", + "function": { + "name": "search", + "description": "通过搜索引擎搜索互联网上的内容。\n\n当你的知识无法回答用户提出的问题，或用户请求你进行联网搜索时，调用此工具。请从与用户的对话中提取用户想要搜索的内容作为 query 参数的值。\n搜索结果包含网站的标题、网站的地址（URL）以及网站简介。", + "parameters": { + "type": "object", + "required": ["query"], + "properties": { + "query": { + "type": "string", + "description": "用户搜索的内容，请从用户的提问或聊天上下文中提取。" + } + } + } + } + }, + ]; + + async function main() { + const response = await client.chat.completions.create({ + model: "kimi-k3", + messages: [ + { "role": "user", "content": "请联网搜索 Context Caching 技术。" } + ], + stream: true, + tools: tools, + tool_choice: "auto" + }); + + let messages = [{}, {}]; + let data = ''; + + for await (const chunk of response) { + for (const choice of chunk.choices) { + const index = choice.index; + const message = messages[index]; + const delta = choice.delta; + const role = delta.role; + if (role) message.role = role; + const content = delta.content; + if (content) message.content = (message.content || "") + content; + + const tool_calls = delta.tool_calls; + if (tool_calls) { + if (!message.tool_calls) message.tool_calls = []; + for (const tool_call of tool_calls) { + const tool_call_index = tool_call.index; + if (message.tool_calls.length < tool_call_index + 1) { + for (let i = message.tool_calls.length; i < tool_call_index + 1; i++) { + message.tool_calls.push({}); + } + } + const tool_call_object = message.tool_calls[tool_call_index]; + tool_call_object.index = tool_call_index; + + const tool_call_id = tool_call.id; + if (tool_call_id) tool_call_object.id = tool_call_id; + const tool_call_type = tool_call.type; + if (tool_call_type) tool_call_object.type = tool_call_type; + const tool_call_function = tool_call.function; + if (tool_call_function) { + if (!tool_call_object.function) tool_call_object.function = {}; + const tool_call_function_name = tool_call_function.name; + if (tool_call_function_name) tool_call_object.function.name = tool_call_function_name; + const tool_call_function_arguments = tool_call_function.arguments; + if (tool_call_function_arguments) { + if (!tool_call_object.function.arguments) { + tool_call_object.function.arguments = tool_call_function_arguments; + } else { + tool_call_object.function.arguments += tool_call_function_arguments; + } + } + } + message.tool_calls[tool_call_index] = tool_call_object; + } + } + } + } + + for (let index = 0; index < messages.length; index++) { + console.log("index:", index); + console.log("message:", JSON.stringify(messages[index], null, 2)); + console.log(""); + } + } + + main().catch(console.error); + ``` + + + +## 用 tool\_calls 代替 function\_call + +`tool_calls` 由函数调用（`function_call`）进化而来，`function_call` 是 `tool_calls` 的子集——在某些特定语境下，或阅读兼容性代码时，可以将两者划等号。由于 OpenAI 已将 `function_call` 等参数（例如 `functions`）标记为“已废弃”，我们的 API 将不再支持 `function_call`，请用 `tool_calls` 代替。相比 `function_call`，`tool_calls` 有以下优点： + +* 支持并行调用，Kimi 大模型可以一次返回多个 `tool_calls`，你可以在代码中使用并发的方式同时调用这些 `tool_call` 以减少时间消耗； +* 对于没有依赖关系的 `tool_calls`，Kimi 大模型也会倾向于并行调用，这相比于原顺序调用的 `function_call`，在一定程度上降低了 Tokens 消耗； + +## 注意事项 + +* `finish_reason=tool_calls` 时，`message.content` 偶尔不为空：通常是模型在解释需要调用哪些工具、为什么调用。当工具调用耗时较长，或一轮对话需要串行多次调用工具时，这段描述性语句能减少用户等待的焦虑，也方便用户理解工具调用的流程并及时干预和矫正（例如终止错误的工具调用，或在下一轮对话中通过提示词矫正模型的工具选择）； +* `tools` 参数中的内容也会被计算在总 Tokens 中，请确保 `tools`、`messages` 中的 Tokens 总数合计不超过模型的上下文窗口大小。 + +### 保证每个 tool\_call 都有对应的 tool 消息 + +工具调用场景下，消息不再是 `system` / `user` / `assistant` 的简单交替： + +``` +system: ... +user: ... +assistant: ... +user: ... +assistant: ... +``` + +而是会变成： + +``` +system: ... +user: ... +assistant: ... +tool: ... +tool: ... +assistant: ... +``` + +当 Kimi 大模型生成了 `tool_calls` 时，请确保每一个 `tool_call` 都有对应的 `role=tool` 的 message，并且这条 message 设置了正确的 `tool_call_id`：`role=tool` 的 messages 数量与 `tool_calls` 的数量不一致会导致错误；`role=tool` 的 messages 中的 `tool_call_id` 与 `tool_calls` 中的 `tool_call.id` 无法对应也会导致错误。 + +### 排查 tool\_call\_id not found 错误 + +如果你遇到 `tool_call_id not found` 错误，可能是由于你未将 Kimi API 返回的 `role=assistant` 消息添加到 messages 列表中，正确的消息序列应该看起来像这样： + +``` +system: ... +user: ... +assistant: ... # <-- 也许你并未将这一条 assistant message 添加到 messages 列表中 +tool: ... +tool: ... +assistant: ... +``` + +你可以在每次收到 Kimi API 的返回值后，都执行 `messages.append(message)` 来将 Kimi API 返回的消息添加到消息列表中，以避免出现 `tool_call_id not found` 错误。 + +*注意：添加到 messages 列表中位于 `role=tool` 的 message 之前的 assistant messages，必须完整包含 Kimi API 返回的 `tool_calls` 字段及字段值。我们推荐直接将 Kimi API 返回的 `choice.message` “原封不动”地添加到 messages 列表中，以避免可能产生的错误。* diff --git a/llmsdk_docs/kimi_k3/docs/tool-choice.md b/llmsdk_docs/kimi_k3/docs/tool-choice.md new file mode 100644 index 00000000..d49c55b4 --- /dev/null +++ b/llmsdk_docs/kimi_k3/docs/tool-choice.md @@ -0,0 +1,142 @@ +> Fetch the complete documentation index at: https://platform.kimi.com/docs/llms.txt +> Use this file to discover all available pages before exploring further. + +# 工具调用约束 + +声明工具（`tools`）后，模型默认自行判断本轮是否需要调用工具。`tool_choice` 参数让你显式控制这个行为：强制调用、完全禁止，或保持默认。 + +## 强制模型调用工具：`"required"` + +当工作流必须走工具链路时使用——例如强制检索、强制查询数据库，不允许模型凭记忆直接作答： + +```json theme={null} +{ + "tool_choice": "required" +} +``` + +模型在本轮必须至少调用一个工具。使用时请确保请求中声明了可调用的工具。一个典型用法是工具搜索模式：首轮用 `"required"` 强制模型调用 `search_tools`，检索完成后恢复 `"auto"`，详见 [Kimi K3 API 工具调用最佳实践](/docs/guide/kimi-k3-tool-calling-best-practice)。 + +## 禁止工具调用：`"none"` + +当请求只需要纯文本回复、不希望模型误触发工具时使用： + +```json theme={null} +{ + "tool_choice": "none" +} +``` + +模型会直接输出文本，不产生任何 `tool_calls`，同时降低延迟与 token 消耗。 + +## 让模型自行决定：`"auto"`（默认） + +不传入 `tool_choice` 时即为 `"auto"`：模型根据上下文自行决定是否调用工具，适合常规对话。 + +## 强制调用指定工具：传入函数对象 + +除了三个枚举值，`tool_choice` 还可以传入一个函数对象，强制模型调用指定工具： + +```json theme={null} +{ + "tool_choice": {"type": "function", "function": {"name": "get_weather"}} +} +``` + + + 指定函数调用当前与思考开启不兼容：思考开启时传入会返回 400 错误（`tool_choice 'specified' is incompatible with thinking enabled`）。 + + +## 完整请求示例 + +以下示例声明了一个天气查询工具，并用 `tool_choice: "required"` 强制模型调用它： + + + + ```bash theme={null} + $ curl https://api.moonshot.cn/v1/chat/completions \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $MOONSHOT_API_KEY" \ + -d '{ + "model": "kimi-k3", + "messages": [ + { + "role": "user", + "content": "今天北京的天气怎么样？" + } + ], + "tools": [ + { + "type": "function", + "function": { + "name": "get_weather", + "description": "查询指定城市的实时天气", + "parameters": { + "type": "object", + "properties": { + "city": { + "type": "string", + "description": "城市名称" + } + }, + "required": ["city"] + } + } + } + ], + "tool_choice": "required" + }' + ``` + + + + ```python theme={null} + import os + from openai import OpenAI + + client = OpenAI( + api_key=os.environ["MOONSHOT_API_KEY"], + base_url="https://api.moonshot.cn/v1", + ) + + completion = client.chat.completions.create( + model="kimi-k3", + messages=[ + {"role": "user", "content": "今天北京的天气怎么样？"}, + ], + tools=[ + { + "type": "function", + "function": { + "name": "get_weather", + "description": "查询指定城市的实时天气", + "parameters": { + "type": "object", + "properties": { + "city": {"type": "string", "description": "城市名称"} + }, + "required": ["city"], + }, + }, + } + ], + # 强制模型至少调用一个工具；不传入时默认为 "auto" + tool_choice="required", + ) + + print(completion.choices[0].message.tool_calls) + ``` + + + +## 注意事项 + +* `tool_choice` 是请求级参数，每次请求独立生效，只对本次生成的工具选择行为产生约束； +* 是否设置 `tool_choice` **不会破坏前缀缓存**，可以放心按请求粒度调整该参数。 + +## 相关阅读 + +* [Kimi K3 API 工具调用最佳实践](/docs/guide/kimi-k3-tool-calling-best-practice)：动态加载、tool\_choice 与思考力度的组合实践 +* [动态加载工具](/docs/guide/use-dynamic-tool-loading)：工具数量较多时按需注入工具定义，降低 token 消耗、提升选择准确率 +* [使用 Kimi API 完成工具调用](/docs/guide/use-kimi-api-to-complete-tool-calls)：工具调用的完整流程与示例 +* [模型参数参考](/docs/api/models-overview)：各模型对 `tool_choice` 参数的支持差异 diff --git a/llmsdk_docs/kimi_k3/docs/vision.md b/llmsdk_docs/kimi_k3/docs/vision.md new file mode 100644 index 00000000..0745bbdd --- /dev/null +++ b/llmsdk_docs/kimi_k3/docs/vision.md @@ -0,0 +1,220 @@ +> Fetch the complete documentation index at: https://platform.kimi.com/docs/llms.txt +> Use this file to discover all available pages before exploring further. + +# 配置 Kimi 视觉模型 + +Kimi 视觉模型（包括 `kimi-k3`/`moonshot-v1-8k-vision-preview`/`moonshot-v1-32k-vision-preview`/`moonshot-v1-128k-vision-preview`/`kimi-k2.5`/`kimi-k2.6`/`kimi-k2.7-code`/`kimi-k2.7-code-highspeed`）能够理解视觉内容，包括图片文字、图片颜色和物体形状等内容。`kimi-k3`、`kimi-k2.6`、`kimi-k2.7-code` 和 `kimi-k2.7-code-highspeed` 模型还能理解视频内容。需要让模型识别图片或视频时，按本页方式构造多模态请求。 + +## 用 base64 直接上传图片 + +以下示例把本地图片编码为 base64，通过 `image_url` 类型的消息部分传给 Kimi，并向它提问图片内容： + +```python theme={null} +import os +import base64 + +from openai import OpenAI + +client = OpenAI( + api_key=os.environ.get("MOONSHOT_API_KEY"), + base_url="https://api.moonshot.cn/v1", +) + +# 在这里，你需要将 kimi.png 文件替换为你想让 Kimi 识别的图片的地址 +image_path = "kimi.png" + +with open(image_path, "rb") as f: + image_data = f.read() + +# 我们使用标准库 base64.b64encode 函数将图片编码成 base64 格式的 image_url +image_url = f"data:image/{os.path.splitext(image_path)[1].lstrip('.')};base64,{base64.b64encode(image_data).decode('utf-8')}" + + +completion = client.chat.completions.create( + model="kimi-k3", + messages=[ + {"role": "system", "content": "你是 Kimi。"}, + { + "role": "user", + # 注意这里，content 由原来的 str 类型变更为一个 list，这个 list 中包含多个部分的内容，图片（image_url）是一个部分（part）， + # 文字（text）是一个部分（part） + "content": [ + { + "type": "image_url", # <-- 使用 image_url 类型来上传图片，内容为使用 base64 编码过的图片内容 + "image_url": { + "url": image_url, + }, + }, + { + "type": "text", + "text": "请描述图片的内容。", # <-- 使用 text 类型来提供文字指令，例如"描述图片内容" + }, + ], + }, + ], +) + +print(completion.choices[0].message.content) +``` + +使用 Vision 模型时，`message.content` 必须是 `array[object]`（即 JSON 数组）。**不要** 将 JSON 数组序列化后以 `string` 形式放入 `message.content`；这是非标准格式，不保证被当作视觉输入处理，不同模型或版本的行为可能不同。请始终使用下方的数组格式。 + +正确的格式——`content` 是包含多个部分的 JSON 数组： + +```json theme={null} +{ + "model": "kimi-k3", + "messages": + [ + { + "role": "system", + "content": "你是 Kimi，由 Moonshot AI 提供的人工智能助手，你更擅长中文和英文的对话。你会为用户提供安全，有帮助，准确的回答。同时，你会拒绝一切涉及恐怖主义，种族歧视，黄色暴力等问题的回答。Moonshot AI 为专有名词，不可翻译成其他语言。" + }, + { + "role": "user", + "content": + [ + { + "type": "image_url", + "image_url": + { + "url": "data:image/png;base64,..." + } + }, + { + "type": "text", + "text": "请描述这个图片" + } + ] + } + ] +} +``` + +错误的格式——数组被序列化成了字符串： + +```json theme={null} +{ + "model": "kimi-k3", + "messages": + [ + { + "role": "system", + "content": "你是 Kimi，由 Moonshot AI 提供的人工智能助手，你更擅长中文和英文的对话。你会为用户提供安全，有帮助，准确的回答。同时，你会拒绝一切涉及恐怖主义，种族歧视，黄色暴力等问题的回答。Moonshot AI 为专有名词，不可翻译成其他语言。" + }, + { + "role": "user", + "content": "[{\"type\": \"image_url\", \"image_url\": {\"url\": \"data:image/png;base64,...\"}}, {\"type\": \"text\", \"text\": \"请描述这个图片\"}]" + } + ] +} +``` + +## 用文件 ID 引用已上传的图片或视频 + +视频文件往往更大，可以先把图片或视频上传到 Moonshot，再通过文件 ID 引用，上传方式请参阅 [图片理解上传](/docs/api/files-upload)。以下示例上传一个视频文件，并通过 `ms://` 协议的 `video_url` 请求模型描述视频内容： + +```python theme={null} +import os +from pathlib import Path + +from openai import OpenAI + +client = OpenAI( + api_key=os.environ.get("MOONSHOT_API_KEY"), + base_url="https://api.moonshot.cn/v1", +) + +# 在这里，你需要将 video.mp4 文件替换为你想让 Kimi 识别的图片或视频的地址 +video_path = "video.mp4" + +file_object = client.files.create(file=Path(video_path), purpose="video") # 上传视频到 Moonshot + +completion = client.chat.completions.create( + model="kimi-k3", + messages=[ + { + "role": "system", + "content": "你是 Kimi，由 Moonshot AI 提供的人工智能助手，你更擅长中文和英文的对话。你会为用户提供安全，有帮助，准确的回答。同时，你会拒绝一切涉及恐怖主义，种族歧视，黄色暴力等问题的回答。Moonshot AI 为专有名词，不可翻译成其他语言。" + }, + { + "role": "user", + "content": + [ + { + "type": "video_url", + "video_url": + { + "url": f"ms://{file_object.id}" # 注意这里为 ms:// 而不是 base64 编码后的图片 + } + }, + { + "type": "text", + "text": "请描述这个视频" + } + ] + } + ] +) + +print(completion.choices[0].message.content) +``` + +注意上面例子中 `video_url.url` 的格式为 `ms://`，ms 为 moonshot storage 的缩写，这是 Moonshot 内部引用文件的协议。 + +## 支持的图片与视频格式 + +图片支持以下格式： + +* png +* jpeg +* webp +* gif + +视频支持以下格式： + +* mp4 +* mpeg +* mov +* avi +* x-flv +* mpg +* webm +* wmv +* 3gpp + +## 估算 token 消耗与费用 + +* 图片与视频按动态 token 计算：通过 [计算 token 接口](/docs/api/estimate)，可以在开始理解前获取包含图片或视频的请求的 token 消耗； +* 图片分辨率越高，消耗的 token 越多；视频由若干张关键帧组成，关键帧的数量越多、分辨率越高，token 消耗越多； +* Vision 模型在计费方式上与 `moonshot-v1` 系列模型保持一致，根据模型推理的总 Tokens 计费，token 价格详见 [模型推理价格说明](/docs/pricing/chat-k27-code)。 + +## 控制图片与视频分辨率 + +推荐图片分辨率不超过 4k（4096\*2160），视频分辨率不超过 1080p（1920\*1080）。再高的分辨率只会增加处理时间，也不会对模型理解的效果有提升。 + +## 在 base64 与文件上传之间选择 + +* 由于请求体的整体大小有限制，对于非常大的视频，必须使用上传文件的方式使用视觉理解功能； +* 对于需要多次引用的图片或视频，推荐使用文件上传的方式使用视觉理解功能； +* 关于上传文件的限制，请参阅 [文件上传](/docs/api/files-upload) 文档。 + +## 功能支持与限制 + +Vision 视觉模型支持的特性包括： + +* 多轮对话 +* 流式输出 +* 工具调用 +* JSON Mode +* Partial Mode + +以下功能暂未支持或部分支持： + +* URL 格式的图片：不支持，目前仅支持使用 base64 编码的图片内容和通过文件 ID 上传的图片/视频 + +其他限制： + +* 图片数量：Vision 模型没有图片数量限制，但请确保请求的 Body 大小不超过 100M + +不同模型对 `temperature`、`top_p`、`n` 等参数的取值约束不同，建议不要手动设置；各模型参数差异见 [模型参数配置差异](/docs/api/models-overview) 。 diff --git a/llmsdk_docs/kimi_k3/quickstart.md b/llmsdk_docs/kimi_k3/quickstart.md new file mode 100644 index 00000000..700dbd5b --- /dev/null +++ b/llmsdk_docs/kimi_k3/quickstart.md @@ -0,0 +1,457 @@ +> Fetch the complete documentation index at: https://platform.kimi.com/docs/llms.txt +> Use this file to discover all available pages before exploring further. + +# Kimi K3 + +## Kimi K3 模型介绍 + +Kimi K3 是 Kimi 迄今能力最强的旗舰模型，拥有 2.8 万亿参数，基于 KDA 混合线性注意力机制（Kimi Delta Attention）和注意力残差（Attention Residuals）技术构建，原生支持视觉理解，并拥有 100 万 token 上下文窗口。它是全球首个开源的 3 万亿级别模型，面向长程编程、知识工作和推理等前沿智能场景而设计。 + +完整 Benchmark 与案例请参考 [技术博客](https://www.kimi.com/blog/kimi-k3) 。Kimi 目前正与推理合作伙伴和开源维护者密切协作，对齐技术细节，确保模型能在整个生态中可靠上线。完整模型权重将于 2026 年 7 月 27 日前发布。关于架构、训练和评测的更多细节，将随 Kimi K3 技术报告一同公布。 + +### 3 万亿级开源模型 + +Kimi K3 是首个达到 2.8 万亿参数规模的开源模型。这是 Kimi 持续推进模型规模边界的最新一步：在过去 12 个月（2025/07–2026/07）中的 9 个月里，Kimi 模型都保持着开源模型的规模上限。 + +开源前沿模型规模随时间变化 + +Kimi K3 基于 Kimi Delta Attention（KDA）和 Attention Residuals（AttnRes）构建。这两项架构更新，都是为了让信息在更长序列和更深模型中流动得更顺畅。我们也进一步扩大了 Mixture of Experts（MoE）的稀疏度：结合 Stable LatentMoE 框架后，模型可以在 896 个专家中高效激活 16 个。再加上训练方法和数据配方的优化，这些结构性改进让 Kimi K3 相比 K2 的整体扩展效率提升约 2.5 倍，能更有效地把算力转化为能力。 + +Kimi K3 架构 + +### 编程 + +Kimi K3 具备很强的长程编码能力。在极少人工监督的情况下，它可以持续完成长时间工程任务，理解和处理大型代码库，并协调使用终端工具。 + +Kimi K3 也擅长结合软件工程与视觉推理的任务。它能够利用截图和视觉反馈，优化游戏开发、前端和 CAD 等场景。 + +### 知识工作 + +Kimi K3 推动了端到端知识工作的进展。除了公开基准外，Kimi K3（max）在我们的内部评测中也展现出稳定提升。这些评测来自真实用户与智能体协作流程中反复出现的任务模式和挑战。Kimi K3 在不同生产场景导向的工作流中都表现出一致优势，说明其智能体知识工作能力得到了全面提升。 + +## 访问条件 + +Kimi K3 是旗舰模型：在开放平台完成充值（最低充值金额 10 元）后即可解锁调用。新用户注册认证赠送的 15 元代金券不可用于 Kimi K3。 + +累计充值金额同时决定账户等级与速率限制（并发、RPM、TPM、TPD），详见 [充值与限速](/docs/pricing/limits) 。 + +## 立即开始 + +* [Playground](https://platform.kimi.com/playground) +* [申请 API Key](https://platform.kimi.com/console/api-keys) + +以下示例需要 Python 3.9+ 和 OpenAI SDK。先安装 SDK，并初始化一次客户端；后续 Python 示例复用 `client`。 + +```bash theme={null} +python3 -m pip install --upgrade 'openai>=1.0' +``` + +```python theme={null} +import os + +from openai import OpenAI + +client = OpenAI( + api_key=os.environ["MOONSHOT_API_KEY"], + base_url="https://api.moonshot.cn/v1", +) +``` + +## 基础调用 + + + + ```python theme={null} + completion = client.chat.completions.create( + model="kimi-k3", + messages=[{"role": "user", "content": "用一句话介绍 Kimi K3。"}], + ) + + print(completion.choices[0].message.content) + ``` + + + + ```bash theme={null} + curl https://api.moonshot.cn/v1/chat/completions \ + --header "Authorization: Bearer $MOONSHOT_API_KEY" \ + --header "Content-Type: application/json" \ + --data '{ + "model": "kimi-k3", + "messages": [{"role": "user", "content": "用一句话介绍 Kimi K3。"}] + }' + ``` + + + +## 思考力度 + +K3 始终开启思考模式，并支持通过请求顶层 `reasoning_effort` 配置思考力度。 + + + 思考力度支持 `low` / `high` / `max` 三档（默认 `max`）。用法见 [思考力度](/docs/guide/use-thinking-effort) 。 + + +```python theme={null} +completion = client.chat.completions.create( + model="kimi-k3", + reasoning_effort="max", + messages=[{"role": "user", "content": "证明根号 2 是无理数。"}], +) + +print(completion.choices[0].message.content) +``` + + + 多轮对话和工具调用时，将 API 返回的完整 assistant message 原样加入下一次请求，不要只保留 `content`。 + + +## 流式输出 + +流式响应分别提供推理增量 `reasoning_content` 和最终答案增量 `content`。更多细节见 [流式输出](/docs/guide/utilize-the-streaming-output-feature-of-kimi-api) 。 + +```python theme={null} +stream = client.chat.completions.create( + model="kimi-k3", + messages=[{"role": "user", "content": "解释为什么天空是蓝色的。"}], + stream=True, +) + +for chunk in stream: + delta = chunk.choices[0].delta + reasoning = getattr(delta, "reasoning_content", None) + if reasoning: + print(reasoning, end="", flush=True) + if delta.content: + print(delta.content, end="", flush=True) +``` + +## 视觉输入 + +视觉消息的 `content` 必须是对象数组，而不是序列化后的字符串。完整格式与限制见 [视觉输入](/docs/guide/use-kimi-vision-model) 。 + + + + ```python theme={null} + import base64 + from pathlib import Path + + image_data: str = base64.b64encode(Path("image.png").read_bytes()).decode() + completion = client.chat.completions.create( + model="kimi-k3", + messages=[ + { + "role": "user", + "content": [ + { + "type": "image_url", + "image_url": {"url": f"data:image/png;base64,{image_data}"}, + }, + {"type": "text", "text": "描述这张图片。"}, + ], + } + ], + ) + + print(completion.choices[0].message.content) + ``` + + + + ```python theme={null} + from pathlib import Path + + video = client.files.create(file=Path("video.mp4"), purpose="video") + try: + completion = client.chat.completions.create( + model="kimi-k3", + messages=[ + { + "role": "user", + "content": [ + { + "type": "video_url", + "video_url": {"url": f"ms://{video.id}"}, + }, + {"type": "text", "text": "概括这个视频。"}, + ], + } + ], + ) + print(completion.choices[0].message.content) + finally: + client.files.delete(video.id) + ``` + + + +## 结构化输出 + +使用 `json_schema` 和 `strict: true` 约束最终 `message.content`，只解析该字段，不解析 `reasoning_content`。 + + + ```python theme={null} + import json + + completion = client.chat.completions.create( + model="kimi-k3", + messages=[ + {"role": "user", "content": "小林今年 28 岁。提取姓名和年龄。"} + ], + response_format={ + "type": "json_schema", + "json_schema": { + "name": "person", + "strict": True, + "schema": { + "type": "object", + "properties": { + "name": {"type": "string"}, + "age": {"type": "integer"}, + }, + "required": ["name", "age"], + "additionalProperties": False, + }, + }, + }, + ) + + person: dict[str, object] = json.loads( + completion.choices[0].message.content or "{}" + ) + print(person) + ``` + + +详见 [结构化输出](/docs/guide/response_format) 。 + +## Partial Mode + +在消息末尾添加 `partial=True` 的 assistant message，让模型从指定文本前缀继续生成。最终展示时需要自行拼接前缀。 + +```python theme={null} +prefix: str = "结论：" +completion = client.chat.completions.create( + model="kimi-k3", + messages=[ + {"role": "user", "content": "用一句话说明保持接口兼容的重要性。"}, + {"role": "assistant", "content": prefix, "partial": True}, + ], +) + +print(prefix + (completion.choices[0].message.content or "")) +``` + +详见 [Partial Mode](/docs/guide/use-partial-mode-feature-of-kimi-api) 。 + +## 自定义工具与 `tool_choice` + +首轮用 `tool_choice="required"` 强制至少调用一个工具。执行每个调用后，回传完整 assistant message，并用对应的 `tool_call_id` 逐条追加工具结果。 + + + ```python theme={null} + import json + from typing import Any + + tools: list[dict[str, Any]] = [ + { + "type": "function", + "function": { + "name": "get_weather", + "description": "查询城市天气", + "parameters": { + "type": "object", + "properties": {"city": {"type": "string"}}, + "required": ["city"], + }, + }, + } + ] + messages: list[Any] = [ + {"role": "user", "content": "北京今天天气怎么样？"} + ] + + first = client.chat.completions.create( + model="kimi-k3", + messages=messages, + tools=tools, + tool_choice="required", + ) + assistant_message = first.choices[0].message + messages.append(assistant_message) + + for tool_call in assistant_message.tool_calls or []: + arguments: dict[str, str] = json.loads(tool_call.function.arguments) + result: str = json.dumps( + {"city": arguments["city"], "weather": "晴", "temperature_c": 24}, + ensure_ascii=False, + ) + messages.append( + {"role": "tool", "tool_call_id": tool_call.id, "content": result} + ) + + final = client.chat.completions.create( + model="kimi-k3", + messages=messages, + tools=tools, + ) + print(final.choices[0].message.content) + ``` + + +详见 [工具调用约束](/docs/guide/use-tool-choice) 。 + +## 动态加载工具 + +把完整工具定义放进一条不含 `content` 的 `system` message，即可从该位置起加载工具。 + + + ```python theme={null} + from typing import Any + + dynamic_messages: list[dict[str, Any]] = [ + {"role": "user", "content": "计算 23 乘以 47。"}, + { + "role": "system", + "tools": [ + { + "type": "function", + "function": { + "name": "calculate", + "description": "计算一个算术表达式", + "parameters": { + "type": "object", + "properties": { + "expression": { + "type": "string", + "description": "待计算的算术表达式", + } + }, + "required": ["expression"], + }, + }, + } + ], + }, + ] + completion = client.chat.completions.create( + model="kimi-k3", + messages=dynamic_messages, + ) + + print(completion.choices[0].message.tool_calls) + ``` + + +* 工具定义必须包含完整的 `name`、`description` 和 `parameters`。 +* 声明从该 message 所在位置起生效。 +* 后续请求仍需在历史中携带该 message，服务端不会保存声明。 + +详见 [动态加载工具](/docs/guide/use-dynamic-tool-loading) 。 + +## 1M 上下文与自动缓存 + + + 当前一个请求的 prompt tokens 大于 256 时，新的请求才能命中前缀缓存；当前一个请求的 prompt tokens 小于 256 时，请求不会被缓存而是被丢弃。详见 [上下文缓存](/docs/guide/use-context-caching-feature-of-kimi-api) 。 + + +上下文缓存对普通模型请求自动启用，无需 cache ID、TTL 或额外参数。保持长前缀不变，后续请求会自动尝试命中缓存。 + +```python theme={null} +from pathlib import Path + +knowledge: str = Path("knowledge-base.md").read_text(encoding="utf-8") + +for question in ["总结关键结论。", "列出三个实施风险。"]: + completion = client.chat.completions.create( + model="kimi-k3", + messages=[ + {"role": "system", "content": knowledge}, + {"role": "user", "content": question}, + ], + ) + print(completion.choices[0].message.content) +``` + +详见 [上下文缓存](/docs/guide/use-context-caching-feature-of-kimi-api) 。 + +## 官方工具 + +官方工具通过 Formula 接入： + +1. 从 Formula 的 `/tools` 接口获取工具定义。 +2. 将定义加入 Chat Completions 请求的 `tools`。 +3. 收到 `tool_calls` 后，将对应函数名和参数提交到 Formula 的 `/fibers` 接口。 +4. 将完整 assistant message 和 Fiber 输出作为对应的 tool message 加入历史。 +5. 再次调用 Chat Completions，直到模型返回最终答案。 + +完整客户端与接口契约见 [官方工具](/docs/guide/use-official-tools) 。联网搜索工具正在更新，近期不建议使用。 + +## 重要限制 + +* 思考力度通过请求顶层 `reasoning_effort` 配置，支持 `low` / `high` / `max`（默认 `max`）；K3 始终开启思考模式。 +* `max_completion_tokens` 默认 131072，最大可设置为 1048576。 +* `temperature=1.0`、`top_p=0.95`、`n=1`、`presence_penalty=0`、`frequency_penalty=0` 为固定值，建议不要显式传入。 +* 多轮对话和工具调用必须原样回传完整 assistant message。 +* 视觉输入不支持公网图片 URL；请使用 base64 或 `ms://`，并确保 `content` 是对象数组。 +* 联网搜索正在更新，近期不建议用于生产流程。 + +## 常见问题 + + + + Kimi K3 上下文长度为 1M tokens，计费不按上下文长度分段：所有用量均按量付费，输入（区分缓存命中与未命中）与输出分别按统一单价计费，详见 [Kimi K3 定价](/docs/pricing/chat-k3) 。 + + + + 不可以。模型发布后，国内注册并完成认证的用户获赠的 15 元代金券不可用于体验 Kimi K3，请充值后解锁使用。 + + + + 目前关不了，K3 始终开启思考模式。如果觉得思考过程太长，可以将 `reasoning_effort` 设置为 `low` 降低思考力度，详见 [思考力度](/docs/guide/use-thinking-effort)。 + + + +## 模型价格 + +关于 token 价格，详见 [产品定价](/docs/pricing/chat-k3) 。 + +## 相关文档 + + + + 配置 reasoning\_effort。 + + + + 发送图片与视频。 + + + + 使用严格 JSON Schema。 + + + + 从指定前缀继续生成。 + + + + 控制模型是否调用工具。 + + + + 按需注入工具定义。 + + + + 组合工具调用能力。 + + + + 接入 Formula 工具。 + + + + 查看输入与输出价格。 + + diff --git a/skills/agenthub-python/reference/api.md b/skills/agenthub-python/reference/api.md index f4196c35..d43c4141 100644 --- a/skills/agenthub-python/reference/api.md +++ b/skills/agenthub-python/reference/api.md @@ -39,3 +39,21 @@ def set_history(history: list[UniMessage]) -> None: def clear_history() -> None: """Clear stateful history.""" ``` + +## Module-level helpers + +```python +def list_supported_models(currency: Literal["USD", "CNY"] = "USD") -> list[SupportedModel]: + """List supported models covering official endpoints plus OpenRouter and SiliconFlow. + Each entry carries (model, base_url, client) - mapping onto the AutoLLMClient + constructor (model, base_url, client_type) - plus input/output modalities + (Text/Image/Video/Audio/Embed), context_window, and per-million-token pricing in the + requested currency (official list prices, converted at 7 CNY/USD).""" +``` + +## Errors + +All AgentHub errors subclass `AgentHubError` (a `ValueError`). Unsupported `UniConfig` +values (e.g. `temperature` or `tool_choice` on models that reject them) raise +`UnsupportedParameterError`, which carries `client` and `parameter` attributes. Thinking +levels never raise: every client maps each `ThinkingLevel` to the closest supported level. diff --git a/skills/agenthub-python/reference/models.md b/skills/agenthub-python/reference/models.md index 560b78cd..8be56879 100644 --- a/skills/agenthub-python/reference/models.md +++ b/skills/agenthub-python/reference/models.md @@ -1,11 +1,12 @@ # Model Selection -Use exact model IDs. If a model ID is not listed, ask the user to confirm the exact ID before using it. +Use exact model IDs. If a model ID is not listed, ask the user to confirm the exact ID before using it. The supported model entries (base URL, client, modalities, context window, USD/CNY pricing) are also available at runtime via `agenthub.list_supported_models()`. | Family | Provider | Model IDs | API Key | Base URL | | --- | --- | --- | --- | --- | | Gemini 3 | Official / Vertex AI | `gemini-3.1-pro-preview`, `gemini-3.5-flash`, `gemini-3.1-flash-lite` | `GEMINI_API_KEY` | `GEMINI_BASE_URL` | -| Gemini 3 Image | Official / Vertex AI | `gemini-3.1-flash-image-preview`, `gemini-3-pro-image-preview` | `GEMINI_API_KEY` | `GEMINI_BASE_URL` | +| Gemini 3.6 | Official / Vertex AI | `gemini-3.6-flash`, `gemini-3.5-flash-lite` | `GEMINI_API_KEY` | `GEMINI_BASE_URL` | +| Gemini 3 Image | Official / Vertex AI | `gemini-3.1-flash-image`, `gemini-3-pro-image` | `GEMINI_API_KEY` | `GEMINI_BASE_URL` | | Gemini 3 TTS | Official / Vertex AI | `gemini-3.1-flash-tts-preview` | `GEMINI_API_KEY` | `GEMINI_BASE_URL` | | Gemini Embedding | Official / Vertex AI | `gemini-embedding-2` | `GEMINI_API_KEY` | `GEMINI_BASE_URL` | | Claude 4.6 | Official / ModelVerse | `claude-sonnet-4-6` | `ANTHROPIC_API_KEY` | `ANTHROPIC_BASE_URL` | @@ -22,12 +23,17 @@ Use exact model IDs. If a model ID is not listed, ask the user to confirm the ex | Kimi-K2.6 | Official | `kimi-k2.6` | `MOONSHOT_API_KEY` | `MOONSHOT_BASE_URL` | | Kimi-K2.6 | OpenRouter | `moonshotai/kimi-k2.6` | `MOONSHOT_API_KEY` | `MOONSHOT_BASE_URL` | | Kimi-K2.6 | SiliconFlow | `Pro/moonshotai/Kimi-K2.6` | `MOONSHOT_API_KEY` | `MOONSHOT_BASE_URL` | +| Kimi-K3 | Official | `kimi-k3` | `MOONSHOT_API_KEY` | `MOONSHOT_BASE_URL` | +| Kimi-K3 | OpenRouter | `moonshotai/kimi-k3` | `MOONSHOT_API_KEY` | `MOONSHOT_BASE_URL` | | DeepSeek V4 | Official | `deepseek-v4-pro`, `deepseek-v4-flash` | `DEEPSEEK_API_KEY` | `DEEPSEEK_BASE_URL` | | DeepSeek V4 | OpenRouter | `deepseek/deepseek-v4-pro`, `deepseek/deepseek-v4-flash` | `DEEPSEEK_API_KEY` | `DEEPSEEK_BASE_URL` | | DeepSeek V4 | SiliconFlow | `deepseek-ai/DeepSeek-V4-Pro`, `deepseek-ai/DeepSeek-V4-Flash` | `DEEPSEEK_API_KEY` | `DEEPSEEK_BASE_URL` | | GLM-5.1 | Official | `glm-5.1` | `ZAI_API_KEY` | `ZAI_BASE_URL` | | GLM-5.1 | OpenRouter | `z-ai/glm-5.1` | `ZAI_API_KEY` | `ZAI_BASE_URL` | | GLM-5.1 | SiliconFlow | `Pro/zai-org/GLM-5.1` | `ZAI_API_KEY` | `ZAI_BASE_URL` | +| GLM-5.2 | Official | `glm-5.2` | `ZAI_API_KEY` | `ZAI_BASE_URL` | +| GLM-5.2 | OpenRouter | `z-ai/glm-5.2` | `ZAI_API_KEY` | `ZAI_BASE_URL` | +| GLM-5.2 | SiliconFlow | `zai-org/GLM-5.2` | `ZAI_API_KEY` | `ZAI_BASE_URL` | Common gateway base URLs: diff --git a/skills/agenthub-typescript/reference/api.md b/skills/agenthub-typescript/reference/api.md index 53aff981..74ac64d9 100644 --- a/skills/agenthub-typescript/reference/api.md +++ b/skills/agenthub-typescript/reference/api.md @@ -42,3 +42,24 @@ setHistory(history: UniMessage[]): void; /** Clear stateful history. */ clearHistory(): void; ``` + +## Module-level helpers + +```typescript +/** + * List supported models covering official endpoints plus OpenRouter and + * SiliconFlow. Each entry carries (model, base_url, client) - mapping onto the + * AutoLLMClient constructor (model, baseUrl, clientType) - plus input/output + * modalities (Text/Image/Video/Audio/Embed), context_window, and + * per-million-token pricing in the requested currency (official list prices, + * converted at 7 CNY/USD). + */ +function listSupportedModels(currency?: "USD" | "CNY"): SupportedModel[]; +``` + +## Errors + +All AgentHub errors subclass `AgentHubError`. Unsupported `UniConfig` values (e.g. +`temperature` or `tool_choice` on models that reject them) throw +`UnsupportedParameterError`, which carries `client` and `parameter` fields. Thinking +levels never throw: every client maps each `ThinkingLevel` to the closest supported level. diff --git a/skills/agenthub-typescript/reference/models.md b/skills/agenthub-typescript/reference/models.md index 3727633a..8dff6cf4 100644 --- a/skills/agenthub-typescript/reference/models.md +++ b/skills/agenthub-typescript/reference/models.md @@ -1,11 +1,12 @@ # Model Selection -Use exact model IDs. If a model ID is not listed, ask the user to confirm the exact ID before using it. +Use exact model IDs. If a model ID is not listed, ask the user to confirm the exact ID before using it. The supported model entries (base URL, client, modalities, context window, USD/CNY pricing) are also available at runtime via `listSupportedModels()`. | Family | Provider | Model IDs | API Key | Base URL | | --- | --- | --- | --- | --- | | Gemini 3 | Official / Vertex AI | `gemini-3.1-pro-preview`, `gemini-3.5-flash`, `gemini-3.1-flash-lite` | `GEMINI_API_KEY` | `GEMINI_BASE_URL` | -| Gemini 3 Image | Official / Vertex AI | `gemini-3.1-flash-image-preview`, `gemini-3-pro-image-preview` | `GEMINI_API_KEY` | `GEMINI_BASE_URL` | +| Gemini 3.6 | Official / Vertex AI | `gemini-3.6-flash`, `gemini-3.5-flash-lite` | `GEMINI_API_KEY` | `GEMINI_BASE_URL` | +| Gemini 3 Image | Official / Vertex AI | `gemini-3.1-flash-image`, `gemini-3-pro-image` | `GEMINI_API_KEY` | `GEMINI_BASE_URL` | | Gemini 3 TTS | Official / Vertex AI | `gemini-3.1-flash-tts-preview` | `GEMINI_API_KEY` | `GEMINI_BASE_URL` | | Gemini Embedding | Official / Vertex AI | `gemini-embedding-2` | `GEMINI_API_KEY` | `GEMINI_BASE_URL` | | Claude 4.6 | Official / ModelVerse | `claude-sonnet-4-6` | `ANTHROPIC_API_KEY` | `ANTHROPIC_BASE_URL` | @@ -22,12 +23,17 @@ Use exact model IDs. If a model ID is not listed, ask the user to confirm the ex | Kimi-K2.6 | Official | `kimi-k2.6` | `MOONSHOT_API_KEY` | `MOONSHOT_BASE_URL` | | Kimi-K2.6 | OpenRouter | `moonshotai/kimi-k2.6` | `MOONSHOT_API_KEY` | `MOONSHOT_BASE_URL` | | Kimi-K2.6 | SiliconFlow | `Pro/moonshotai/Kimi-K2.6` | `MOONSHOT_API_KEY` | `MOONSHOT_BASE_URL` | +| Kimi-K3 | Official | `kimi-k3` | `MOONSHOT_API_KEY` | `MOONSHOT_BASE_URL` | +| Kimi-K3 | OpenRouter | `moonshotai/kimi-k3` | `MOONSHOT_API_KEY` | `MOONSHOT_BASE_URL` | | DeepSeek V4 | Official | `deepseek-v4-pro`, `deepseek-v4-flash` | `DEEPSEEK_API_KEY` | `DEEPSEEK_BASE_URL` | | DeepSeek V4 | OpenRouter | `deepseek/deepseek-v4-pro`, `deepseek/deepseek-v4-flash` | `DEEPSEEK_API_KEY` | `DEEPSEEK_BASE_URL` | | DeepSeek V4 | SiliconFlow | `deepseek-ai/DeepSeek-V4-Pro`, `deepseek-ai/DeepSeek-V4-Flash` | `DEEPSEEK_API_KEY` | `DEEPSEEK_BASE_URL` | | GLM-5.1 | Official | `glm-5.1` | `ZAI_API_KEY` | `ZAI_BASE_URL` | | GLM-5.1 | OpenRouter | `z-ai/glm-5.1` | `ZAI_API_KEY` | `ZAI_BASE_URL` | | GLM-5.1 | SiliconFlow | `Pro/zai-org/GLM-5.1` | `ZAI_API_KEY` | `ZAI_BASE_URL` | +| GLM-5.2 | Official | `glm-5.2` | `ZAI_API_KEY` | `ZAI_BASE_URL` | +| GLM-5.2 | OpenRouter | `z-ai/glm-5.2` | `ZAI_API_KEY` | `ZAI_BASE_URL` | +| GLM-5.2 | SiliconFlow | `zai-org/GLM-5.2` | `ZAI_API_KEY` | `ZAI_BASE_URL` | Common gateway base URLs: diff --git a/src_py/agenthub/__init__.py b/src_py/agenthub/__init__.py index aa36ed07..0b1c069e 100644 --- a/src_py/agenthub/__init__.py +++ b/src_py/agenthub/__init__.py @@ -13,15 +13,22 @@ # limitations under the License. from .auto_client import AutoLLMClient -from .errors import AgentHubError, EmptyResponseError, ToolCallArgumentParseError +from .errors import AgentHubError, EmptyResponseError, ToolCallArgumentParseError, UnsupportedParameterError +from .registry import Currency, Modality, ModelPricing, SupportedModel, list_supported_models from .types import PromptCaching, ThinkingLevel __all__ = [ "AgentHubError", "AutoLLMClient", + "Currency", "EmptyResponseError", + "Modality", + "ModelPricing", "PromptCaching", + "SupportedModel", "ThinkingLevel", "ToolCallArgumentParseError", + "UnsupportedParameterError", + "list_supported_models", ] diff --git a/src_py/agenthub/auto_client.py b/src_py/agenthub/auto_client.py index eae05d53..38f942a8 100644 --- a/src_py/agenthub/auto_client.py +++ b/src_py/agenthub/auto_client.py @@ -47,7 +47,14 @@ def _create_client_for_model( ) -> LLMClient: """Create the appropriate client for the given model.""" client_type = (client_type or os.getenv("CLIENT_TYPE", model)).lower() + # gemini-3.6 must be matched before the broader gemini-3 prefix below if any( + prefix in client_type for prefix in ("gemini-3.6", "gemini-3.5-flash-lite") + ): # e.g., gemini-3.6-flash; gemini-3.5-flash-lite shares the sampling-parameter deprecation + from .gemini3_6 import Gemini3_6Client + + return Gemini3_6Client(model=model, api_key=api_key, base_url=base_url) + elif any( prefix in client_type for prefix in ("gemini-3", "gemini-embedding") ): # e.g., gemini-3-flash-preview, gemini-embedding-2 from .gemini3 import Gemini3Client @@ -67,10 +74,18 @@ def _create_client_for_model( from .gpt5_5 import GPT5_5Client return GPT5_5Client(model=model, api_key=api_key, base_url=base_url) + elif "glm-5.2" in client_type: + from .glm5_2 import GLM5_2Client + + return GLM5_2Client(model=model, api_key=api_key, base_url=base_url) elif "glm-5" in client_type or "glm-5.1" in client_type: from .glm5_1 import GLM5_1Client return GLM5_1Client(model=model, api_key=api_key, base_url=base_url) + elif "kimi-k3" in client_type: + from .kimi_k3 import KimiK3Client + + return KimiK3Client(model=model, api_key=api_key, base_url=base_url) elif "kimi-k2.5" in client_type or "kimi-k2.6" in client_type: from .kimi_k2_6 import KimiK2_6Client @@ -90,7 +105,7 @@ def _create_client_for_model( else: raise ValueError( f"{client_type} is not supported. " - "Supported client types: gemini-3, claude-5, claude-4-8, claude-4-7, claude-4-6, gpt-5.5, gpt-5.4, glm-5.1, kimi-k2.6, kimi-k2.5, deepseek-v4, openai-embedding, openai." + "Supported client types: gemini-3.6, gemini-3, claude-5, claude-4-8, claude-4-7, claude-4-6, gpt-5.5, gpt-5.4, glm-5.2, glm-5.1, kimi-k3, kimi-k2.6, kimi-k2.5, deepseek-v4, openai-embedding, openai." ) def transform_uni_config_to_model_config(self, config: UniConfig) -> Any: diff --git a/src_py/agenthub/claude4_6/client.py b/src_py/agenthub/claude4_6/client.py index 42469303..50ae34f9 100644 --- a/src_py/agenthub/claude4_6/client.py +++ b/src_py/agenthub/claude4_6/client.py @@ -23,7 +23,7 @@ from anthropic.types.beta import BetaMessageParam, BetaRawMessageStreamEvent from ..base_client import LLMClient -from ..errors import parse_tool_call_arguments +from ..errors import UnsupportedParameterError, parse_tool_call_arguments from ..types import ( EventType, FinishReason, @@ -118,7 +118,9 @@ def _convert_tool_choice(self, tool_choice: ToolChoice) -> dict[str, str]: """Convert ToolChoice to Claude's tool_choice format.""" if isinstance(tool_choice, list): if len(tool_choice) > 1: - raise ValueError("Claude supports only one tool choice.") + raise UnsupportedParameterError( + self.__class__.__name__, "tool_choice", "Claude supports only one tool choice." + ) return {"type": "any", "name": tool_choice[0]} elif tool_choice == "none": diff --git a/src_py/agenthub/claude5/client.py b/src_py/agenthub/claude5/client.py index 8ca199e1..0b833dc1 100644 --- a/src_py/agenthub/claude5/client.py +++ b/src_py/agenthub/claude5/client.py @@ -23,7 +23,7 @@ from anthropic.types.beta import BetaMessageParam, BetaRawMessageStreamEvent from ..base_client import LLMClient -from ..errors import parse_tool_call_arguments +from ..errors import UnsupportedParameterError, parse_tool_call_arguments from ..types import ( EventType, FinishReason, @@ -118,7 +118,9 @@ def _convert_tool_choice(self, tool_choice: ToolChoice) -> dict[str, str]: """Convert ToolChoice to Claude's tool_choice format.""" if isinstance(tool_choice, list): if len(tool_choice) > 1: - raise ValueError("Claude supports only one tool choice.") + raise UnsupportedParameterError( + self.__class__.__name__, "tool_choice", "Claude supports only one tool choice." + ) return {"type": "any", "name": tool_choice[0]} elif tool_choice == "none": @@ -149,7 +151,9 @@ def transform_uni_config_to_model_config(self, config: UniConfig) -> dict[str, A claude_config["max_tokens"] = 64000 # Claude requires max_tokens to be specified if config.get("temperature") is not None and config["temperature"] != 1.0: - raise ValueError("Claude 4.8 does not support setting temperature.") + raise UnsupportedParameterError( + self.__class__.__name__, "temperature", "Claude 4.8 does not support setting temperature." + ) if config.get("thinking_level") is not None: claude_config.update(self._convert_thinking_level_to_thinking_config(config["thinking_level"])) diff --git a/src_py/agenthub/deepseek_v4/client.py b/src_py/agenthub/deepseek_v4/client.py index 99e27168..3282c5b0 100644 --- a/src_py/agenthub/deepseek_v4/client.py +++ b/src_py/agenthub/deepseek_v4/client.py @@ -20,7 +20,7 @@ from openai.types.chat import ChatCompletionChunk, ChatCompletionMessageParam from ..base_client import LLMClient -from ..errors import parse_tool_call_arguments +from ..errors import UnsupportedParameterError, parse_tool_call_arguments from ..types import ( EventType, FinishReason, @@ -72,7 +72,9 @@ def _convert_tool_choice(self, tool_choice: ToolChoice) -> str: """Convert ToolChoice to DeepSeek's OpenAI-compatible tool_choice format.""" if tool_choice in ["auto", "none"]: return tool_choice - raise ValueError("DeepSeek V4 only supports 'auto' and 'none' for tool_choice.") + raise UnsupportedParameterError( + self.__class__.__name__, "tool_choice", "DeepSeek V4 only supports 'auto' and 'none' for tool_choice." + ) def transform_uni_config_to_model_config(self, config: UniConfig) -> dict[str, Any]: """ @@ -90,7 +92,9 @@ def transform_uni_config_to_model_config(self, config: UniConfig) -> dict[str, A deepseek_config["max_tokens"] = config["max_tokens"] if config.get("temperature") is not None and config["temperature"] != 1.0: - raise ValueError("DeepSeek V4 does not support setting temperature.") + raise UnsupportedParameterError( + self.__class__.__name__, "temperature", "DeepSeek V4 does not support setting temperature." + ) thinking_level = config.get("thinking_level") if thinking_level is not None: @@ -106,7 +110,9 @@ def transform_uni_config_to_model_config(self, config: UniConfig) -> dict[str, A deepseek_config["tool_choice"] = self._convert_tool_choice(config["tool_choice"]) if config.get("prompt_caching") is not None and config["prompt_caching"] != PromptCaching.ENABLE: - raise ValueError("prompt_caching must be ENABLE for DeepSeek.") + raise UnsupportedParameterError( + self.__class__.__name__, "prompt_caching", "prompt_caching must be ENABLE for DeepSeek." + ) return deepseek_config diff --git a/src_py/agenthub/errors.py b/src_py/agenthub/errors.py index 94e1a692..3cf37861 100644 --- a/src_py/agenthub/errors.py +++ b/src_py/agenthub/errors.py @@ -29,6 +29,20 @@ class AgentHubError(ValueError): """Base class for errors raised by AgentHub clients.""" +class UnsupportedParameterError(AgentHubError): + """Raised when a UniConfig parameter value is not supported by the target model client. + + Thinking levels never raise this by design: every client maps each ThinkingLevel + onto the closest level the model supports. Parameters such as temperature and + tool_choice may reject unsupported values with this error. + """ + + def __init__(self, client: str, parameter: str, message: str) -> None: + self.client = client + self.parameter = parameter + super().__init__(message) + + class EmptyResponseError(AgentHubError): """Raised when a completed response carries no non-thinking content and no tool calls. diff --git a/src_py/agenthub/gemini3/client.py b/src_py/agenthub/gemini3/client.py index bfaa87d5..c83f3053 100644 --- a/src_py/agenthub/gemini3/client.py +++ b/src_py/agenthub/gemini3/client.py @@ -25,6 +25,7 @@ from google.oauth2 import service_account from ..base_client import LLMClient +from ..errors import UnsupportedParameterError from ..types import ( ContentItem, EventType, @@ -148,7 +149,9 @@ def transform_uni_config_to_model_config(self, config: UniConfig) -> types.Gener config_params["tool_config"] = types.ToolConfig(function_calling_config=tool_config) if config.get("prompt_caching") is not None and config["prompt_caching"] != PromptCaching.ENABLE: - raise ValueError("prompt_caching must be ENABLE for Gemini 3.") + raise UnsupportedParameterError( + self.__class__.__name__, "prompt_caching", "prompt_caching must be ENABLE for Gemini 3." + ) if config.get("image_config") is not None: config_params["image_config"] = types.ImageConfig(**config["image_config"]) diff --git a/src_py/agenthub/gemini3_6/__init__.py b/src_py/agenthub/gemini3_6/__init__.py new file mode 100644 index 00000000..b0b468ff --- /dev/null +++ b/src_py/agenthub/gemini3_6/__init__.py @@ -0,0 +1,18 @@ +# Copyright 2025 Prism Shadow. and/or its affiliates +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from .client import Gemini3_6Client + + +__all__ = ["Gemini3_6Client"] diff --git a/src_py/agenthub/gemini3_6/client.py b/src_py/agenthub/gemini3_6/client.py new file mode 100644 index 00000000..21dc935a --- /dev/null +++ b/src_py/agenthub/gemini3_6/client.py @@ -0,0 +1,458 @@ +# Copyright 2025 Prism Shadow. and/or its affiliates +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import base64 +import json +import mimetypes +import os +import re +from typing import AsyncIterator + +import httpx +from google import genai +from google.genai import types +from google.oauth2 import service_account + +from ..base_client import LLMClient +from ..errors import UnsupportedParameterError +from ..types import ( + ContentItem, + EventType, + Fidelity, + FinishReason, + PartialContentItem, + PromptCaching, + ThinkingLevel, + ToolChoice, + UniConfig, + UniEvent, + UniMessage, + UsageMetadata, +) + + +class Gemini3_6Client(LLMClient): + """Client for the Gemini 3.6 protocol generation (gemini-3.6-*, gemini-3.5-flash-lite). + + Starting with these models the API deprecates the temperature/top_p/top_k sampling + parameters (silently ignored today, HTTP 400 in future generations), so this client + rejects them instead of sending a no-op. + """ + + def __init__(self, model: str, api_key: str | None = None, base_url: str | None = None): + """Initialize Gemini 3.6 client with model and API key.""" + self._model = model + api_key = api_key or os.getenv("GEMINI_API_KEY") + base_url = base_url or os.getenv("GEMINI_BASE_URL") + http_options = {"base_url": base_url} if base_url else None + if api_key and api_key.startswith("{"): + service_account_info = json.loads(api_key) + credentials = service_account.Credentials.from_service_account_info( + service_account_info, scopes=["https://www.googleapis.com/auth/cloud-platform"] + ) + self._client = genai.Client( + vertexai=True, + credentials=credentials, + project=service_account_info["project_id"], + location="global", + http_options=http_options, + ) + else: + self._client = genai.Client(api_key=api_key, http_options=http_options) + + self._history: list[UniMessage] = [] + + def _detect_image_mime_type(self, url: str) -> str: + """Detect MIME type from URL extension for image.""" + mime_type, _ = mimetypes.guess_type(url) + return mime_type or "image/jpeg" + + async def _get_image_bytes_and_mime_type(self, url: str) -> dict[str, bytes | str]: + """Get image bytes and MIME type from URL.""" + if url.startswith("data:"): + match = re.match(r"data:([^;]+);base64,(.+)", url) + if match: + mime_type = match.group(1) + base64_string = match.group(2) + image_bytes = base64.b64decode(base64_string) + else: + raise ValueError(f"Invalid base64 image: {url}") + else: + async with httpx.AsyncClient() as client: + response = await client.get(url) + response.raise_for_status() + image_bytes = response.content + mime_type = self._detect_image_mime_type(url) + + return {"data": image_bytes, "mime_type": mime_type} + + def _convert_thinking_level(self, thinking_level: ThinkingLevel | None) -> types.ThinkingLevel | None: + """Convert ThinkingLevel enum to Gemini's ThinkingLevel.""" + mapping = { + ThinkingLevel.NONE: types.ThinkingLevel.MINIMAL, + ThinkingLevel.LOW: types.ThinkingLevel.LOW, + ThinkingLevel.MEDIUM: types.ThinkingLevel.MEDIUM, + ThinkingLevel.HIGH: types.ThinkingLevel.HIGH, + ThinkingLevel.XHIGH: types.ThinkingLevel.HIGH, + } + return mapping.get(thinking_level) + + def _convert_tool_choice(self, tool_choice: ToolChoice) -> types.FunctionCallingConfig: + """Convert ToolChoice to Gemini's tool config.""" + if isinstance(tool_choice, list): + return types.FunctionCallingConfig(mode="ANY", allowed_function_names=tool_choice) + elif tool_choice == "none": + return types.FunctionCallingConfig(mode="NONE") + elif tool_choice == "auto": + return types.FunctionCallingConfig(mode="AUTO") + elif tool_choice == "required": + return types.FunctionCallingConfig(mode="ANY") + + def transform_uni_config_to_model_config(self, config: UniConfig) -> types.GenerateContentConfig | None: + """ + Transform universal configuration to Gemini 3.6-specific configuration. + + Args: + config: Universal configuration dict + + Returns: + Gemini GenerateContentConfig object or None if no config needed + """ + config_params = {} + if config.get("system_prompt") is not None: + config_params["system_instruction"] = config["system_prompt"] + + if config.get("max_tokens") is not None: + config_params["max_output_tokens"] = config["max_tokens"] + + if config.get("temperature") is not None: + raise UnsupportedParameterError( + self.__class__.__name__, + "temperature", + "Gemini 3.6 generation models do not support setting temperature.", + ) + + thinking_summary = config.get("thinking_summary") + thinking_level = config.get("thinking_level") + if thinking_summary is not None or thinking_level is not None: + config_params["thinking_config"] = types.ThinkingConfig( + include_thoughts=thinking_summary, thinking_level=self._convert_thinking_level(thinking_level) + ) + + if config.get("tools") is not None: + config_params["tools"] = [types.Tool(function_declarations=config["tools"])] + tool_choice = config.get("tool_choice") + if tool_choice is not None: + tool_config = self._convert_tool_choice(tool_choice) + config_params["tool_config"] = types.ToolConfig(function_calling_config=tool_config) + + if config.get("prompt_caching") is not None and config["prompt_caching"] != PromptCaching.ENABLE: + raise UnsupportedParameterError( + self.__class__.__name__, "prompt_caching", "prompt_caching must be ENABLE for Gemini 3.6." + ) + + if config.get("image_config") is not None: + config_params["image_config"] = types.ImageConfig(**config["image_config"]) + + # tts config + if "tts" in self._model.lower(): + config_params["response_modalities"] = ["AUDIO"] + tts_config = config.get("tts_config") or [{"voice": "Kore"}] + if len(tts_config) not in (1, 2): + raise ValueError("tts_config must contain 1 or 2 entries.") + + if len(tts_config) == 1: + config_params["speech_config"] = types.SpeechConfig( + voice_config=types.VoiceConfig( + prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name=tts_config[0]["voice"]) + ) + ) + else: + speaker_voice_configs = [] + for speaker_config in tts_config: + speaker = speaker_config.get("speaker") + if not speaker: + raise ValueError("speaker is required when tts_config has 2 entries.") + + speaker_voice_configs.append( + types.SpeakerVoiceConfig( + speaker=speaker, + voice_config=types.VoiceConfig( + prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name=speaker_config["voice"]) + ), + ) + ) + + config_params["speech_config"] = types.SpeechConfig( + multi_speaker_voice_config=types.MultiSpeakerVoiceConfig( + speaker_voice_configs=speaker_voice_configs + ) + ) + + return types.GenerateContentConfig(**config_params) if config_params else None + + @staticmethod + def _part_fidelity(part: types.Part) -> dict[str, Fidelity]: + """Wrap a part's thought signature as a fidelity payload, or nothing when absent.""" + if part.thought_signature is None: + return {} + + return {"fidelity": {"signature": part.thought_signature}} + + @staticmethod + def _item_thought_signature(item: ContentItem) -> str | bytes | None: + """Read the thought signature recorded in an item's fidelity payload.""" + return (item.get("fidelity") or {}).get("signature") + + async def transform_uni_message_to_model_input(self, messages: list[UniMessage]) -> list[types.Content]: + """ + Transform universal message format to Gemini's Content format. + + Args: + messages: List of universal message dictionaries + + Returns: + List of Gemini Content objects + """ + mapping = {"user": "user", "assistant": "model"} + contents = [] + for msg in messages: + parts = [] + for item in msg["content_items"]: + if item["type"] == "text": + parts.append(types.Part(text=item["text"], thought_signature=self._item_thought_signature(item))) + elif item["type"] == "image_url": + image_url = item["image_url"] + image_data = await self._get_image_bytes_and_mime_type(image_url) + parts.append(types.Part.from_bytes(**image_data)) + elif item["type"] == "inline_data": + inline_data = types.Blob(data=item["data"], mime_type=item["mime_type"]) + parts.append( + types.Part(inline_data=inline_data, thought_signature=self._item_thought_signature(item)) + ) + elif item["type"] == "thinking": + parts.append( + types.Part( + text=item["thinking"], thought=True, thought_signature=self._item_thought_signature(item) + ) + ) + elif item["type"] == "inline_thinking": + inline_data = types.Blob(data=item["data"], mime_type=item["mime_type"]) + parts.append( + types.Part( + inline_data=inline_data, thought=True, thought_signature=self._item_thought_signature(item) + ) + ) + elif item["type"] == "tool_call": + function_call = types.FunctionCall(name=item["name"], args=item["arguments"]) + parts.append( + types.Part(function_call=function_call, thought_signature=self._item_thought_signature(item)) + ) + elif item["type"] == "tool_result": + if "tool_call_id" not in item: + raise ValueError("tool_call_id is required for tool result.") + + tool_result = {"result": item["text"]} + multimodal_parts = [] + if "images" in item: + for image_url in item["images"]: + image_data = await self._get_image_bytes_and_mime_type(image_url) + multimodal_parts.append( + types.FunctionResponsePart(inline_data=types.FunctionResponseBlob(**image_data)) + ) + + parts.append( + types.Part.from_function_response( + name=item["tool_call_id"], + response=tool_result, + parts=multimodal_parts if multimodal_parts else None, + ) + ) + else: + raise ValueError(f"Unknown item: {item}") + + contents.append(types.Content(role=mapping[msg["role"]], parts=parts)) + + return contents + + def transform_model_output_to_uni_event(self, model_output: types.GenerateContentResponse) -> UniEvent: + """ + Transform Gemini 3.6 model output to universal event format. + + Args: + model_output: Gemini response chunk + + Returns: + Universal event dictionary + """ + event_type: EventType = "delta" + content_items: list[PartialContentItem] = [] + usage_metadata: UsageMetadata | None = None + finish_reason: FinishReason | None = None + + if model_output.candidates: + candidate = model_output.candidates[0] + content = getattr(candidate, "content", None) + for part in getattr(content, "parts", None) or []: + if part.function_call is not None: + content_items.append( + { + "type": "tool_call", + "name": part.function_call.name, + "arguments": part.function_call.args or {}, + "tool_call_id": part.function_call.name, + **self._part_fidelity(part), + } + ) + elif part.thought: + if part.text is not None: + content_items.append({"type": "thinking", "thinking": part.text, **self._part_fidelity(part)}) + elif part.inline_data is not None: + content_items.append( + { + "type": "inline_thinking", + "data": part.inline_data.data, + "mime_type": part.inline_data.mime_type, + **self._part_fidelity(part), + } + ) + elif part.inline_data is not None: + content_items.append( + { + "type": "inline_data", + "data": part.inline_data.data, + "mime_type": part.inline_data.mime_type, + **self._part_fidelity(part), + } + ) + elif part.text is not None: + content_items.append({"type": "text", "text": part.text, **self._part_fidelity(part)}) + else: + raise ValueError(f"Unknown output: {part}") + + if candidate.finish_reason: + event_type = "stop" + stop_reason_mapping = { + types.FinishReason.STOP: "stop", + types.FinishReason.MAX_TOKENS: "length", + } + finish_reason = stop_reason_mapping.get(candidate.finish_reason, "unknown") + + if model_output.usage_metadata: + event_type = event_type or "delta" # deal with separate usage data + + prompt_tokens = model_output.usage_metadata.prompt_token_count or 0 + cached_tokens = model_output.usage_metadata.cached_content_token_count or 0 + usage_metadata = { + "cached_tokens": model_output.usage_metadata.cached_content_token_count, + "prompt_tokens": prompt_tokens - cached_tokens, + "thoughts_tokens": model_output.usage_metadata.thoughts_token_count, + "response_tokens": model_output.usage_metadata.candidates_token_count, + } + + return { + "role": "assistant", + "event_type": event_type, + "content_items": content_items, + "usage_metadata": usage_metadata, + "finish_reason": finish_reason, + } + + async def _embed_messages_internal( + self, + messages: list[UniMessage], + config: UniConfig, + ) -> AsyncIterator[UniEvent]: + """Embed transformed messages and return them as a streaming event.""" + contents = await self.transform_uni_message_to_model_input(messages) + + embedding_config = config.get("embedding_config") or {} + gemini_config = None + if embedding_config.get("dimensions") is not None: + gemini_config = types.EmbedContentConfig(output_dimensionality=embedding_config["dimensions"]) + + result = await self._client.aio.models.embed_content( + model=self._model, + contents=contents, + config=gemini_config, + ) + + yield { + "role": "assistant", + "event_type": "stop", + "content_items": [ + {"type": "embedding", "embedding": list(embedding.values or [])} + for embedding in (result.embeddings or []) + ], + "usage_metadata": { + "cached_tokens": None, + "prompt_tokens": result.metadata.billable_character_count if result.metadata else None, + "thoughts_tokens": None, + "response_tokens": None, + }, + "finish_reason": "stop", + } + + async def _streaming_response_internal( + self, + messages: list[UniMessage], + config: UniConfig, + ) -> AsyncIterator[UniEvent]: + """Stream generate using Gemini SDK with unified conversion methods.""" + if "embedding" in self._model.lower(): + async for event in self._embed_messages_internal(messages, config): + yield event + return + + # Use unified config conversion + gemini_config = self.transform_uni_config_to_model_config(config) + + # check if all items are text for tts model + if "tts" in self._model.lower(): + invalid_item = next( + (item for message in messages for item in message["content_items"] if item["type"] != "text"), + None, + ) + if invalid_item is not None: + raise ValueError(f"Gemini TTS only supports text input, got content item type={invalid_item['type']}.") + + # Use unified message conversion + contents = await self.transform_uni_message_to_model_input(messages) + + # Stream generate + response_stream = await self._client.aio.models.generate_content_stream( + model=self._model, contents=contents, config=gemini_config + ) + async for chunk in response_stream: + event = self.transform_model_output_to_uni_event(chunk) + for item in event["content_items"]: + if item["type"] == "tool_call": + # gemini 3.6 does not support partial tool call, mock a partial tool call event + yield { + "role": "assistant", + "event_type": "delta", + "content_items": [ + { + "type": "partial_tool_call", + "name": item["name"], + "arguments": json.dumps(item["arguments"], ensure_ascii=False), + "tool_call_id": item["tool_call_id"], + "fidelity": item.get("fidelity"), + } + ], + "usage_metadata": None, + "finish_reason": None, + } + + yield event diff --git a/src_py/agenthub/glm5_1/client.py b/src_py/agenthub/glm5_1/client.py index c39e42ab..16d52ac2 100644 --- a/src_py/agenthub/glm5_1/client.py +++ b/src_py/agenthub/glm5_1/client.py @@ -20,7 +20,7 @@ from openai.types.chat import ChatCompletionChunk, ChatCompletionMessageParam from ..base_client import LLMClient -from ..errors import parse_tool_call_arguments +from ..errors import UnsupportedParameterError, parse_tool_call_arguments from ..types import ( EventType, FinishReason, @@ -63,7 +63,9 @@ def _convert_tool_choice(self, tool_choice: ToolChoice) -> str: if tool_choice == "auto": return "auto" else: - raise ValueError("GLM only supports 'auto' for tool_choice.") + raise UnsupportedParameterError( + self.__class__.__name__, "tool_choice", "GLM only supports 'auto' for tool_choice." + ) def transform_uni_config_to_model_config(self, config: UniConfig) -> dict[str, Any]: """ @@ -96,7 +98,9 @@ def transform_uni_config_to_model_config(self, config: UniConfig) -> dict[str, A glm_config["tool_choice"] = self._convert_tool_choice(config["tool_choice"]) if config.get("prompt_caching") is not None and config["prompt_caching"] != PromptCaching.ENABLE: - raise ValueError("prompt_caching must be ENABLE for GLM.") + raise UnsupportedParameterError( + self.__class__.__name__, "prompt_caching", "prompt_caching must be ENABLE for GLM." + ) return glm_config diff --git a/src_py/agenthub/glm5_2/__init__.py b/src_py/agenthub/glm5_2/__init__.py new file mode 100644 index 00000000..fcd08b6e --- /dev/null +++ b/src_py/agenthub/glm5_2/__init__.py @@ -0,0 +1,18 @@ +# Copyright 2025 Prism Shadow. and/or its affiliates +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from .client import GLM5_2Client + + +__all__ = ["GLM5_2Client"] diff --git a/src_py/agenthub/glm5_2/client.py b/src_py/agenthub/glm5_2/client.py new file mode 100644 index 00000000..e96c86b3 --- /dev/null +++ b/src_py/agenthub/glm5_2/client.py @@ -0,0 +1,410 @@ +# Copyright 2025 Prism Shadow. and/or its affiliates +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import json +import os +from typing import Any, AsyncIterator + +from openai import AsyncOpenAI +from openai.types.chat import ChatCompletionChunk, ChatCompletionMessageParam + +from ..base_client import LLMClient +from ..errors import UnsupportedParameterError, parse_tool_call_arguments +from ..types import ( + EventType, + FinishReason, + PartialContentItem, + PromptCaching, + ThinkingLevel, + ToolChoice, + UniConfig, + UniEvent, + UniMessage, + UsageMetadata, +) +from ..utils import fix_openrouter_usage_metadata + + +class GLM5_2Client(LLMClient): + """GLM-5.2-specific LLM client implementation using OpenAI-compatible API.""" + + def __init__(self, model: str, api_key: str | None = None, base_url: str | None = None): + """Initialize GLM-5.2 client with model and API key.""" + self._model = model + api_key = api_key or os.getenv("ZAI_API_KEY") + base_url = base_url or os.getenv("ZAI_BASE_URL") or "https://api.z.ai/api/paas/v4/" + self._client = AsyncOpenAI(api_key=api_key, base_url=base_url) + self._history: list[UniMessage] = [] + + def _convert_thinking_level_to_config(self, thinking_level: ThinkingLevel) -> dict[str, str | bool]: + """Convert ThinkingLevel enum to GLM-5.2's thinking configuration.""" + mapping = { + ThinkingLevel.NONE: {"type": "disabled"}, + ThinkingLevel.LOW: {"type": "enabled", "clear_thinking": False}, + ThinkingLevel.MEDIUM: {"type": "enabled", "clear_thinking": False}, + ThinkingLevel.HIGH: {"type": "enabled", "clear_thinking": False}, + ThinkingLevel.XHIGH: {"type": "enabled", "clear_thinking": False}, + } + return mapping.get(thinking_level) + + def _convert_thinking_level_to_reasoning_effort(self, thinking_level: ThinkingLevel) -> str | None: + """Convert ThinkingLevel enum to GLM-5.2's reasoning_effort. + + The server maps low/medium to high and xhigh to max; NONE disables thinking instead. + """ + mapping = { + ThinkingLevel.NONE: None, + ThinkingLevel.LOW: "low", + ThinkingLevel.MEDIUM: "medium", + ThinkingLevel.HIGH: "high", + ThinkingLevel.XHIGH: "xhigh", + } + return mapping.get(thinking_level) + + def _convert_tool_choice(self, tool_choice: ToolChoice) -> str: + """Convert ToolChoice to OpenAI's tool_choice format.""" + if tool_choice == "auto": + return "auto" + else: + raise UnsupportedParameterError( + self.__class__.__name__, "tool_choice", "GLM only supports 'auto' for tool_choice." + ) + + def transform_uni_config_to_model_config(self, config: UniConfig) -> dict[str, Any]: + """ + Transform universal configuration to GLM-5.2-specific configuration. + + Args: + config: Universal configuration dict + + Returns: + GLM configuration dictionary + """ + glm_config = {"model": self._model, "stream": True, "extra_body": {"tool_stream": True}} + + if config.get("max_tokens") is not None: + glm_config["max_tokens"] = config["max_tokens"] + + if config.get("temperature") is not None: + glm_config["temperature"] = config["temperature"] + + # NOTE: glm-5 always provides thinking summary + if config.get("thinking_level") is not None: + thinking_config = self._convert_thinking_level_to_config(config["thinking_level"]) + # thinking is only effective when using the official API endpoint + glm_config.setdefault("extra_body", {})["thinking"] = thinking_config + reasoning_effort = self._convert_thinking_level_to_reasoning_effort(config["thinking_level"]) + if reasoning_effort is not None: + glm_config["reasoning_effort"] = reasoning_effort + + if config.get("tools") is not None: + glm_config["tools"] = [{"type": "function", "function": tool} for tool in config["tools"]] + + if config.get("tool_choice") is not None: + glm_config["tool_choice"] = self._convert_tool_choice(config["tool_choice"]) + + if config.get("prompt_caching") is not None and config["prompt_caching"] != PromptCaching.ENABLE: + raise UnsupportedParameterError( + self.__class__.__name__, "prompt_caching", "prompt_caching must be ENABLE for GLM." + ) + + return glm_config + + def transform_uni_message_to_model_input(self, messages: list[UniMessage]) -> list[ChatCompletionMessageParam]: + """ + Transform universal message format to OpenAI's message format. + + Args: + messages: List of universal message dictionaries + + Returns: + List of OpenAI message dictionaries + """ + openai_messages = [] + + for msg in messages: + content_parts = [] # may be empty for tool results + tool_calls = [] # may be empty for no tool calls + thinking = "" + thinking_fields: set[str | None] = set() + for item in msg["content_items"]: + if item["type"] == "text": + content_parts.append({"type": "text", "text": item["text"]}) + elif item["type"] == "image_url": + raise ValueError("GLM-5 does not support image inputs.") + elif item["type"] == "thinking": + thinking += item["thinking"] + thinking_fields.add((item.get("fidelity") or {}).get("reasoning_field")) + elif item["type"] == "tool_call": + tool_calls.append( + { + "id": item["tool_call_id"], + "type": "function", + "function": { + "name": item["name"], + "arguments": json.dumps(item["arguments"], ensure_ascii=False), + }, + } + ) + elif item["type"] == "tool_result": + if "tool_call_id" not in item: + raise ValueError("tool_call_id is required for tool result.") + + if "images" in item and item["images"]: + raise ValueError("GLM-5 does not support images in tool results.") + + # Tool results are sent as separate messages + openai_messages.append( + { + "role": "tool", + "tool_call_id": item["tool_call_id"], + "content": item["text"], + } + ) + else: + raise ValueError(f"Unknown item type: {item['type']}") + + message = {"role": msg["role"]} + if content_parts: + message["content"] = content_parts + + if tool_calls: + message["tool_calls"] = tool_calls + + if thinking: + # send thinking back through the exact field the upstream produced (recorded + # in the item fidelity); servers may reject the spelling they did not emit + if thinking_fields == {"reasoning_content"}: + message["reasoning_content"] = thinking + elif thinking_fields == {"reasoning"}: + message["reasoning"] = thinking + else: + message["reasoning_content"] = thinking # vLLM & siliconflow compatibility + message["reasoning"] = thinking # openrouter compatibility + + # message may be empty for tool results + if len(message.keys()) > 1: + openai_messages.append(message) + + return openai_messages + + def transform_model_output_to_uni_event(self, model_output: ChatCompletionChunk) -> UniEvent: + """ + Transform GLM model output to universal event format. + + Args: + model_output: OpenAI streaming chunk + + Returns: + Universal event dictionary + """ + event_type: EventType | None = None + content_items: list[PartialContentItem] = [] + usage_metadata: UsageMetadata | None = None + finish_reason: FinishReason | None = None + + if len(model_output.choices) > 0: + choice = model_output.choices[0] + delta = choice.delta + + if delta.content: + event_type = "delta" + content_items.append({"type": "text", "text": delta.content}) + + # the thinking field name differs by server: vLLM & siliconflow use reasoning_content + # while openrouter uses reasoning; record the wire field that carried each delta + # so a replay can reproduce exactly the field the upstream produced + reasoning_content = getattr(delta, "reasoning_content", None) + reasoning = getattr(delta, "reasoning", None) + if reasoning_content and reasoning: + event_type = "delta" + # ambiguous origin: record no fidelity so a replay sends both fields back + content_items.append({"type": "thinking", "thinking": reasoning_content}) + elif reasoning_content: + event_type = "delta" + content_items.append( + { + "type": "thinking", + "thinking": reasoning_content, + "fidelity": {"reasoning_field": "reasoning_content"}, + } + ) + elif reasoning: + event_type = "delta" + content_items.append( + {"type": "thinking", "thinking": reasoning, "fidelity": {"reasoning_field": "reasoning"}} + ) + + if delta.tool_calls: + event_type = "delta" + for tool_call in delta.tool_calls: + content_items.append( + { + "type": "partial_tool_call", + "name": tool_call.function.name or "", + "arguments": tool_call.function.arguments or "", + "tool_call_id": tool_call.id or "", + } + ) + + if choice.finish_reason: + event_type = event_type or "stop" + finish_reason_mapping = { + "stop": "stop", + "length": "length", + "tool_calls": "tool_call", + "content_filter": "stop", + } + finish_reason = finish_reason_mapping.get(choice.finish_reason, "unknown") + + if model_output.usage: + event_type = event_type or "stop" # deal with separate usage data + + if model_output.usage.prompt_tokens_details: + cached_tokens = model_output.usage.prompt_tokens_details.cached_tokens + else: + cached_tokens = None + + if model_output.usage.completion_tokens_details: + reasoning_tokens = model_output.usage.completion_tokens_details.reasoning_tokens + else: + reasoning_tokens = None + + if cached_tokens is not None: + prompt_tokens = model_output.usage.prompt_tokens - cached_tokens + else: + prompt_tokens = model_output.usage.prompt_tokens + + if reasoning_tokens is not None: + response_tokens = model_output.usage.completion_tokens - reasoning_tokens + else: + response_tokens = model_output.usage.completion_tokens + + usage_metadata = { + "cached_tokens": cached_tokens, + "prompt_tokens": prompt_tokens, + "thoughts_tokens": reasoning_tokens, + "response_tokens": response_tokens, + } + usage_metadata = fix_openrouter_usage_metadata(usage_metadata, str(self._client.base_url)) + + return { + "role": "assistant", + "event_type": event_type, + "content_items": content_items, + "usage_metadata": usage_metadata, + "finish_reason": finish_reason, + } + + async def _streaming_response_internal( + self, + messages: list[UniMessage], + config: UniConfig, + ) -> AsyncIterator[UniEvent]: + """Stream generate using GLM SDK with unified conversion methods.""" + # Use unified config conversion + glm_config = self.transform_uni_config_to_model_config(config) + + # Use unified message conversion + glm_messages = self.transform_uni_message_to_model_input(messages) + + # Extract system prompt if present + if config.get("system_prompt"): + glm_messages.insert(0, {"role": "system", "content": config["system_prompt"]}) + + # Stream generate + stream = await self._client.chat.completions.create(**glm_config, messages=glm_messages) + + partial_tool_call = {} + partial_usage = {} + async for chunk in stream: + event = self.transform_model_output_to_uni_event(chunk) + # the finish reason and usage metadata should be accumulated + partial_usage["finish_reason"] = event["finish_reason"] or partial_usage.get("finish_reason") + partial_usage["usage_metadata"] = event["usage_metadata"] or partial_usage.get("usage_metadata") + if event["event_type"] == "delta": + for item in event["content_items"]: + if item["type"] == "partial_tool_call": + if not partial_tool_call: + # start new partial tool call + partial_tool_call = { + "name": item["name"], + "arguments": item["arguments"], + "tool_call_id": item["tool_call_id"], + } + elif item["name"]: + # finish previous partial tool call + yield { + "role": "assistant", + "event_type": "delta", + "content_items": [ + { + "type": "tool_call", + "name": partial_tool_call["name"], + "arguments": parse_tool_call_arguments( + partial_tool_call["arguments"], + self.__class__.__name__, + partial_tool_call["name"], + partial_tool_call["tool_call_id"], + ), + "tool_call_id": partial_tool_call["tool_call_id"], + } + ], + "usage_metadata": None, + "finish_reason": None, + } + # start new partial tool call + partial_tool_call = { + "name": item["name"], + "arguments": item["arguments"], + "tool_call_id": item["tool_call_id"], + } + else: + # update partial tool call + partial_tool_call["arguments"] += item["arguments"] + + yield event + elif event["event_type"] == "stop": + if partial_tool_call: + # finish partial tool call + yield { + "role": "assistant", + "event_type": "delta", + "content_items": [ + { + "type": "tool_call", + "name": partial_tool_call["name"], + "arguments": parse_tool_call_arguments( + partial_tool_call["arguments"], + self.__class__.__name__, + partial_tool_call["name"], + partial_tool_call["tool_call_id"], + ), + "tool_call_id": partial_tool_call["tool_call_id"], + } + ], + "usage_metadata": None, + "finish_reason": None, + } + partial_tool_call = {} + + if partial_usage.get("finish_reason") and partial_usage.get("usage_metadata"): + yield { + "role": "assistant", + "event_type": "stop", + "content_items": [], + "usage_metadata": partial_usage["usage_metadata"], + "finish_reason": partial_usage["finish_reason"], + } + partial_usage = {} diff --git a/src_py/agenthub/gpt5_5/client.py b/src_py/agenthub/gpt5_5/client.py index eafb2033..8b3cbc78 100644 --- a/src_py/agenthub/gpt5_5/client.py +++ b/src_py/agenthub/gpt5_5/client.py @@ -20,7 +20,7 @@ from openai.types.responses import ResponseInputParam, ResponseStreamEvent from ..base_client import LLMClient -from ..errors import parse_tool_call_arguments +from ..errors import UnsupportedParameterError, parse_tool_call_arguments from ..types import ( EventType, FinishReason, @@ -84,7 +84,9 @@ def transform_uni_config_to_model_config(self, config: UniConfig) -> dict[str, A openai_config["max_output_tokens"] = config["max_tokens"] if config.get("temperature") is not None and config["temperature"] != 1.0: - raise ValueError("GPT-5.5 does not support setting temperature.") + raise UnsupportedParameterError( + self.__class__.__name__, "temperature", "GPT-5.5 does not support setting temperature." + ) if config.get("thinking_level") is not None: openai_config["reasoning"] = {"effort": self._convert_thinking_level_to_effort(config["thinking_level"])} @@ -98,7 +100,9 @@ def transform_uni_config_to_model_config(self, config: UniConfig) -> dict[str, A openai_config["tool_choice"] = self._convert_tool_choice(config["tool_choice"]) if config.get("prompt_caching") is not None and config["prompt_caching"] != PromptCaching.ENABLE: - raise ValueError("prompt_caching must be ENABLE for GPT-5.5.") + raise UnsupportedParameterError( + self.__class__.__name__, "prompt_caching", "prompt_caching must be ENABLE for GPT-5.5." + ) return openai_config diff --git a/src_py/agenthub/kimi_k2_6/client.py b/src_py/agenthub/kimi_k2_6/client.py index a3578147..0102dafe 100644 --- a/src_py/agenthub/kimi_k2_6/client.py +++ b/src_py/agenthub/kimi_k2_6/client.py @@ -23,7 +23,7 @@ from openai.types.chat import ChatCompletionChunk, ChatCompletionMessageParam from ..base_client import LLMClient -from ..errors import parse_tool_call_arguments +from ..errors import UnsupportedParameterError, parse_tool_call_arguments from ..types import ( EventType, FinishReason, @@ -88,7 +88,9 @@ def _convert_tool_choice(self, tool_choice: ToolChoice) -> str: elif tool_choice == "none": return "none" else: - raise ValueError("Kimi only supports 'auto' and 'none' for tool_choice.") + raise UnsupportedParameterError( + self.__class__.__name__, "tool_choice", "Kimi only supports 'auto' and 'none' for tool_choice." + ) def transform_uni_config_to_model_config(self, config: UniConfig) -> dict[str, Any]: """ @@ -106,7 +108,9 @@ def transform_uni_config_to_model_config(self, config: UniConfig) -> dict[str, A kimi_config["max_completion_tokens"] = config["max_tokens"] if config.get("temperature") is not None and config["temperature"] != 1.0: - raise ValueError("Kimi does not support setting temperature.") + raise UnsupportedParameterError( + self.__class__.__name__, "temperature", "Kimi does not support setting temperature." + ) if config.get("thinking_level") is not None: thinking_config = self._convert_thinking_level_to_config(config["thinking_level"]) @@ -119,7 +123,9 @@ def transform_uni_config_to_model_config(self, config: UniConfig) -> dict[str, A kimi_config["tool_choice"] = self._convert_tool_choice(config["tool_choice"]) if config.get("prompt_caching") is not None and config["prompt_caching"] != PromptCaching.ENABLE: - raise ValueError("prompt_caching must be ENABLE for Kimi.") + raise UnsupportedParameterError( + self.__class__.__name__, "prompt_caching", "prompt_caching must be ENABLE for Kimi." + ) if config.get("trace_id") is not None: # use trace_id as the prompt cache key kimi_config["prompt_cache_key"] = config["trace_id"] diff --git a/src_py/agenthub/kimi_k3/__init__.py b/src_py/agenthub/kimi_k3/__init__.py new file mode 100644 index 00000000..3b6c1bcd --- /dev/null +++ b/src_py/agenthub/kimi_k3/__init__.py @@ -0,0 +1,18 @@ +# Copyright 2025 Prism Shadow. and/or its affiliates +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from .client import KimiK3Client + + +__all__ = ["KimiK3Client"] diff --git a/src_py/agenthub/kimi_k3/client.py b/src_py/agenthub/kimi_k3/client.py new file mode 100644 index 00000000..dbcb8b4c --- /dev/null +++ b/src_py/agenthub/kimi_k3/client.py @@ -0,0 +1,436 @@ +# Copyright 2025 Prism Shadow. and/or its affiliates +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import base64 +import json +import mimetypes +import os +from typing import Any, AsyncIterator + +import httpx +from openai import AsyncOpenAI +from openai.types.chat import ChatCompletionChunk, ChatCompletionMessageParam + +from ..base_client import LLMClient +from ..errors import UnsupportedParameterError, parse_tool_call_arguments +from ..types import ( + EventType, + FinishReason, + PartialContentItem, + PromptCaching, + ThinkingLevel, + ToolChoice, + UniConfig, + UniEvent, + UniMessage, + UsageMetadata, +) +from ..utils import fix_openrouter_usage_metadata + + +class KimiK3Client(LLMClient): + """Kimi K3-specific LLM client implementation using OpenAI-compatible API.""" + + def __init__(self, model: str, api_key: str | None = None, base_url: str | None = None): + """Initialize Kimi K3 client with model and API key.""" + self._model = model + api_key = api_key or os.getenv("MOONSHOT_API_KEY") + base_url = base_url or os.getenv("MOONSHOT_BASE_URL") or "https://api.moonshot.cn/v1" + self._client = AsyncOpenAI(api_key=api_key, base_url=base_url) + self._history: list[UniMessage] = [] + + async def _convert_image_url_to_base64(self, url: str) -> str: + """Convert image URL to base64-encoded string. + + Args: + url: Image URL to convert + + Returns: + Base64-encoded image string + """ + if url.startswith("data:"): + return url + + async with httpx.AsyncClient() as client: + response = await client.get(url) + response.raise_for_status() + image_bytes = response.content + mime_type = mimetypes.guess_type(url)[0] or "image/jpeg" + base64_string = base64.b64encode(image_bytes).decode("utf-8") + return f"data:{mime_type};base64,{base64_string}" + + def _convert_thinking_level_to_reasoning_effort(self, thinking_level: ThinkingLevel) -> str: + """Convert ThinkingLevel enum to Kimi K3's reasoning_effort. + + K3 cannot disable reasoning, so NONE degrades to the lowest effort instead of raising. + """ + mapping = { + ThinkingLevel.NONE: "low", + ThinkingLevel.LOW: "low", + ThinkingLevel.MEDIUM: "high", + ThinkingLevel.HIGH: "high", + ThinkingLevel.XHIGH: "max", + } + return mapping.get(thinking_level) + + def _convert_tool_choice(self, tool_choice: ToolChoice) -> str: + """Convert ToolChoice to OpenAI's tool_choice format.""" + if tool_choice == "auto": + return "auto" + elif tool_choice == "none": + return "none" + elif tool_choice == "required": + return "required" + else: + # forcing a specific tool is incompatible with K3's always-on reasoning + raise UnsupportedParameterError( + self.__class__.__name__, + "tool_choice", + "Kimi K3 does not support forcing specific tools; only 'auto', 'none' and 'required' are supported.", + ) + + def transform_uni_config_to_model_config(self, config: UniConfig) -> dict[str, Any]: + """ + Transform universal configuration to Kimi K3-specific configuration. + + Args: + config: Universal configuration dict + + Returns: + Kimi configuration dictionary + """ + kimi_config = {"model": self._model, "stream": True, "stream_options": {"include_usage": True}} + + if config.get("max_tokens") is not None: + kimi_config["max_completion_tokens"] = config["max_tokens"] + + if config.get("temperature") is not None and config["temperature"] != 1.0: + raise UnsupportedParameterError( + self.__class__.__name__, "temperature", "Kimi K3 does not support setting temperature." + ) + + if config.get("thinking_level") is not None: + kimi_config["reasoning_effort"] = self._convert_thinking_level_to_reasoning_effort( + config["thinking_level"] + ) + + if config.get("tools") is not None: + kimi_config["tools"] = [{"type": "function", "function": tool} for tool in config["tools"]] + + if config.get("tool_choice") is not None: + kimi_config["tool_choice"] = self._convert_tool_choice(config["tool_choice"]) + + if config.get("prompt_caching") is not None and config["prompt_caching"] != PromptCaching.ENABLE: + raise UnsupportedParameterError( + self.__class__.__name__, "prompt_caching", "prompt_caching must be ENABLE for Kimi K3." + ) + + # K3 context caching is automatic; trace_id is intentionally not sent as prompt_cache_key + return kimi_config + + async def transform_uni_message_to_model_input( + self, messages: list[UniMessage] + ) -> list[ChatCompletionMessageParam]: + """ + Transform universal message format to OpenAI's message format. + + Args: + messages: List of universal message dictionaries + + Returns: + List of OpenAI message dictionaries + """ + openai_messages = [] + + for msg in messages: + content_parts = [] # may be empty for tool results + tool_calls = [] # may be empty for no tool calls + thinking = "" + thinking_fields: set[str | None] = set() + for item in msg["content_items"]: + if item["type"] == "text": + content_parts.append({"type": "text", "text": item["text"]}) + elif item["type"] == "image_url": + base64_image = await self._convert_image_url_to_base64(item["image_url"]) + content_parts.append({"type": "image_url", "image_url": {"url": base64_image}}) + elif item["type"] == "thinking": + thinking += item["thinking"] + thinking_fields.add((item.get("fidelity") or {}).get("reasoning_field")) + elif item["type"] == "tool_call": + tool_calls.append( + { + "id": item["tool_call_id"], + "type": "function", + "function": { + "name": item["name"], + "arguments": json.dumps(item["arguments"], ensure_ascii=False), + }, + } + ) + elif item["type"] == "tool_result": + if "tool_call_id" not in item: + raise ValueError("tool_call_id is required for tool result.") + + content = [{"type": "text", "text": item["text"]}] + + if "images" in item and item["images"]: + for image_url in item["images"]: + base64_image = await self._convert_image_url_to_base64(image_url) + if "siliconflow.cn" in str(self._client.base_url): + # siliconflow does not support image_url in tool result + content_parts.append({"type": "image_url", "image_url": {"url": base64_image}}) + else: + content.append({"type": "image_url", "image_url": {"url": base64_image}}) + + # Tool results are sent as separate messages + openai_messages.append( + { + "role": "tool", + "tool_call_id": item["tool_call_id"], + "content": content, + } + ) + else: + raise ValueError(f"Unknown item type: {item['type']}") + + message = {"role": msg["role"]} + if content_parts: + message["content"] = content_parts + + if tool_calls: + message["tool_calls"] = tool_calls + + if thinking: + # send thinking back through the exact field the upstream produced (recorded + # in the item fidelity); servers may reject the spelling they did not emit + if thinking_fields == {"reasoning_content"}: + message["reasoning_content"] = thinking + elif thinking_fields == {"reasoning"}: + message["reasoning"] = thinking + else: + message["reasoning_content"] = thinking # vLLM & siliconflow compatibility + message["reasoning"] = thinking # openrouter compatibility + + # message may be empty for tool results + if len(message.keys()) > 1: + openai_messages.append(message) + + return openai_messages + + def transform_model_output_to_uni_event(self, model_output: ChatCompletionChunk) -> UniEvent: + """ + Transform Kimi K3 model output to universal event format. + + Args: + model_output: OpenAI streaming chunk + + Returns: + Universal event dictionary + """ + event_type: EventType | None = None + content_items: list[PartialContentItem] = [] + usage_metadata: UsageMetadata | None = None + finish_reason: FinishReason | None = None + + if len(model_output.choices) > 0: + choice = model_output.choices[0] + delta = choice.delta + + if delta.content: + event_type = "delta" + content_items.append({"type": "text", "text": delta.content}) + + # the thinking field name differs by server: vLLM & siliconflow use reasoning_content + # while openrouter uses reasoning; record the wire field that carried each delta + # so a replay can reproduce exactly the field the upstream produced + reasoning_content = getattr(delta, "reasoning_content", None) + reasoning = getattr(delta, "reasoning", None) + if reasoning_content and reasoning: + event_type = "delta" + # ambiguous origin: record no fidelity so a replay sends both fields back + content_items.append({"type": "thinking", "thinking": reasoning_content}) + elif reasoning_content: + event_type = "delta" + content_items.append( + { + "type": "thinking", + "thinking": reasoning_content, + "fidelity": {"reasoning_field": "reasoning_content"}, + } + ) + elif reasoning: + event_type = "delta" + content_items.append( + {"type": "thinking", "thinking": reasoning, "fidelity": {"reasoning_field": "reasoning"}} + ) + + if delta.tool_calls: + event_type = "delta" + for tool_call in delta.tool_calls: + content_items.append( + { + "type": "partial_tool_call", + "name": tool_call.function.name or "", + "arguments": tool_call.function.arguments or "", + "tool_call_id": tool_call.id or "", + } + ) + + if choice.finish_reason: + event_type = event_type or "stop" + finish_reason_mapping = { + "stop": "stop", + "length": "length", + "tool_calls": "tool_call", + "content_filter": "stop", + } + finish_reason = finish_reason_mapping.get(choice.finish_reason, "unknown") + + if model_output.usage: + event_type = event_type or "stop" # deal with separate usage data + + if model_output.usage.prompt_tokens_details: + cached_tokens = model_output.usage.prompt_tokens_details.cached_tokens + else: + cached_tokens = None + + if model_output.usage.completion_tokens_details: + reasoning_tokens = model_output.usage.completion_tokens_details.reasoning_tokens + else: + reasoning_tokens = None + + if cached_tokens is not None: + prompt_tokens = model_output.usage.prompt_tokens - cached_tokens + else: + prompt_tokens = model_output.usage.prompt_tokens + + if reasoning_tokens is not None: + response_tokens = model_output.usage.completion_tokens - reasoning_tokens + else: + response_tokens = model_output.usage.completion_tokens + + usage_metadata = { + "cached_tokens": cached_tokens, + "prompt_tokens": prompt_tokens, + "thoughts_tokens": reasoning_tokens, + "response_tokens": response_tokens, + } + usage_metadata = fix_openrouter_usage_metadata(usage_metadata, str(self._client.base_url)) + + return { + "role": "assistant", + "event_type": event_type, + "content_items": content_items, + "usage_metadata": usage_metadata, + "finish_reason": finish_reason, + } + + async def _streaming_response_internal( + self, + messages: list[UniMessage], + config: UniConfig, + ) -> AsyncIterator[UniEvent]: + """Stream generate using Kimi SDK with unified conversion methods.""" + kimi_config = self.transform_uni_config_to_model_config(config) + kimi_messages = await self.transform_uni_message_to_model_input(messages) + + # Extract system prompt if present + if config.get("system_prompt"): + kimi_messages.insert(0, {"role": "system", "content": config["system_prompt"]}) + + # Stream generate + stream = await self._client.chat.completions.create(**kimi_config, messages=kimi_messages) + + partial_tool_call = {} + partial_usage = {} + async for chunk in stream: + event = self.transform_model_output_to_uni_event(chunk) + # the finish reason and usage metadata should be accumulated + partial_usage["finish_reason"] = event["finish_reason"] or partial_usage.get("finish_reason") + partial_usage["usage_metadata"] = event["usage_metadata"] or partial_usage.get("usage_metadata") + if event["event_type"] == "delta": + for item in event["content_items"]: + if item["type"] == "partial_tool_call": + if not partial_tool_call: + # start new partial tool call + partial_tool_call = { + "name": item["name"], + "arguments": item["arguments"], + "tool_call_id": item["tool_call_id"], + } + elif item["name"]: + # finish previous partial tool call + yield { + "role": "assistant", + "event_type": "delta", + "content_items": [ + { + "type": "tool_call", + "name": partial_tool_call["name"], + "arguments": parse_tool_call_arguments( + partial_tool_call["arguments"], + self.__class__.__name__, + partial_tool_call["name"], + partial_tool_call["tool_call_id"], + ), + "tool_call_id": partial_tool_call["tool_call_id"], + } + ], + "usage_metadata": None, + "finish_reason": None, + } + # start new partial tool call + partial_tool_call = { + "name": item["name"], + "arguments": item["arguments"], + "tool_call_id": item["tool_call_id"], + } + else: + # update partial tool call + partial_tool_call["arguments"] += item["arguments"] + + yield event + elif event["event_type"] == "stop": + if partial_tool_call: + # finish partial tool call + yield { + "role": "assistant", + "event_type": "delta", + "content_items": [ + { + "type": "tool_call", + "name": partial_tool_call["name"], + "arguments": parse_tool_call_arguments( + partial_tool_call["arguments"], + self.__class__.__name__, + partial_tool_call["name"], + partial_tool_call["tool_call_id"], + ), + "tool_call_id": partial_tool_call["tool_call_id"], + } + ], + "usage_metadata": None, + "finish_reason": None, + } + partial_tool_call = {} + + if partial_usage.get("finish_reason") and partial_usage.get("usage_metadata"): + yield { + "role": "assistant", + "event_type": "stop", + "content_items": [], + "usage_metadata": partial_usage["usage_metadata"], + "finish_reason": partial_usage["finish_reason"], + } + partial_usage = {} diff --git a/src_py/agenthub/openai/client.py b/src_py/agenthub/openai/client.py index 369bb52f..c67d876b 100644 --- a/src_py/agenthub/openai/client.py +++ b/src_py/agenthub/openai/client.py @@ -23,7 +23,7 @@ from openai.types.chat import ChatCompletionChunk, ChatCompletionMessageParam from ..base_client import LLMClient -from ..errors import parse_tool_call_arguments +from ..errors import UnsupportedParameterError, parse_tool_call_arguments from ..types import ( EventType, FinishReason, @@ -107,7 +107,9 @@ def transform_uni_config_to_model_config(self, config: UniConfig) -> dict[str, A openai_config["tool_choice"] = self._convert_tool_choice(config["tool_choice"]) if config.get("prompt_caching") is not None and config["prompt_caching"] != PromptCaching.ENABLE: - raise ValueError("prompt_caching must be ENABLE for OpenAI.") + raise UnsupportedParameterError( + self.__class__.__name__, "prompt_caching", "prompt_caching must be ENABLE for OpenAI." + ) return openai_config diff --git a/src_py/agenthub/registry.py b/src_py/agenthub/registry.py new file mode 100644 index 00000000..8115bd21 --- /dev/null +++ b/src_py/agenthub/registry.py @@ -0,0 +1,564 @@ +# Copyright 2025 Prism Shadow. and/or its affiliates +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from typing import Literal, NotRequired, TypedDict + + +Modality = Literal["Text", "Image", "Video", "Audio", "Embed"] +Currency = Literal["USD", "CNY"] + + +class ModelPricing(TypedDict): + """List prices per million tokens for AgentHub's usage buckets. + + Keys mirror ``usage_metadata``: ``cached_tokens`` (cache-hit price, absent when the + platform publishes none), ``prompt_tokens`` (non-cached input), and + ``thoughts_tokens``/``response_tokens``, which both carry the vendor's output price. + Values are in the currency requested from ``list_supported_models``. + """ + + currency: Currency + prompt_tokens: float + thoughts_tokens: float + response_tokens: float + cached_tokens: NotRequired[float] + + +class SupportedModel(TypedDict): + """One supported model entry. + + (model, base_url, client) maps directly onto the AutoLLMClient constructor: + ``AutoLLMClient(model=entry["model"], base_url=entry["base_url"], client_type=entry["client"])``. + Modalities describe what is usable through that client; ``context_window`` and + ``pricing`` are omitted where the platform publishes no authoritative value. + """ + + model: str + base_url: str + client: str + input_modalities: list[Modality] + output_modalities: list[Modality] + context_window: NotRequired[int] + pricing: NotRequired[ModelPricing] + + +_GOOGLE = "https://generativelanguage.googleapis.com" +_ANTHROPIC = "https://api.anthropic.com" +_OPENAI = "https://api.openai.com/v1" +_ZAI = "https://api.z.ai/api/paas/v4/" +_MOONSHOT = "https://api.moonshot.cn/v1" +_DEEPSEEK = "https://api.deepseek.com" +_OPENROUTER = "https://openrouter.ai/api/v1" +_SILICONFLOW = "https://api.siliconflow.cn/v1" + +# Display convention shared with the AgentHub apps: prices are stored in USD (official CNY +# list prices pre-converted at 7 CNY/USD), so requesting CNY shows the vendor's numbers. +_CNY_PER_USD = 7.0 + + +def _usd(prompt: float, output: float, cached: float | None = None) -> ModelPricing: + # thoughts and response tokens are both billed at the vendor's output price + pricing: ModelPricing = { + "currency": "USD", + "prompt_tokens": prompt, + "thoughts_tokens": output, + "response_tokens": output, + } + if cached is not None: + pricing["cached_tokens"] = cached + return pricing + + +def _cny(prompt: float, output: float, cached: float | None = None) -> ModelPricing: + """Declare a CNY-denominated official list price; storage stays USD (converted at 7 CNY/USD).""" + + def rate(value: float) -> float: + return round(value / _CNY_PER_USD, 6) + + return _usd(rate(prompt), rate(output), rate(cached) if cached is not None else None) + + +# Prices in USD per million tokens (official CNY prices pre-converted at 7 CNY/USD); platform +# data (context windows, OpenRouter USD prices, modality flags) verified against the live +# /models APIs on 2026-07-22, SiliconFlow CNY prices from the vendors' official price lists. +_SUPPORTED_MODELS: list[SupportedModel] = [ + # official vendor endpoints + { + "model": "gemini-3.6-flash", + "base_url": _GOOGLE, + "client": "gemini-3.6", + "input_modalities": ["Text", "Image", "Video", "Audio"], + "output_modalities": ["Text"], + "context_window": 1048576, + "pricing": _usd(1.5, 7.5, cached=0.15), + }, + { + "model": "gemini-3.5-flash-lite", + "base_url": _GOOGLE, + "client": "gemini-3.6", + "input_modalities": ["Text", "Image", "Video", "Audio"], + "output_modalities": ["Text"], + "context_window": 1048576, + "pricing": _usd(0.3, 2.5, cached=0.03), + }, + { + "model": "gemini-3.5-flash", + "base_url": _GOOGLE, + "client": "gemini-3", + "input_modalities": ["Text", "Image", "Video", "Audio"], + "output_modalities": ["Text"], + "context_window": 1048576, + "pricing": _usd(1.5, 9.0, cached=0.15), + }, + { + "model": "gemini-3.1-flash-image", + "base_url": _GOOGLE, + "client": "gemini-3", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Image"], + }, + { + "model": "gemini-3.1-flash-tts-preview", + "base_url": _GOOGLE, + "client": "gemini-3", + "input_modalities": ["Text"], + "output_modalities": ["Audio"], + }, + { + "model": "gemini-embedding-2", + "base_url": _GOOGLE, + "client": "gemini-3", + "input_modalities": ["Text"], + "output_modalities": ["Embed"], + }, + { + "model": "claude-fable-5", + "base_url": _ANTHROPIC, + "client": "claude-5", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 1000000, + "pricing": _usd(10.0, 50.0, cached=1.0), + }, + { + "model": "claude-sonnet-5", + "base_url": _ANTHROPIC, + "client": "claude-5", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 1000000, + "pricing": _usd(2.0, 10.0, cached=0.2), + }, + { + "model": "claude-opus-4-8", + "base_url": _ANTHROPIC, + "client": "claude-5", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 1000000, + "pricing": _usd(5.0, 25.0, cached=0.5), + }, + { + "model": "claude-sonnet-4-6", + "base_url": _ANTHROPIC, + "client": "claude-4-6", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 1000000, + "pricing": _usd(3.0, 15.0, cached=0.3), + }, + { + "model": "gpt-5.5", + "base_url": _OPENAI, + "client": "gpt-5.5", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 1050000, + "pricing": _usd(5.0, 30.0, cached=0.5), + }, + { + "model": "text-embedding-3-large", + "base_url": _OPENAI, + "client": "openai-embedding", + "input_modalities": ["Text"], + "output_modalities": ["Embed"], + "pricing": _usd(0.13, 0.0), + }, + { + "model": "glm-5.2", + "base_url": _ZAI, + "client": "glm-5.2", + "input_modalities": ["Text"], + "output_modalities": ["Text"], + "context_window": 1000000, + "pricing": _usd(1.4, 4.4, cached=0.26), + }, + { + "model": "glm-5.1", + "base_url": _ZAI, + "client": "glm-5.1", + "input_modalities": ["Text"], + "output_modalities": ["Text"], + "context_window": 200000, + "pricing": _usd(1.4, 4.4, cached=0.26), + }, + { + "model": "kimi-k3", + "base_url": _MOONSHOT, + "client": "kimi-k3", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 1048576, + "pricing": _cny(20.0, 100.0, cached=2.0), + }, + { + "model": "kimi-k2.6", + "base_url": _MOONSHOT, + "client": "kimi-k2.6", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 262144, + "pricing": _cny(6.5, 27.0, cached=1.1), + }, + { + "model": "deepseek-v4-flash", + "base_url": _DEEPSEEK, + "client": "deepseek-v4", + "input_modalities": ["Text"], + "output_modalities": ["Text"], + "context_window": 1000000, + "pricing": _cny(1.0, 2.0, cached=0.02), + }, + { + "model": "deepseek-v4-pro", + "base_url": _DEEPSEEK, + "client": "deepseek-v4", + "input_modalities": ["Text"], + "output_modalities": ["Text"], + "context_window": 1000000, + "pricing": _cny(3.0, 6.0, cached=0.025), + }, + # OpenRouter (USD prices, context windows and modality flags from the live /models API) + { + "model": "anthropic/claude-fable-5", + "base_url": _OPENROUTER, + "client": "openai", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 1000000, + "pricing": _usd(10.0, 50.0, cached=1.0), + }, + { + "model": "anthropic/claude-opus-4.8", + "base_url": _OPENROUTER, + "client": "openai", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 1000000, + "pricing": _usd(5.0, 25.0, cached=0.5), + }, + { + "model": "anthropic/claude-opus-4.7", + "base_url": _OPENROUTER, + "client": "openai", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 1000000, + "pricing": _usd(5.0, 25.0, cached=0.5), + }, + { + "model": "anthropic/claude-sonnet-5", + "base_url": _OPENROUTER, + "client": "openai", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 1000000, + "pricing": _usd(2.0, 10.0, cached=0.2), + }, + { + "model": "deepseek/deepseek-v4-flash", + "base_url": _OPENROUTER, + "client": "deepseek-v4", + "input_modalities": ["Text"], + "output_modalities": ["Text"], + "context_window": 1048576, + "pricing": _usd(0.098, 0.196, cached=0.0196), + }, + { + "model": "deepseek/deepseek-v4-pro", + "base_url": _OPENROUTER, + "client": "deepseek-v4", + "input_modalities": ["Text"], + "output_modalities": ["Text"], + "context_window": 1048576, + "pricing": _usd(0.435, 0.87, cached=0.003625), + }, + { + "model": "google/gemini-3.5-flash", + "base_url": _OPENROUTER, + "client": "openai", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 1048576, + "pricing": _usd(1.5, 9.0, cached=0.15), + }, + { + "model": "minimax/minimax-m3", + "base_url": _OPENROUTER, + "client": "openai", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 1048576, + "pricing": _usd(0.3, 1.2, cached=0.06), + }, + { + "model": "moonshotai/kimi-k3", + "base_url": _OPENROUTER, + "client": "kimi-k3", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 1048576, + "pricing": _usd(3.0, 15.0, cached=0.3), + }, + { + "model": "moonshotai/kimi-k2.6", + "base_url": _OPENROUTER, + "client": "kimi-k2.6", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 262144, + "pricing": _usd(0.684, 3.42, cached=0.144), + }, + { + "model": "nvidia/nemotron-3-ultra-550b-a55b:free", + "base_url": _OPENROUTER, + "client": "openai", + "input_modalities": ["Text"], + "output_modalities": ["Text"], + "context_window": 1000000, + "pricing": _usd(0.0, 0.0), + }, + { + "model": "openai/gpt-5.6-sol", + "base_url": _OPENROUTER, + "client": "openai", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 1050000, + "pricing": _usd(5.0, 30.0, cached=0.5), + }, + { + "model": "openai/gpt-5.6-terra", + "base_url": _OPENROUTER, + "client": "openai", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 1050000, + "pricing": _usd(2.5, 15.0, cached=0.25), + }, + { + "model": "openai/gpt-5.5", + "base_url": _OPENROUTER, + "client": "openai", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 1050000, + "pricing": _usd(5.0, 30.0, cached=0.5), + }, + { + "model": "qwen/qwen3.6-35b-a3b", + "base_url": _OPENROUTER, + "client": "openai", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 262144, + "pricing": _usd(0.14, 1.0), + }, + { + "model": "qwen/qwen3-embedding-4b", + "base_url": _OPENROUTER, + "client": "openai-embedding", + "input_modalities": ["Text"], + "output_modalities": ["Embed"], + "context_window": 32768, + "pricing": _usd(0.02, 0.0), + }, + { + "model": "stepfun/step-3.7-flash", + "base_url": _OPENROUTER, + "client": "openai", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 262144, + "pricing": _usd(0.2, 1.15, cached=0.04), + }, + { + "model": "tencent/hy3", + "base_url": _OPENROUTER, + "client": "openai", + "input_modalities": ["Text"], + "output_modalities": ["Text"], + "context_window": 262144, + "pricing": _usd(0.14, 0.58, cached=0.035), + }, + { + "model": "x-ai/grok-4.5", + "base_url": _OPENROUTER, + "client": "openai", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 500000, + "pricing": _usd(2.0, 6.0, cached=0.3), + }, + { + "model": "xiaomi/mimo-v2.5", + "base_url": _OPENROUTER, + "client": "openai", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 1050000, + "pricing": _usd(0.14, 0.28, cached=0.0028), + }, + { + "model": "z-ai/glm-5.2", + "base_url": _OPENROUTER, + "client": "glm-5.2", + "input_modalities": ["Text"], + "output_modalities": ["Text"], + "context_window": 1048576, + "pricing": _usd(0.8204, 2.5784, cached=0.15236), + }, + { + "model": "z-ai/glm-5.1", + "base_url": _OPENROUTER, + "client": "glm-5.1", + "input_modalities": ["Text"], + "output_modalities": ["Text"], + "context_window": 204800, + "pricing": _usd(0.966, 3.036, cached=0.1794), + }, + # SiliconFlow (official CNY price lists pre-converted to USD; no public pricing API) + { + "model": "deepseek-ai/DeepSeek-V4-Flash", + "base_url": _SILICONFLOW, + "client": "deepseek-v4", + "input_modalities": ["Text"], + "output_modalities": ["Text"], + "context_window": 1000000, + "pricing": _cny(1.0, 2.0, cached=0.02), + }, + { + "model": "deepseek-ai/DeepSeek-V4-Pro", + "base_url": _SILICONFLOW, + "client": "deepseek-v4", + "input_modalities": ["Text"], + "output_modalities": ["Text"], + "context_window": 1000000, + "pricing": _cny(12.0, 24.0, cached=0.1), + }, + { + "model": "meituan-longcat/LongCat-2.0", + "base_url": _SILICONFLOW, + "client": "openai", + "input_modalities": ["Text"], + "output_modalities": ["Text"], + "context_window": 1000000, + "pricing": _cny(5.0, 20.0, cached=0.1), + }, + { + "model": "moonshotai/Kimi-K2.7-Code", + "base_url": _SILICONFLOW, + "client": "openai", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 262144, + "pricing": _cny(6.5, 27.0, cached=1.3), + }, + { + "model": "zai-org/GLM-5.2", + "base_url": _SILICONFLOW, + "client": "glm-5.2", + "input_modalities": ["Text"], + "output_modalities": ["Text"], + "context_window": 1000000, + "pricing": _cny(8.0, 28.0, cached=2.0), + }, + { + "model": "Pro/zai-org/GLM-5.1", + "base_url": _SILICONFLOW, + "client": "glm-5.1", + "input_modalities": ["Text"], + "output_modalities": ["Text"], + "context_window": 200000, + }, + { + "model": "Pro/moonshotai/Kimi-K2.6", + "base_url": _SILICONFLOW, + "client": "kimi-k2.6", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 262144, + }, + { + "model": "Qwen/Qwen3.6-35B-A3B", + "base_url": _SILICONFLOW, + "client": "openai", + "input_modalities": ["Text", "Image"], + "output_modalities": ["Text"], + "context_window": 262144, + }, + { + "model": "Qwen/Qwen3-Embedding-8B", + "base_url": _SILICONFLOW, + "client": "openai-embedding", + "input_modalities": ["Text"], + "output_modalities": ["Embed"], + }, +] + + +def _convert_pricing(pricing: ModelPricing, currency: Currency) -> ModelPricing: + if currency == "USD": + return dict(pricing) + + converted: ModelPricing = { + "currency": "CNY", + "prompt_tokens": 0.0, + "thoughts_tokens": 0.0, + "response_tokens": 0.0, + } + for key in ("prompt_tokens", "thoughts_tokens", "response_tokens", "cached_tokens"): + if key in pricing: + converted[key] = round(pricing[key] * _CNY_PER_USD, 6) + return converted + + +def list_supported_models(currency: Currency = "USD") -> list[SupportedModel]: + """List supported models with base URL, client, modalities, context window, and pricing. + + Covers the official vendor endpoints plus the OpenRouter and SiliconFlow platforms; + ``client`` is the ``client_type`` token that routes the model to its protocol client. + Prices are per million tokens for AgentHub's usage buckets (cached_tokens, + prompt_tokens, thoughts_tokens, response_tokens), stored in USD and converted to + ``currency`` at 7 CNY/USD on request. + """ + entries: list[SupportedModel] = [] + for entry in _SUPPORTED_MODELS: + copied = dict(entry) + copied["input_modalities"] = list(entry["input_modalities"]) + copied["output_modalities"] = list(entry["output_modalities"]) + if "pricing" in entry: + copied["pricing"] = _convert_pricing(entry["pricing"], currency) + entries.append(copied) + return entries diff --git a/src_py/pyproject.toml b/src_py/pyproject.toml index 7697e7ce..2a610582 100644 --- a/src_py/pyproject.toml +++ b/src_py/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "agenthub-python" -version = "0.4.0" +version = "0.4.1" description = "AgentHub is the LLM API Hub for the Agent era, built for high-precision autonomous agents." keywords = ["agent", "llm", "gemini", "claude", "gpt"] readme = "README.md" diff --git a/src_py/tests/test_client.py b/src_py/tests/test_client.py index fd690c22..c57eaeef 100644 --- a/src_py/tests/test_client.py +++ b/src_py/tests/test_client.py @@ -23,7 +23,7 @@ import httpx import pytest -from agenthub import AutoLLMClient, ThinkingLevel +from agenthub import AutoLLMClient, ThinkingLevel, list_supported_models IMAGE = "https://cdn.britannica.com/80/120980-050-D1DA5C61/Poet-narcissus.jpg" @@ -49,10 +49,10 @@ def __repr__(self) -> str: AVAILABLE_MODELS: list[Model] = [] if os.getenv("GEMINI_API_KEY"): - AVAILABLE_MODELS.append(Model(name="gemini-3.5-flash")) + AVAILABLE_MODELS.append(Model(name="gemini-3.6-flash", support_temperature=False)) AVAILABLE_MODELS.append( Model( - name="gemini-3.1-flash-image-preview", + name="gemini-3.1-flash-image", support_text=False, support_temperature=False, support_image_understanding=False, @@ -95,10 +95,10 @@ def __repr__(self) -> str: ) if os.getenv("ZAI_API_KEY"): - AVAILABLE_MODELS.append(Model(name="glm-5.1", support_image_understanding=False)) + AVAILABLE_MODELS.append(Model(name="glm-5.2", support_image_understanding=False)) if os.getenv("MOONSHOT_API_KEY"): - AVAILABLE_MODELS.append(Model(name="kimi-k2.6", support_temperature=False)) + AVAILABLE_MODELS.append(Model(name="kimi-k3", support_temperature=False)) if os.getenv("DEEPSEEK_API_KEY"): AVAILABLE_MODELS.append( @@ -109,10 +109,10 @@ def __repr__(self) -> str: AVAILABLE_MODELS.append(Model(name="global.anthropic.claude-sonnet-4-6", provider="bedrock")) if os.getenv("VERTEX_API_KEY"): - AVAILABLE_MODELS.append(Model(name="gemini-3.5-flash", provider="vertex")) + AVAILABLE_MODELS.append(Model(name="gemini-3.6-flash", provider="vertex", support_temperature=False)) AVAILABLE_MODELS.append( Model( - name="gemini-3.1-flash-image-preview", + name="gemini-3.1-flash-image", provider="vertex", support_text=False, support_temperature=False, @@ -134,7 +134,7 @@ def __repr__(self) -> str: RUN_SLOW_TEST = os.getenv("RUN_SLOW_TEST", "0") == "1" if os.getenv("OPENROUTER_API_KEY") and RUN_SLOW_TEST: - AVAILABLE_MODELS.append(Model(name="z-ai/glm-5.1", provider="openrouter", support_image_understanding=False)) + AVAILABLE_MODELS.append(Model(name="z-ai/glm-5.2", provider="openrouter", support_image_understanding=False)) AVAILABLE_MODELS.append(Model(name="qwen/qwen3.6-35b-a3b", provider="openrouter", client_type="openai")) AVAILABLE_MODELS.append( Model( @@ -147,12 +147,10 @@ def __repr__(self) -> str: client_type="openai-embedding", ) ) - AVAILABLE_MODELS.append(Model(name="moonshotai/kimi-k2.6", provider="openrouter", support_temperature=False)) + AVAILABLE_MODELS.append(Model(name="moonshotai/kimi-k3", provider="openrouter", support_temperature=False)) if os.getenv("SILICONFLOW_API_KEY") and RUN_SLOW_TEST: - AVAILABLE_MODELS.append( - Model(name="Pro/zai-org/GLM-5.1", provider="siliconflow", support_image_understanding=False) - ) + AVAILABLE_MODELS.append(Model(name="zai-org/GLM-5.2", provider="siliconflow", support_image_understanding=False)) AVAILABLE_MODELS.append(Model(name="Qwen/Qwen3.6-35B-A3B", provider="siliconflow", client_type="openai")) AVAILABLE_MODELS.append(Model(name="Pro/moonshotai/Kimi-K2.6", provider="siliconflow", support_temperature=False)) AVAILABLE_MODELS.append( @@ -396,6 +394,44 @@ async def test_unknown_model(): AutoLLMClient(model="unknown-model") +@pytest.mark.asyncio +async def test_list_supported_models(): + """Test that the registry lists model entries accepted by AutoLLMClient.""" + entries = list_supported_models() + kimi = next(entry for entry in entries if entry["model"] == "kimi-k3") + assert kimi["base_url"] == "https://api.moonshot.cn/v1" + assert kimi["client"] == "kimi-k3" + assert kimi["context_window"] == 1048576 + assert kimi["input_modalities"] == ["Text", "Image"] + assert kimi["output_modalities"] == ["Text"] + # stored in USD (official CNY prices pre-converted at 7 CNY/USD) + assert kimi["pricing"] == { + "currency": "USD", + "prompt_tokens": 2.857143, + "thoughts_tokens": 14.285714, + "response_tokens": 14.285714, + "cached_tokens": 0.285714, + } + + kimi_cny = next(entry for entry in list_supported_models(currency="CNY") if entry["model"] == "kimi-k3") + assert kimi_cny["pricing"]["currency"] == "CNY" + assert kimi_cny["pricing"]["prompt_tokens"] == pytest.approx(20.0, abs=1e-4) + assert kimi_cny["pricing"]["thoughts_tokens"] == pytest.approx(100.0, abs=1e-4) + assert kimi_cny["pricing"]["response_tokens"] == pytest.approx(100.0, abs=1e-4) + assert kimi_cny["pricing"]["cached_tokens"] == pytest.approx(2.0, abs=1e-4) + + glm_5_2 = next(entry for entry in entries if entry["model"] == "z-ai/glm-5.2") + assert glm_5_2["base_url"] == "https://openrouter.ai/api/v1" + assert glm_5_2["client"] == "glm-5.2" + + for entry in entries: + assert {"model", "base_url", "client", "input_modalities", "output_modalities"} <= set(entry) + client = AutoLLMClient( + model=entry["model"], api_key="test-key", base_url=entry["base_url"], client_type=entry["client"] + ) + assert client._client is not None + + @pytest.mark.asyncio @pytest.mark.parametrize( ("client_type", "client_name"), diff --git a/src_ts/package-lock.json b/src_ts/package-lock.json index 9107deaf..1bd6bb0b 100644 --- a/src_ts/package-lock.json +++ b/src_ts/package-lock.json @@ -1,12 +1,12 @@ { "name": "@prismshadow/agenthub", - "version": "0.4.0", + "version": "0.4.1", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@prismshadow/agenthub", - "version": "0.4.0", + "version": "0.4.1", "license": "Apache-2.0", "dependencies": { "@anthropic-ai/bedrock-sdk": "^0.26.4", diff --git a/src_ts/package.json b/src_ts/package.json index 92608086..58ec8401 100644 --- a/src_ts/package.json +++ b/src_ts/package.json @@ -1,6 +1,6 @@ { "name": "@prismshadow/agenthub", - "version": "0.4.0", + "version": "0.4.1", "description": "AgentHub is the LLM API Hub for the Agent era, built for high-precision autonomous agents.", "main": "dist/index.js", "types": "dist/index.d.ts", diff --git a/src_ts/src/autoClient.ts b/src_ts/src/autoClient.ts index de319408..d4a491c4 100644 --- a/src_ts/src/autoClient.ts +++ b/src_ts/src/autoClient.ts @@ -14,11 +14,14 @@ import { LLMClient } from "./baseClient"; import { Gemini3Client } from "./gemini3"; +import { Gemini3_6Client } from "./gemini3_6"; import { Claude4_6Client } from "./claude4_6"; import { Claude5Client } from "./claude5"; import { GPT5_5Client } from "./gpt5_5"; import { GLM5_1Client } from "./glm5_1"; +import { GLM5_2Client } from "./glm5_2"; import { KimiK2_6Client } from "./kimi_k2_6"; +import { KimiK3Client } from "./kimi_k3"; import { OpenaiClient } from "./openai"; import { OpenaiEmbeddingClient } from "./openai_embedding"; import { DeepSeekV4Client } from "./deepseek_v4"; @@ -75,7 +78,14 @@ export class AutoLLMClient extends LLMClient { model.toLowerCase() ).toLowerCase(); + // gemini-3.6 must be matched before the broader gemini-3 prefix below if ( + clientType.includes("gemini-3.6") || + clientType.includes("gemini-3.5-flash-lite") + ) { + // gemini-3.5-flash-lite shares the sampling-parameter deprecation with gemini-3.6 + return new Gemini3_6Client({ model, apiKey, baseUrl }); + } else if ( clientType.includes("gemini-3") || clientType.includes("gemini-embedding") ) { @@ -94,8 +104,12 @@ export class AutoLLMClient extends LLMClient { clientType.includes("gpt-5.5") ) { return new GPT5_5Client({ model, apiKey, baseUrl }); + } else if (clientType.includes("glm-5.2")) { + return new GLM5_2Client({ model, apiKey, baseUrl }); } else if (clientType.includes("glm-5") || clientType.includes("glm-5.1")) { return new GLM5_1Client({ model, apiKey, baseUrl }); + } else if (clientType.includes("kimi-k3")) { + return new KimiK3Client({ model, apiKey, baseUrl }); } else if ( clientType.includes("kimi-k2.5") || clientType.includes("kimi-k2.6") @@ -116,7 +130,7 @@ export class AutoLLMClient extends LLMClient { } else { throw new Error( `${clientType} is not supported. ` + - "Supported client types: gemini-3, claude-5, claude-4-8, claude-4-7, claude-4-6, gpt-5.5, gpt-5.4, glm-5.1, kimi-k2.6, kimi-k2.5, deepseek-v4, openai-embedding, openai.", + "Supported client types: gemini-3.6, gemini-3, claude-5, claude-4-8, claude-4-7, claude-4-6, gpt-5.5, gpt-5.4, glm-5.2, glm-5.1, kimi-k3, kimi-k2.6, kimi-k2.5, deepseek-v4, openai-embedding, openai.", ); } } diff --git a/src_ts/src/claude4_6/client.ts b/src_ts/src/claude4_6/client.ts index 97ddf7f5..f075e1d4 100644 --- a/src_ts/src/claude4_6/client.ts +++ b/src_ts/src/claude4_6/client.ts @@ -20,7 +20,10 @@ import { } from "@anthropic-ai/sdk/resources/beta/messages"; import { Stream } from "@anthropic-ai/sdk/core/streaming"; import { LLMClient } from "../baseClient"; -import { parseToolCallArguments } from "../errors"; +import { + parseToolCallArguments, + UnsupportedParameterError, +} from "../errors"; import { EventType, FinishReason, @@ -169,7 +172,11 @@ export class Claude4_6Client extends LLMClient { private _convertToolChoice(toolChoice: ToolChoice): any { if (Array.isArray(toolChoice)) { if (toolChoice.length > 1) { - throw new Error("Claude supports only one tool choice."); + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "tool_choice", + message: "Claude supports only one tool choice.", + }); } return { type: "any", name: toolChoice[0] }; } else if (toolChoice === "none") { diff --git a/src_ts/src/claude5/client.ts b/src_ts/src/claude5/client.ts index af32daa5..6df906c1 100644 --- a/src_ts/src/claude5/client.ts +++ b/src_ts/src/claude5/client.ts @@ -20,7 +20,10 @@ import { } from "@anthropic-ai/sdk/resources/beta/messages"; import { Stream } from "@anthropic-ai/sdk/core/streaming"; import { LLMClient } from "../baseClient"; -import { parseToolCallArguments } from "../errors"; +import { + parseToolCallArguments, + UnsupportedParameterError, +} from "../errors"; import { EventType, FinishReason, @@ -169,7 +172,11 @@ export class Claude5Client extends LLMClient { private _convertToolChoice(toolChoice: ToolChoice): any { if (Array.isArray(toolChoice)) { if (toolChoice.length > 1) { - throw new Error("Claude supports only one tool choice."); + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "tool_choice", + message: "Claude supports only one tool choice.", + }); } return { type: "any", name: toolChoice[0] }; } else if (toolChoice === "none") { @@ -203,7 +210,11 @@ export class Claude5Client extends LLMClient { } if (config.temperature !== undefined && config.temperature !== 1.0) { - throw new Error("Claude 4.8 does not support setting temperature."); + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "temperature", + message: "Claude 4.8 does not support setting temperature.", + }); } if (config.thinking_level !== undefined) { diff --git a/src_ts/src/deepseek_v4/client.ts b/src_ts/src/deepseek_v4/client.ts index da3fa343..e90ced26 100644 --- a/src_ts/src/deepseek_v4/client.ts +++ b/src_ts/src/deepseek_v4/client.ts @@ -19,7 +19,10 @@ import type { ChatCompletionCreateParamsStreaming, } from "openai/resources/chat/completions"; import { LLMClient } from "../baseClient"; -import { parseToolCallArguments } from "../errors"; +import { + parseToolCallArguments, + UnsupportedParameterError, +} from "../errors"; import { EventType, FinishReason, @@ -81,9 +84,11 @@ export class DeepSeekV4Client extends LLMClient { if (toolChoice === "auto" || toolChoice === "none") { return toolChoice; } - throw new Error( - 'DeepSeek V4 only supports "auto" and "none" for tool_choice.', - ); + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "tool_choice", + message: 'DeepSeek V4 only supports "auto" and "none" for tool_choice.', + }); } // eslint-disable-next-line @typescript-eslint/no-explicit-any @@ -100,7 +105,11 @@ export class DeepSeekV4Client extends LLMClient { } if (config.temperature !== undefined && config.temperature !== 1.0) { - throw new Error("DeepSeek V4 does not support setting temperature."); + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "temperature", + message: "DeepSeek V4 does not support setting temperature.", + }); } if (config.thinking_level !== undefined) { @@ -134,7 +143,11 @@ export class DeepSeekV4Client extends LLMClient { config.prompt_caching !== undefined && config.prompt_caching !== PromptCaching.ENABLE ) { - throw new Error("prompt_caching must be ENABLE for DeepSeek."); + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "prompt_caching", + message: "prompt_caching must be ENABLE for DeepSeek.", + }); } return deepseekConfig; diff --git a/src_ts/src/errors.ts b/src_ts/src/errors.ts index 93045e9f..5b584745 100644 --- a/src_ts/src/errors.ts +++ b/src_ts/src/errors.ts @@ -28,6 +28,25 @@ export class AgentHubError extends Error { } } +/** + * Raised when a UniConfig parameter value is not supported by the target model client. + * + * Thinking levels never raise this by design: every client maps each ThinkingLevel + * onto the closest level the model supports. Parameters such as temperature and + * tool_choice may reject unsupported values with this error. + */ +export class UnsupportedParameterError extends AgentHubError { + readonly client: string; + readonly parameter: string; + + constructor(args: { client: string; parameter: string; message: string }) { + super(args.message); + this.name = "UnsupportedParameterError"; + this.client = args.client; + this.parameter = args.parameter; + } +} + /** * Raised when a completed response carries no non-thinking content and no tool calls. * diff --git a/src_ts/src/gemini3/client.ts b/src_ts/src/gemini3/client.ts index 9be71ef9..793c6d0a 100644 --- a/src_ts/src/gemini3/client.ts +++ b/src_ts/src/gemini3/client.ts @@ -38,6 +38,7 @@ import { } from "@google/genai"; import * as path from "path"; import { LLMClient } from "../baseClient"; +import { UnsupportedParameterError } from "../errors"; import { EventType, Fidelity, @@ -256,7 +257,11 @@ export class Gemini3Client extends LLMClient { config.prompt_caching !== undefined && config.prompt_caching !== PromptCaching.ENABLE ) { - throw new Error("prompt_caching must be ENABLE for Gemini 3."); + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "prompt_caching", + message: "prompt_caching must be ENABLE for Gemini 3.", + }); } if (config.image_config !== undefined) { diff --git a/src_ts/src/gemini3_6/client.ts b/src_ts/src/gemini3_6/client.ts new file mode 100644 index 00000000..36a922df --- /dev/null +++ b/src_ts/src/gemini3_6/client.ts @@ -0,0 +1,646 @@ +// Copyright 2025 Prism Shadow. and/or its affiliates +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +import { + GoogleGenAI, + Content, + GenerateContentConfig, + ImageConfig as GeminiImageConfig, + MultiSpeakerVoiceConfig, + Part, + PrebuiltVoiceConfig, + FunctionCall, + ThinkingConfig, + ThinkingLevel as GeminiThinkingLevel, + FunctionCallingConfig, + SpeakerVoiceConfig, + SpeechConfig, + Tool, + ToolConfig, + GenerateContentResponse, + FinishReason as GeminiFinishReason, + FunctionResponsePart, + FunctionResponseBlob, + FunctionResponse, + VoiceConfig, + EmbedContentConfig, +} from "@google/genai"; +import * as path from "path"; +import { LLMClient } from "../baseClient"; +import { UnsupportedParameterError } from "../errors"; +import { + EventType, + Fidelity, + FinishReason, + PartialContentItem, + PromptCaching, + ThinkingLevel, + ToolChoice, + UniConfig, + UniEvent, + UniMessage, + UsageMetadata, +} from "../types"; + +/** + * Wrap a part's thought signature as a fidelity payload, or nothing when absent. + */ +function partFidelity(part: Part): { fidelity?: Fidelity } { + if (part.thoughtSignature == null) { + return {}; + } + return { fidelity: { signature: part.thoughtSignature } }; +} + +/** + * Read the thought signature recorded in an item's fidelity payload. + */ +function itemThoughtSignature(item: { + fidelity?: Fidelity; +}): string | undefined { + return item.fidelity?.signature; +} + +/** + * Client for the Gemini 3.6 protocol generation (gemini-3.6-*, + * gemini-3.5-flash-lite). + * + * Starting with these models the API deprecates the temperature/top_p/top_k + * sampling parameters (silently ignored today, HTTP 400 in future + * generations), so this client rejects them instead of sending a no-op. + */ +export class Gemini3_6Client extends LLMClient { + protected _model: string; + private _client: GoogleGenAI; + + /** + * Initialize Gemini 3.6 client with model and API key. + */ + constructor(options: { + model: string; + apiKey?: string; + baseUrl?: string | null; + clientType?: string | null; + }) { + super(); + this._model = options.model; + const key = options.apiKey || process.env.GEMINI_API_KEY || undefined; + const url = options.baseUrl || process.env.GEMINI_BASE_URL || undefined; + const httpOptions = url ? { baseUrl: url } : undefined; + if (key && key.startsWith("{")) { + const credentials = JSON.parse(key); + const googleAuthOptions = { + credentials, + scopes: ["https://www.googleapis.com/auth/cloud-platform"], + }; + this._client = new GoogleGenAI({ + vertexai: true, + location: "global", + project: credentials.project_id, + googleAuthOptions, + httpOptions, + }); + } else { + this._client = new GoogleGenAI({ + apiKey: key, + httpOptions, + }); + } + } + + /** + * Detect MIME type from URL extension for image. + */ + private _detectImageMimeType(url: string): string { + const ext = path.extname(url).toLowerCase(); + const mimeTypes: { [key: string]: string } = { + ".bmp": "image/bmp", + ".gif": "image/gif", + ".jpg": "image/jpeg", + ".jpeg": "image/jpeg", + ".png": "image/png", + ".svg": "image/svg+xml", + ".tiff": "image/tiff", + ".webp": "image/webp", + }; + return mimeTypes[ext] || "image/jpeg"; + } + + /** + * Get image bytes and MIME type from URL. + */ + private async _getImageBytesAndMimeType( + url: string, + signal?: AbortSignal, + ): Promise<{ data: Buffer; mimeType: string }> { + if (url.startsWith("data:")) { + const match = url.match(/^data:([^;]+);base64,(.+)$/); + if (match) { + const mimeType = match[1]; + const base64Data = match[2]; + const data = Buffer.from(base64Data, "base64"); + return { data, mimeType }; + } else { + throw new Error(`Invalid base64 image: ${url}`); + } + } else { + const response = await fetch(url, { signal }); + if (!response.ok) { + throw new Error(`Failed to fetch image: ${url}`); + } + const arrayBuffer = await response.arrayBuffer(); + const data = Buffer.from(arrayBuffer); + const mimeType = this._detectImageMimeType(url); + return { data, mimeType }; + } + } + + /** + * Convert ThinkingLevel enum to Gemini's ThinkingLevel. + */ + private _convertThinkingLevel( + thinkingLevel: ThinkingLevel | undefined, + ): GeminiThinkingLevel | undefined { + if (!thinkingLevel) return undefined; + + const mapping: { [key: string]: GeminiThinkingLevel } = { + [ThinkingLevel.NONE]: GeminiThinkingLevel.MINIMAL, + [ThinkingLevel.LOW]: GeminiThinkingLevel.LOW, + [ThinkingLevel.MEDIUM]: GeminiThinkingLevel.MEDIUM, + [ThinkingLevel.HIGH]: GeminiThinkingLevel.HIGH, + [ThinkingLevel.XHIGH]: GeminiThinkingLevel.HIGH, + }; + return mapping[thinkingLevel]; + } + + /** + * Convert ToolChoice to Gemini's tool config. + */ + private _convertToolChoice( + toolChoice: ToolChoice, + ): FunctionCallingConfig | undefined { + if (Array.isArray(toolChoice)) { + return { + mode: "ANY", + allowedFunctionNames: toolChoice, + } as FunctionCallingConfig; + } else if (toolChoice === "none") { + return { mode: "NONE" } as FunctionCallingConfig; + } else if (toolChoice === "auto") { + return { mode: "AUTO" } as FunctionCallingConfig; + } else if (toolChoice === "required") { + return { mode: "ANY" } as FunctionCallingConfig; + } + return undefined; + } + + private _withAbortSignal( + config: T | undefined, + signal?: AbortSignal, + ): T | undefined { + if (!signal) { + return config; + } + return { ...(config ?? {}), abortSignal: signal } as T; + } + + /** + * Transform universal configuration to Gemini-specific configuration. + */ + transformUniConfigToModelConfig( + config: UniConfig, + ): GenerateContentConfig | undefined { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const configParams: any = {}; + + if (config.system_prompt !== undefined) { + configParams.systemInstruction = config.system_prompt; + } + + if (config.max_tokens !== undefined) { + configParams.maxOutputTokens = config.max_tokens; + } + + if (config.temperature !== undefined) { + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "temperature", + message: + "Gemini 3.6 generation models do not support setting temperature.", + }); + } + + const thinkingSummary = config.thinking_summary; + const thinkingLevel = config.thinking_level; + if (thinkingSummary !== undefined || thinkingLevel !== undefined) { + configParams.thinkingConfig = { + includeThoughts: thinkingSummary, + thinkingLevel: this._convertThinkingLevel(thinkingLevel), + } as ThinkingConfig; + } + + if (config.tools !== undefined) { + configParams.tools = [{ functionDeclarations: config.tools } as Tool]; + const toolChoice = config.tool_choice; + if (toolChoice !== undefined) { + const toolConfig = this._convertToolChoice(toolChoice); + if (toolConfig) { + configParams.toolConfig = { + functionCallingConfig: toolConfig, + } as ToolConfig; + } + } + } + + if ( + config.prompt_caching !== undefined && + config.prompt_caching !== PromptCaching.ENABLE + ) { + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "prompt_caching", + message: "prompt_caching must be ENABLE for Gemini 3.6.", + }); + } + + if (config.image_config !== undefined) { + configParams.imageConfig = { + aspectRatio: config.image_config.aspect_ratio, + imageSize: config.image_config.image_size, + } as GeminiImageConfig; + } + + const isTtsModel = this._model.toLowerCase().includes("tts"); + if (isTtsModel) { + configParams.responseModalities = ["AUDIO"]; + const ttsConfig = config.tts_config ?? [{ voice: "Kore" }]; + if (![1, 2].includes(ttsConfig.length)) { + throw new Error("tts_config must contain 1 or 2 entries."); + } + + if (ttsConfig.length === 1) { + configParams.speechConfig = { + voiceConfig: { + prebuiltVoiceConfig: { + voiceName: ttsConfig[0].voice, + } as PrebuiltVoiceConfig, + } as VoiceConfig, + } as SpeechConfig; + } else { + const speakerVoiceConfigs = ttsConfig.map((speakerConfig) => { + if (!speakerConfig.speaker) { + throw new Error( + "speaker is required when tts_config has 2 entries.", + ); + } + + return { + speaker: speakerConfig.speaker, + voiceConfig: { + prebuiltVoiceConfig: { + voiceName: speakerConfig.voice, + } as PrebuiltVoiceConfig, + } as VoiceConfig, + } as SpeakerVoiceConfig; + }); + + configParams.speechConfig = { + multiSpeakerVoiceConfig: { + speakerVoiceConfigs, + } as MultiSpeakerVoiceConfig, + } as SpeechConfig; + } + } + + return Object.keys(configParams).length > 0 + ? (configParams as GenerateContentConfig) + : undefined; + } + + /** + * Transform universal message format to Gemini's Content format. + */ + async transformUniMessageToModelInput( + messages: UniMessage[], + signal?: AbortSignal, + ): Promise { + const mapping: { [key: string]: string } = { + user: "user", + assistant: "model", + }; + + const contents: Content[] = []; + for (const msg of messages) { + const parts: Part[] = []; + for (const item of msg.content_items) { + if (item.type === "text") { + parts.push({ + text: item.text, + thoughtSignature: itemThoughtSignature(item), + } as Part); + } else if (item.type === "image_url") { + const urlValue = item.image_url; + const imageData = await this._getImageBytesAndMimeType( + urlValue, + signal, + ); + parts.push({ + inlineData: { + mimeType: imageData.mimeType, + data: imageData.data.toString("base64"), + }, + } as Part); + } else if (item.type === "inline_data") { + parts.push({ + inlineData: { + mimeType: item.mime_type, + data: item.data.toString("base64"), + }, + thoughtSignature: itemThoughtSignature(item), + } as Part); + } else if (item.type === "thinking") { + parts.push({ + text: item.thinking, + thought: true, + thoughtSignature: itemThoughtSignature(item), + } as Part); + } else if (item.type === "inline_thinking") { + parts.push({ + inlineData: { + mimeType: item.mime_type, + data: item.data.toString("base64"), + }, + thought: true, + thoughtSignature: itemThoughtSignature(item), + } as Part); + } else if (item.type === "tool_call") { + const functionCall: FunctionCall = { + name: item.name, + args: item.arguments, + }; + parts.push({ + functionCall: functionCall, + thoughtSignature: itemThoughtSignature(item), + } as Part); + } else if (item.type === "tool_result") { + if (!item.tool_call_id) { + throw new Error("tool_call_id is required for tool result."); + } + + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const toolResult: Record = { result: item.text }; + const multimodalParts: FunctionResponsePart[] = []; + + if (item.images) { + for (const imageUrl of item.images) { + const imageData = await this._getImageBytesAndMimeType( + imageUrl, + signal, + ); + multimodalParts.push({ + inlineData: { + mimeType: imageData.mimeType, + data: imageData.data.toString("base64"), + } as FunctionResponseBlob, + } as FunctionResponsePart); + } + } + + parts.push({ + functionResponse: { + name: item.tool_call_id, + response: toolResult, + parts: multimodalParts.length > 0 ? multimodalParts : undefined, + } as FunctionResponse, + } as Part); + } else { + throw new Error(`Unknown item: ${JSON.stringify(item)}`); + } + } + + contents.push({ + role: mapping[msg.role], + parts: parts, + } as Content); + } + + return contents; + } + + /** + * Transform Gemini model output to universal event format. + */ + transformModelOutputToUniEvent( + modelOutput: GenerateContentResponse, + ): UniEvent { + let eventType: EventType = "delta"; + const contentItems: PartialContentItem[] = []; + let usageMetadata: UsageMetadata | null = null; + let finishReason: FinishReason | null = null; + + if ( + modelOutput.candidates?.length !== undefined && + modelOutput.candidates?.length > 0 + ) { + const candidate = modelOutput.candidates?.[0]; + for (const part of candidate.content?.parts || []) { + if (part.functionCall) { + contentItems.push({ + type: "tool_call", + name: part.functionCall.name || "", + arguments: part.functionCall.args || {}, + tool_call_id: part.functionCall.name || "", + ...partFidelity(part), + }); + } else if (part.thought) { + if (part.text !== undefined) { + contentItems.push({ + type: "thinking", + thinking: part.text, + ...partFidelity(part), + }); + } else if (part.inlineData) { + contentItems.push({ + type: "inline_thinking", + data: Buffer.from(part.inlineData.data || "", "base64"), + mime_type: part.inlineData.mimeType || "application/octet-stream", + ...partFidelity(part), + }); + } + } else if (part.inlineData) { + contentItems.push({ + type: "inline_data", + data: Buffer.from(part.inlineData.data || "", "base64"), + mime_type: part.inlineData.mimeType || "application/octet-stream", + ...partFidelity(part), + }); + } else if (part.text !== undefined) { + contentItems.push({ + type: "text", + text: part.text, + ...partFidelity(part), + }); + } else { + throw new Error(`Unknown output: ${JSON.stringify(part)}`); + } + } + + if (candidate.finishReason) { + eventType = "stop"; + const stopReasonMapping: { [key: string]: FinishReason } = { + [GeminiFinishReason.STOP]: "stop", + [GeminiFinishReason.MAX_TOKENS]: "length", + }; + finishReason = stopReasonMapping[candidate.finishReason] || "unknown"; + } + } + + if (modelOutput.usageMetadata) { + eventType = eventType || "delta"; // deal with separate usage data + + const promptTokens = modelOutput.usageMetadata.promptTokenCount || 0; + const cachedTokens = + modelOutput.usageMetadata.cachedContentTokenCount || 0; + usageMetadata = { + cached_tokens: + modelOutput.usageMetadata.cachedContentTokenCount || null, + prompt_tokens: promptTokens - cachedTokens, + thoughts_tokens: modelOutput.usageMetadata.thoughtsTokenCount || null, + response_tokens: modelOutput.usageMetadata.candidatesTokenCount || null, + }; + } + + return { + role: "assistant", + event_type: eventType, + content_items: contentItems, + usage_metadata: usageMetadata, + finish_reason: finishReason, + }; + } + + private async *_embedMessagesInternal(options: { + messages: UniMessage[]; + config: UniConfig; + signal?: AbortSignal; + }): AsyncGenerator { + // Embed transformed messages and return them as a streaming event. + const contents = await this.transformUniMessageToModelInput( + options.messages, + options.signal, + ); + + const geminiConfig = this._withAbortSignal( + options.config.embedding_config?.dimensions != null + ? { + outputDimensionality: options.config.embedding_config.dimensions, + } + : undefined, + options.signal, + ); + + const result = await this._client.models.embedContent({ + model: this._model, + contents, + config: geminiConfig, + }); + + yield { + role: "assistant", + event_type: "stop", + content_items: + result.embeddings?.map((embedding) => ({ + type: "embedding", + embedding: embedding.values ?? [], + })) ?? [], + usage_metadata: { + cached_tokens: null, + prompt_tokens: result.metadata?.billableCharacterCount ?? null, + thoughts_tokens: null, + response_tokens: null, + }, + finish_reason: "stop", + }; + } + + /** + * Stream generate using Gemini SDK with unified conversion methods. + */ + async *_streamingResponseInternal(options: { + messages: UniMessage[]; + config: UniConfig; + signal?: AbortSignal; + }): AsyncGenerator { + if (this._model.toLowerCase().includes("embedding")) { + for await (const event of this._embedMessagesInternal(options)) { + yield event; + } + return; + } + + // check if all items are text for tts model + const isTtsModel = this._model.toLowerCase().includes("tts"); + if (isTtsModel) { + const invalidItem = options.messages + .flatMap((message) => message.content_items) + .find((item) => item.type !== "text"); + if (invalidItem) { + throw new Error( + `Gemini TTS only supports text input, got content item type=${JSON.stringify(invalidItem.type)}.`, + ); + } + } + + const geminiConfig = this._withAbortSignal( + this.transformUniConfigToModelConfig(options.config), + options.signal, + ); + const contents = await this.transformUniMessageToModelInput( + options.messages, + options.signal, + ); + + const responseStream = await this._client.models.generateContentStream({ + model: this._model, + contents: contents, + config: geminiConfig, + }); + + for await (const chunk of responseStream) { + const event = this.transformModelOutputToUniEvent(chunk); + for (const item of event.content_items) { + if (item.type === "tool_call") { + // gemini 3.6 does not support partial tool call, mock a partial tool call event + yield { + role: "assistant", + event_type: "delta", + content_items: [ + { + type: "partial_tool_call", + name: item.name, + arguments: JSON.stringify(item.arguments), + tool_call_id: item.tool_call_id, + fidelity: item.fidelity, + }, + ], + usage_metadata: null, + finish_reason: null, + }; + } + } + + yield event; + } + } +} diff --git a/src_ts/src/gemini3_6/index.ts b/src_ts/src/gemini3_6/index.ts new file mode 100644 index 00000000..e0151401 --- /dev/null +++ b/src_ts/src/gemini3_6/index.ts @@ -0,0 +1,15 @@ +// Copyright 2025 Prism Shadow. and/or its affiliates +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +export { Gemini3_6Client } from "./client"; diff --git a/src_ts/src/glm5_1/client.ts b/src_ts/src/glm5_1/client.ts index a38188bb..27c353da 100644 --- a/src_ts/src/glm5_1/client.ts +++ b/src_ts/src/glm5_1/client.ts @@ -19,7 +19,10 @@ import type { ChatCompletionCreateParamsStreaming, } from "openai/resources/chat/completions"; import { LLMClient } from "../baseClient"; -import { parseToolCallArguments } from "../errors"; +import { + parseToolCallArguments, + UnsupportedParameterError, +} from "../errors"; import { EventType, FinishReason, @@ -86,7 +89,11 @@ export class GLM5_1Client extends LLMClient { if (toolChoice === "auto") { return "auto"; } else { - throw new Error('GLM only supports "auto" for tool_choice.'); + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "tool_choice", + message: 'GLM only supports "auto" for tool_choice.', + }); } } @@ -135,7 +142,11 @@ export class GLM5_1Client extends LLMClient { config.prompt_caching !== undefined && config.prompt_caching !== PromptCaching.ENABLE ) { - throw new Error("prompt_caching must be ENABLE for GLM."); + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "prompt_caching", + message: "prompt_caching must be ENABLE for GLM.", + }); } return glmConfig; diff --git a/src_ts/src/glm5_2/client.ts b/src_ts/src/glm5_2/client.ts new file mode 100644 index 00000000..97b1ccef --- /dev/null +++ b/src_ts/src/glm5_2/client.ts @@ -0,0 +1,512 @@ +// Copyright 2025 Prism Shadow. and/or its affiliates +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +import OpenAI from "openai"; +import type { + ChatCompletionChunk, + ChatCompletionMessageParam, + ChatCompletionCreateParamsStreaming, +} from "openai/resources/chat/completions"; +import { LLMClient } from "../baseClient"; +import { + parseToolCallArguments, + UnsupportedParameterError, +} from "../errors"; +import { + EventType, + FinishReason, + PartialContentItem, + PromptCaching, + ThinkingLevel, + ToolChoice, + UniConfig, + UniEvent, + UniMessage, + UsageMetadata, +} from "../types"; +import { fixOpenrouterUsageMetadata } from "../utils"; + +/** + * GLM-5.2-specific LLM client implementation using OpenAI-compatible API. + */ +export class GLM5_2Client extends LLMClient { + protected _model: string; + private _client: OpenAI; + + /** + * Initialize GLM-5.2 client with model and API key. + */ + constructor(options: { + model: string; + apiKey?: string; + baseUrl?: string | null; + clientType?: string | null; + }) { + super(); + this._model = options.model; + const key = options.apiKey || process.env.ZAI_API_KEY || undefined; + const url = + options.baseUrl || + process.env.ZAI_BASE_URL || + "https://api.z.ai/api/paas/v4/"; + this._client = new OpenAI({ apiKey: key, baseURL: url }); + } + + /** + * Convert ThinkingLevel enum to GLM-5.2's thinking configuration. + */ + private _convertThinkingLevelToConfig(thinkingLevel: ThinkingLevel): { + type: string; + clear_thinking?: boolean; + } { + const mapping: { + [key: string]: { type: string; clear_thinking?: boolean }; + } = { + [ThinkingLevel.NONE]: { type: "disabled" }, + [ThinkingLevel.LOW]: { type: "enabled", clear_thinking: false }, + [ThinkingLevel.MEDIUM]: { type: "enabled", clear_thinking: false }, + [ThinkingLevel.HIGH]: { type: "enabled", clear_thinking: false }, + [ThinkingLevel.XHIGH]: { type: "enabled", clear_thinking: false }, + }; + return mapping[thinkingLevel]; + } + + /** + * Convert ThinkingLevel enum to GLM-5.2's reasoning_effort. + * + * The server maps low/medium to high and xhigh to max; NONE disables + * thinking instead. + */ + private _convertThinkingLevelToReasoningEffort( + thinkingLevel: ThinkingLevel, + ): string | undefined { + const mapping: { [key: string]: string } = { + [ThinkingLevel.LOW]: "low", + [ThinkingLevel.MEDIUM]: "medium", + [ThinkingLevel.HIGH]: "high", + [ThinkingLevel.XHIGH]: "xhigh", + }; + return mapping[thinkingLevel]; + } + + /** + * Convert ToolChoice to OpenAI's tool_choice format. + */ + private _convertToolChoice(toolChoice: ToolChoice): string { + if (toolChoice === "auto") { + return "auto"; + } else { + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "tool_choice", + message: 'GLM only supports "auto" for tool_choice.', + }); + } + } + + /** + * Transform universal configuration to GLM-5.2-specific configuration. + */ + // eslint-disable-next-line @typescript-eslint/no-explicit-any + transformUniConfigToModelConfig(config: UniConfig): any { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const glmConfig: any = { + model: this._model, + stream: true, + extra_body: { tool_stream: true }, + }; + + if (config.max_tokens !== undefined) { + glmConfig.max_tokens = config.max_tokens; + } + + if (config.temperature !== undefined) { + glmConfig.temperature = config.temperature; + } + + if (config.thinking_level !== undefined) { + const thinkingConfig = this._convertThinkingLevelToConfig( + config.thinking_level, + ); + glmConfig.extra_body = { + ...(glmConfig.extra_body || {}), + thinking: thinkingConfig, + }; + const reasoningEffort = this._convertThinkingLevelToReasoningEffort( + config.thinking_level, + ); + if (reasoningEffort !== undefined) { + glmConfig.reasoning_effort = reasoningEffort; + } + } + + if (config.tools !== undefined) { + glmConfig.tools = config.tools.map((tool) => ({ + type: "function", + function: tool, + })); + } + + if (config.tool_choice !== undefined) { + glmConfig.tool_choice = this._convertToolChoice(config.tool_choice); + } + + if ( + config.prompt_caching !== undefined && + config.prompt_caching !== PromptCaching.ENABLE + ) { + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "prompt_caching", + message: "prompt_caching must be ENABLE for GLM.", + }); + } + + return glmConfig; + } + + /** + * Transform universal message format to OpenAI's message format. + */ + transformUniMessageToModelInput( + messages: UniMessage[], + _signal?: AbortSignal, + ): ChatCompletionMessageParam[] { + const openaiMessages: ChatCompletionMessageParam[] = []; + + for (const msg of messages) { + const contentParts: Array<{ + type: string; + text?: string; + image_url?: { url: string }; + }> = []; + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const toolCalls: any[] = []; + let thinking = ""; + const thinkingFields = new Set(); + + for (const item of msg.content_items) { + if (item.type === "text") { + contentParts.push({ type: "text", text: item.text }); + } else if (item.type === "image_url") { + throw new Error("GLM-5 does not support image inputs."); + } else if (item.type === "thinking") { + thinking += item.thinking; + thinkingFields.add(item.fidelity?.reasoning_field); + } else if (item.type === "tool_call") { + toolCalls.push({ + id: item.tool_call_id, + type: "function", + function: { + name: item.name, + arguments: JSON.stringify(item.arguments, null, 0), + }, + }); + } else if (item.type === "tool_result") { + if (!item.tool_call_id) { + throw new Error("tool_call_id is required for tool result."); + } + + if (item.images && item.images.length > 0) { + throw new Error("GLM-5 does not support images in tool results."); + } + + openaiMessages.push({ + role: "tool", + tool_call_id: item.tool_call_id, + content: item.text, + }); + } else { + throw new Error( + `Unknown item type: ${(item as { type: string }).type}`, + ); + } + } + + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const message: any = { role: msg.role }; + if (contentParts.length > 0) { + message.content = contentParts; + } + + if (toolCalls.length > 0) { + message.tool_calls = toolCalls; + } + + if (thinking) { + // send thinking back through the exact field the upstream produced (recorded + // in the item fidelity); servers may reject the spelling they did not emit + if ( + thinkingFields.size === 1 && + thinkingFields.has("reasoning_content") + ) { + message.reasoning_content = thinking; + } else if ( + thinkingFields.size === 1 && + thinkingFields.has("reasoning") + ) { + message.reasoning = thinking; + } else { + message.reasoning_content = thinking; // vLLM & siliconflow compatibility + message.reasoning = thinking; // openrouter compatibility + } + } + + if (Object.keys(message).length > 1) { + openaiMessages.push(message); + } + } + + return openaiMessages; + } + + /** + * Transform GLM model output to universal event format. + */ + transformModelOutputToUniEvent(modelOutput: ChatCompletionChunk): UniEvent { + let eventType: EventType | null = null; + const contentItems: PartialContentItem[] = []; + let usageMetadata: UsageMetadata | null = null; + let finishReason: FinishReason | null = null; + + if (modelOutput.choices.length > 0) { + const choice = modelOutput.choices[0]; + const delta = choice?.delta; + + if (delta?.content) { + eventType = "delta"; + contentItems.push({ type: "text", text: delta.content }); + } + + // the thinking field name differs by server: vLLM & siliconflow use + // reasoning_content while openrouter uses reasoning; record the wire + // field that carried each delta so a replay can reproduce exactly the + // field the upstream produced + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const reasoningContent = (delta as any)?.reasoning_content; + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const reasoning = (delta as any)?.reasoning; + if (reasoningContent && reasoning) { + eventType = "delta"; + // ambiguous origin: record no fidelity so a replay sends both fields back + contentItems.push({ type: "thinking", thinking: reasoningContent }); + } else if (reasoningContent) { + eventType = "delta"; + contentItems.push({ + type: "thinking", + thinking: reasoningContent, + fidelity: { reasoning_field: "reasoning_content" }, + }); + } else if (reasoning) { + eventType = "delta"; + contentItems.push({ + type: "thinking", + thinking: reasoning, + fidelity: { reasoning_field: "reasoning" }, + }); + } + + if (delta?.tool_calls) { + eventType = "delta"; + for (const toolCall of delta.tool_calls) { + contentItems.push({ + type: "partial_tool_call", + name: toolCall.function?.name || "", + arguments: toolCall.function?.arguments || "", + tool_call_id: toolCall.id || "", + }); + } + } + + if (choice?.finish_reason) { + eventType = eventType || "stop"; + const finishReasonMapping: { [key: string]: FinishReason } = { + stop: "stop", + length: "length", + tool_calls: "tool_call", + content_filter: "stop", + }; + finishReason = finishReasonMapping[choice.finish_reason] || "unknown"; + } + } + + if (modelOutput.usage) { + eventType = eventType || "stop"; + + const cachedTokens = + modelOutput.usage.prompt_tokens_details?.cached_tokens || null; + const reasoningTokens = + modelOutput.usage.completion_tokens_details?.reasoning_tokens || null; + + const promptTokens = + cachedTokens !== null + ? modelOutput.usage.prompt_tokens - cachedTokens + : modelOutput.usage.prompt_tokens; + const responseTokens = + reasoningTokens !== null + ? modelOutput.usage.completion_tokens - reasoningTokens + : modelOutput.usage.completion_tokens; + + usageMetadata = { + cached_tokens: cachedTokens, + prompt_tokens: promptTokens, + thoughts_tokens: reasoningTokens, + response_tokens: responseTokens, + }; + usageMetadata = fixOpenrouterUsageMetadata( + usageMetadata, + this._client.baseURL, + ); + } + + return { + role: "assistant", + event_type: eventType as EventType, + content_items: contentItems, + usage_metadata: usageMetadata, + finish_reason: finishReason, + }; + } + + /** + * Stream generate using GLM SDK with unified conversion methods. + */ + async *_streamingResponseInternal(options: { + messages: UniMessage[]; + config: UniConfig; + signal?: AbortSignal; + }): AsyncGenerator { + const glmConfig = this.transformUniConfigToModelConfig(options.config); + const glmMessages = this.transformUniMessageToModelInput( + options.messages, + options.signal, + ); + + if (options.config.system_prompt) { + glmMessages.unshift({ + role: "system", + content: options.config.system_prompt, + }); + } + + const params: ChatCompletionCreateParamsStreaming = { + ...glmConfig, + messages: glmMessages, + stream: true, + }; + + const stream = await this._client.chat.completions.create(params, { + signal: options.signal, + }); + + const partialToolCall: { + name?: string; + arguments?: string; + tool_call_id?: string; + } = {}; + let partialUsage: { + finish_reason?: FinishReason | null; + usage_metadata?: UsageMetadata | null; + } = {}; + + for await (const chunk of stream) { + const event = this.transformModelOutputToUniEvent(chunk); + // the finish reason and usage metadata should be accumulated + partialUsage.finish_reason = + event.finish_reason || partialUsage.finish_reason; + partialUsage.usage_metadata = + event.usage_metadata || partialUsage.usage_metadata; + if (event.event_type === "delta") { + for (const item of event.content_items) { + if (item.type === "partial_tool_call") { + if (!partialToolCall.name) { + // start a new partial tool call + partialToolCall.name = item.name; + partialToolCall.arguments = item.arguments; + partialToolCall.tool_call_id = item.tool_call_id; + } else if (item.name) { + // finish the previous partial tool call + yield { + role: "assistant", + event_type: "delta", + content_items: [ + { + type: "tool_call", + name: partialToolCall.name, + arguments: parseToolCallArguments( + partialToolCall.arguments, + this.constructor.name, + partialToolCall.name || "", + partialToolCall.tool_call_id || "", + ), + tool_call_id: partialToolCall.tool_call_id || "", + }, + ], + usage_metadata: null, + finish_reason: null, + }; + // start a new partial tool call + partialToolCall.name = item.name; + partialToolCall.arguments = item.arguments; + partialToolCall.tool_call_id = item.tool_call_id; + } else { + // update partial tool call + partialToolCall.arguments = + (partialToolCall.arguments || "") + item.arguments; + } + } + } + yield event; + } else if (event.event_type === "stop") { + if (partialToolCall.name) { + // finish the partial tool call + yield { + role: "assistant", + event_type: "delta", + content_items: [ + { + type: "tool_call", + name: partialToolCall.name, + arguments: parseToolCallArguments( + partialToolCall.arguments, + this.constructor.name, + partialToolCall.name || "", + partialToolCall.tool_call_id || "", + ), + tool_call_id: partialToolCall.tool_call_id || "", + }, + ], + usage_metadata: null, + finish_reason: null, + }; + partialToolCall.name = undefined; + partialToolCall.arguments = undefined; + partialToolCall.tool_call_id = undefined; + } + + if (partialUsage.finish_reason && partialUsage.usage_metadata) { + yield { + role: "assistant", + event_type: "stop", + content_items: [], + usage_metadata: partialUsage.usage_metadata, + finish_reason: partialUsage.finish_reason, + }; + partialUsage.finish_reason = null; + partialUsage.usage_metadata = null; + } + } + } + } +} diff --git a/src_ts/src/glm5_2/index.ts b/src_ts/src/glm5_2/index.ts new file mode 100644 index 00000000..551be084 --- /dev/null +++ b/src_ts/src/glm5_2/index.ts @@ -0,0 +1,15 @@ +// Copyright 2025 Prism Shadow. and/or its affiliates +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +export { GLM5_2Client } from "./client"; diff --git a/src_ts/src/gpt5_5/client.ts b/src_ts/src/gpt5_5/client.ts index 64bd9e13..ab76165a 100644 --- a/src_ts/src/gpt5_5/client.ts +++ b/src_ts/src/gpt5_5/client.ts @@ -19,7 +19,10 @@ import type { ResponseCreateParamsStreaming, } from "openai/resources/responses/responses"; import { LLMClient } from "../baseClient"; -import { parseToolCallArguments } from "../errors"; +import { + parseToolCallArguments, + UnsupportedParameterError, +} from "../errors"; import { EventType, FinishReason, @@ -106,7 +109,11 @@ export class GPT5_5Client extends LLMClient { } if (config.temperature !== undefined && config.temperature !== 1.0) { - throw new Error("GPT-5.5 does not support setting temperature."); + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "temperature", + message: "GPT-5.5 does not support setting temperature.", + }); } if (config.thinking_level !== undefined) { @@ -133,7 +140,11 @@ export class GPT5_5Client extends LLMClient { config.prompt_caching !== undefined && config.prompt_caching !== PromptCaching.ENABLE ) { - throw new Error("prompt_caching must be ENABLE for GPT-5.5."); + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "prompt_caching", + message: "prompt_caching must be ENABLE for GPT-5.5.", + }); } return openaiConfig; diff --git a/src_ts/src/index.ts b/src_ts/src/index.ts index 30487697..c8ffae16 100644 --- a/src_ts/src/index.ts +++ b/src_ts/src/index.ts @@ -17,5 +17,13 @@ export { AgentHubError, EmptyResponseError, ToolCallArgumentParseError, + UnsupportedParameterError, } from "./errors"; +export { + listSupportedModels, + Currency, + Modality, + ModelPricing, + SupportedModel, +} from "./registry"; export * from "./types"; diff --git a/src_ts/src/kimi_k2_6/client.ts b/src_ts/src/kimi_k2_6/client.ts index e565ff3f..12b76107 100644 --- a/src_ts/src/kimi_k2_6/client.ts +++ b/src_ts/src/kimi_k2_6/client.ts @@ -20,7 +20,10 @@ import type { ChatCompletionCreateParamsStreaming, } from "openai/resources/chat/completions"; import { LLMClient } from "../baseClient"; -import { parseToolCallArguments } from "../errors"; +import { + parseToolCallArguments, + UnsupportedParameterError, +} from "../errors"; import { EventType, FinishReason, @@ -129,7 +132,11 @@ export class KimiK2_6Client extends LLMClient { } else if (toolChoice === "none") { return "none"; } else { - throw new Error('Kimi only supports "auto" and "none" for tool_choice.'); + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "tool_choice", + message: 'Kimi only supports "auto" and "none" for tool_choice.', + }); } } @@ -150,7 +157,11 @@ export class KimiK2_6Client extends LLMClient { } if (config.temperature !== undefined && config.temperature !== 1.0) { - throw new Error("Kimi does not support setting temperature."); + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "temperature", + message: "Kimi does not support setting temperature.", + }); } if (config.thinking_level !== undefined) { @@ -178,7 +189,11 @@ export class KimiK2_6Client extends LLMClient { config.prompt_caching !== undefined && config.prompt_caching !== PromptCaching.ENABLE ) { - throw new Error("prompt_caching must be ENABLE for Kimi."); + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "prompt_caching", + message: "prompt_caching must be ENABLE for Kimi.", + }); } if (config.trace_id !== undefined) { diff --git a/src_ts/src/kimi_k3/client.ts b/src_ts/src/kimi_k3/client.ts new file mode 100644 index 00000000..6047d320 --- /dev/null +++ b/src_ts/src/kimi_k3/client.ts @@ -0,0 +1,566 @@ +// Copyright 2025 Prism Shadow. and/or its affiliates +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +import * as path from "path"; +import OpenAI from "openai"; +import type { + ChatCompletionChunk, + ChatCompletionMessageParam, + ChatCompletionCreateParamsStreaming, +} from "openai/resources/chat/completions"; +import { LLMClient } from "../baseClient"; +import { + parseToolCallArguments, + UnsupportedParameterError, +} from "../errors"; +import { + EventType, + FinishReason, + PartialContentItem, + PromptCaching, + ThinkingLevel, + ToolChoice, + UniConfig, + UniEvent, + UniMessage, + UsageMetadata, +} from "../types"; +import { fixOpenrouterUsageMetadata } from "../utils"; + +/** + * Kimi K3-specific LLM client implementation using OpenAI-compatible API. + */ +export class KimiK3Client extends LLMClient { + protected _model: string; + private _client: OpenAI; + + /** + * Initialize Kimi K3 client with model and API key. + */ + constructor(options: { + model: string; + apiKey?: string; + baseUrl?: string | null; + clientType?: string | null; + }) { + super(); + this._model = options.model; + const key = options.apiKey || process.env.MOONSHOT_API_KEY || undefined; + const url = + options.baseUrl || + process.env.MOONSHOT_BASE_URL || + "https://api.moonshot.cn/v1"; + this._client = new OpenAI({ apiKey: key, baseURL: url }); + } + + /** + * Detect MIME type from URL extension for image. + */ + private _detectImageMimeType(url: string): string { + const ext = path.extname(url).toLowerCase(); + const mimeTypes: { [key: string]: string } = { + ".bmp": "image/bmp", + ".gif": "image/gif", + ".jpg": "image/jpeg", + ".jpeg": "image/jpeg", + ".png": "image/png", + ".svg": "image/svg+xml", + ".tiff": "image/tiff", + ".webp": "image/webp", + }; + return mimeTypes[ext] || "image/jpeg"; + } + + /** + * Convert image URL to base64-encoded data URL. + */ + private async _convertImageUrlToBase64( + url: string, + signal?: AbortSignal, + ): Promise { + if (url.startsWith("data:")) { + return url; + } + + const response = await fetch(url, { signal }); + if (!response.ok) { + throw new Error( + `Failed to fetch image: ${response.status} ${response.statusText}`, + ); + } + const arrayBuffer = await response.arrayBuffer(); + const buffer = Buffer.from(arrayBuffer); + const mimeType = this._detectImageMimeType(url); + const base64String = buffer.toString("base64"); + return `data:${mimeType};base64,${base64String}`; + } + + /** + * Convert ThinkingLevel enum to Kimi K3's reasoning_effort. + * + * K3 cannot disable reasoning, so NONE degrades to the lowest effort + * instead of throwing. + */ + private _convertThinkingLevelToReasoningEffort( + thinkingLevel: ThinkingLevel, + ): string { + const mapping: { [key: string]: string } = { + [ThinkingLevel.NONE]: "low", + [ThinkingLevel.LOW]: "low", + [ThinkingLevel.MEDIUM]: "high", + [ThinkingLevel.HIGH]: "high", + [ThinkingLevel.XHIGH]: "max", + }; + return mapping[thinkingLevel]; + } + + /** + * Convert ToolChoice to OpenAI's tool_choice format. + */ + private _convertToolChoice(toolChoice: ToolChoice): string { + if (toolChoice === "auto") { + return "auto"; + } else if (toolChoice === "none") { + return "none"; + } else if (toolChoice === "required") { + return "required"; + } else { + // forcing a specific tool is incompatible with K3's always-on reasoning + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "tool_choice", + message: + 'Kimi K3 does not support forcing specific tools; only "auto", "none" and "required" are supported.', + }); + } + } + + /** + * Transform universal configuration to Kimi K3-specific configuration. + */ + // eslint-disable-next-line @typescript-eslint/no-explicit-any + transformUniConfigToModelConfig(config: UniConfig): any { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const kimiConfig: any = { + model: this._model, + stream: true, + stream_options: { include_usage: true }, + }; + + if (config.max_tokens !== undefined) { + kimiConfig.max_completion_tokens = config.max_tokens; + } + + if (config.temperature !== undefined && config.temperature !== 1.0) { + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "temperature", + message: "Kimi K3 does not support setting temperature.", + }); + } + + if (config.thinking_level !== undefined) { + kimiConfig.reasoning_effort = this._convertThinkingLevelToReasoningEffort( + config.thinking_level, + ); + } + + if (config.tools !== undefined) { + kimiConfig.tools = config.tools.map((tool) => ({ + type: "function", + function: tool, + })); + } + + if (config.tool_choice !== undefined) { + kimiConfig.tool_choice = this._convertToolChoice(config.tool_choice); + } + + if ( + config.prompt_caching !== undefined && + config.prompt_caching !== PromptCaching.ENABLE + ) { + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "prompt_caching", + message: "prompt_caching must be ENABLE for Kimi K3.", + }); + } + + // K3 context caching is automatic; trace_id is intentionally not sent + // as prompt_cache_key + return kimiConfig; + } + + /** + * Transform universal message format to OpenAI's message format. + */ + async transformUniMessageToModelInput( + messages: UniMessage[], + signal?: AbortSignal, + ): Promise { + const openaiMessages: ChatCompletionMessageParam[] = []; + + for (const msg of messages) { + const contentParts: Array<{ + type: string; + text?: string; + image_url?: { url: string }; + }> = []; + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const toolCalls: any[] = []; + let thinking = ""; + const thinkingFields = new Set(); + + for (const item of msg.content_items) { + if (item.type === "text") { + contentParts.push({ type: "text", text: item.text }); + } else if (item.type === "image_url") { + const base64Image = await this._convertImageUrlToBase64( + item.image_url, + signal, + ); + contentParts.push({ + type: "image_url", + image_url: { url: base64Image }, + }); + } else if (item.type === "thinking") { + thinking += item.thinking; + thinkingFields.add(item.fidelity?.reasoning_field); + } else if (item.type === "tool_call") { + toolCalls.push({ + id: item.tool_call_id, + type: "function", + function: { + name: item.name, + arguments: JSON.stringify(item.arguments, null, 0), + }, + }); + } else if (item.type === "tool_result") { + if (!item.tool_call_id) { + throw new Error("tool_call_id is required for tool result."); + } + + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const content: any[] = [{ type: "text", text: item.text }]; + + if (item.images && item.images.length > 0) { + for (const imageUrl of item.images) { + const base64Image = await this._convertImageUrlToBase64( + imageUrl, + signal, + ); + if (this._client.baseURL.includes("siliconflow.cn")) { + // siliconflow does not support image_url in tool result + contentParts.push({ + type: "image_url", + image_url: { url: base64Image }, + }); + } else { + content.push({ + type: "image_url", + image_url: { url: base64Image }, + }); + } + } + } + + openaiMessages.push({ + role: "tool", + tool_call_id: item.tool_call_id, + content, + }); + } else { + throw new Error( + `Unknown item type: ${(item as { type: string }).type}`, + ); + } + } + + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const message: any = { role: msg.role }; + if (contentParts.length > 0) { + message.content = contentParts; + } + + if (toolCalls.length > 0) { + message.tool_calls = toolCalls; + } + + if (thinking) { + // send thinking back through the exact field the upstream produced (recorded + // in the item fidelity); servers may reject the spelling they did not emit + if ( + thinkingFields.size === 1 && + thinkingFields.has("reasoning_content") + ) { + message.reasoning_content = thinking; + } else if ( + thinkingFields.size === 1 && + thinkingFields.has("reasoning") + ) { + message.reasoning = thinking; + } else { + message.reasoning_content = thinking; // vLLM & siliconflow compatibility + message.reasoning = thinking; // openrouter compatibility + } + } + + if (Object.keys(message).length > 1) { + openaiMessages.push(message); + } + } + + return openaiMessages; + } + + /** + * Transform Kimi K3 model output to universal event format. + */ + transformModelOutputToUniEvent(modelOutput: ChatCompletionChunk): UniEvent { + let eventType: EventType | null = null; + const contentItems: PartialContentItem[] = []; + let usageMetadata: UsageMetadata | null = null; + let finishReason: FinishReason | null = null; + + if (modelOutput.choices.length > 0) { + const choice = modelOutput.choices[0]; + const delta = choice?.delta; + + if (delta?.content) { + eventType = "delta"; + contentItems.push({ type: "text", text: delta.content }); + } + + // the thinking field name differs by server: vLLM & siliconflow use + // reasoning_content while openrouter uses reasoning; record the wire + // field that carried each delta so a replay can reproduce exactly the + // field the upstream produced + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const reasoningContent = (delta as any)?.reasoning_content; + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const reasoning = (delta as any)?.reasoning; + if (reasoningContent && reasoning) { + eventType = "delta"; + // ambiguous origin: record no fidelity so a replay sends both fields back + contentItems.push({ type: "thinking", thinking: reasoningContent }); + } else if (reasoningContent) { + eventType = "delta"; + contentItems.push({ + type: "thinking", + thinking: reasoningContent, + fidelity: { reasoning_field: "reasoning_content" }, + }); + } else if (reasoning) { + eventType = "delta"; + contentItems.push({ + type: "thinking", + thinking: reasoning, + fidelity: { reasoning_field: "reasoning" }, + }); + } + + if (delta?.tool_calls) { + eventType = "delta"; + for (const toolCall of delta.tool_calls) { + contentItems.push({ + type: "partial_tool_call", + name: toolCall.function?.name || "", + arguments: toolCall.function?.arguments || "", + tool_call_id: toolCall.id || "", + }); + } + } + + if (choice?.finish_reason) { + eventType = eventType || "stop"; + const finishReasonMapping: { [key: string]: FinishReason } = { + stop: "stop", + length: "length", + tool_calls: "tool_call", + content_filter: "stop", + }; + finishReason = finishReasonMapping[choice.finish_reason] || "unknown"; + } + } + + if (modelOutput.usage) { + eventType = eventType || "stop"; + + const cachedTokens = + modelOutput.usage.prompt_tokens_details?.cached_tokens || null; + const reasoningTokens = + modelOutput.usage.completion_tokens_details?.reasoning_tokens || null; + + const promptTokens = + cachedTokens !== null + ? modelOutput.usage.prompt_tokens - cachedTokens + : modelOutput.usage.prompt_tokens; + const responseTokens = + reasoningTokens !== null + ? modelOutput.usage.completion_tokens - reasoningTokens + : modelOutput.usage.completion_tokens; + + usageMetadata = { + cached_tokens: cachedTokens, + prompt_tokens: promptTokens, + thoughts_tokens: reasoningTokens, + response_tokens: responseTokens, + }; + usageMetadata = fixOpenrouterUsageMetadata( + usageMetadata, + this._client.baseURL, + ); + } + + return { + role: "assistant", + event_type: eventType as EventType, + content_items: contentItems, + usage_metadata: usageMetadata, + finish_reason: finishReason, + }; + } + + /** + * Stream generate using Kimi SDK with unified conversion methods. + */ + async *_streamingResponseInternal(options: { + messages: UniMessage[]; + config: UniConfig; + signal?: AbortSignal; + }): AsyncGenerator { + const kimiConfig = this.transformUniConfigToModelConfig(options.config); + const kimiMessages = await this.transformUniMessageToModelInput( + options.messages, + options.signal, + ); + + if (options.config.system_prompt) { + kimiMessages.unshift({ + role: "system", + content: options.config.system_prompt, + }); + } + + const params: ChatCompletionCreateParamsStreaming = { + ...kimiConfig, + messages: kimiMessages, + stream: true, + }; + + const stream = await this._client.chat.completions.create(params, { + signal: options.signal, + }); + + const partialToolCall: { + name?: string; + arguments?: string; + tool_call_id?: string; + } = {}; + let partialUsage: { + finish_reason?: FinishReason | null; + usage_metadata?: UsageMetadata | null; + } = {}; + + for await (const chunk of stream) { + const event = this.transformModelOutputToUniEvent(chunk); + // the finish reason and usage metadata should be accumulated + partialUsage.finish_reason = + event.finish_reason || partialUsage.finish_reason; + partialUsage.usage_metadata = + event.usage_metadata || partialUsage.usage_metadata; + if (event.event_type === "delta") { + for (const item of event.content_items) { + if (item.type === "partial_tool_call") { + if (!partialToolCall.name) { + // start a new partial tool call + partialToolCall.name = item.name; + partialToolCall.arguments = item.arguments; + partialToolCall.tool_call_id = item.tool_call_id; + } else if (item.name) { + // finish the previous partial tool call + yield { + role: "assistant", + event_type: "delta", + content_items: [ + { + type: "tool_call", + name: partialToolCall.name, + arguments: parseToolCallArguments( + partialToolCall.arguments, + this.constructor.name, + partialToolCall.name || "", + partialToolCall.tool_call_id || "", + ), + tool_call_id: partialToolCall.tool_call_id || "", + }, + ], + usage_metadata: null, + finish_reason: null, + }; + // start a new partial tool call + partialToolCall.name = item.name; + partialToolCall.arguments = item.arguments; + partialToolCall.tool_call_id = item.tool_call_id; + } else { + // update partial tool call + partialToolCall.arguments = + (partialToolCall.arguments || "") + item.arguments; + } + } + } + yield event; + } else if (event.event_type === "stop") { + if (partialToolCall.name) { + // finish the partial tool call + yield { + role: "assistant", + event_type: "delta", + content_items: [ + { + type: "tool_call", + name: partialToolCall.name, + arguments: parseToolCallArguments( + partialToolCall.arguments, + this.constructor.name, + partialToolCall.name || "", + partialToolCall.tool_call_id || "", + ), + tool_call_id: partialToolCall.tool_call_id || "", + }, + ], + usage_metadata: null, + finish_reason: null, + }; + partialToolCall.name = undefined; + partialToolCall.arguments = undefined; + partialToolCall.tool_call_id = undefined; + } + + if (partialUsage.finish_reason && partialUsage.usage_metadata) { + yield { + role: "assistant", + event_type: "stop", + content_items: [], + usage_metadata: partialUsage.usage_metadata, + finish_reason: partialUsage.finish_reason, + }; + partialUsage.finish_reason = null; + partialUsage.usage_metadata = null; + } + } + } + } +} diff --git a/src_ts/src/kimi_k3/index.ts b/src_ts/src/kimi_k3/index.ts new file mode 100644 index 00000000..51ec1fd4 --- /dev/null +++ b/src_ts/src/kimi_k3/index.ts @@ -0,0 +1,15 @@ +// Copyright 2025 Prism Shadow. and/or its affiliates +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +export { KimiK3Client } from "./client"; diff --git a/src_ts/src/openai/client.ts b/src_ts/src/openai/client.ts index 1011f0ec..dce19b06 100644 --- a/src_ts/src/openai/client.ts +++ b/src_ts/src/openai/client.ts @@ -20,7 +20,10 @@ import type { ChatCompletionCreateParamsStreaming, } from "openai/resources/chat/completions"; import { LLMClient } from "../baseClient"; -import { parseToolCallArguments } from "../errors"; +import { + parseToolCallArguments, + UnsupportedParameterError, +} from "../errors"; import { EventType, FinishReason, @@ -155,7 +158,11 @@ export class OpenaiClient extends LLMClient { config.prompt_caching !== undefined && config.prompt_caching !== PromptCaching.ENABLE ) { - throw new Error("prompt_caching must be ENABLE for OpenAI."); + throw new UnsupportedParameterError({ + client: this.constructor.name, + parameter: "prompt_caching", + message: "prompt_caching must be ENABLE for OpenAI.", + }); } return openaiConfig; diff --git a/src_ts/src/registry.ts b/src_ts/src/registry.ts new file mode 100644 index 00000000..66af42f7 --- /dev/null +++ b/src_ts/src/registry.ts @@ -0,0 +1,559 @@ +// Copyright 2025 Prism Shadow. and/or its affiliates +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +export type Modality = "Text" | "Image" | "Video" | "Audio" | "Embed"; +export type Currency = "USD" | "CNY"; + +/** + * List prices per million tokens for AgentHub's usage buckets. + * + * Keys mirror `usage_metadata`: `cached_tokens` (cache-hit price, absent when + * the platform publishes none), `prompt_tokens` (non-cached input), and + * `thoughts_tokens`/`response_tokens`, which both carry the vendor's output + * price. Values are in the currency requested from listSupportedModels. + */ +export interface ModelPricing { + currency: Currency; + prompt_tokens: number; + thoughts_tokens: number; + response_tokens: number; + cached_tokens?: number; +} + +/** + * One supported model entry. + * + * (model, base_url, client) maps directly onto the AutoLLMClient constructor: + * `new AutoLLMClient({ model, baseUrl: base_url, clientType: client })`. + * Modalities describe what is usable through that client; `context_window` and + * `pricing` are omitted where the platform publishes no authoritative value. + */ +export interface SupportedModel { + model: string; + base_url: string; + client: string; + input_modalities: Modality[]; + output_modalities: Modality[]; + context_window?: number; + pricing?: ModelPricing; +} + +const GOOGLE = "https://generativelanguage.googleapis.com"; +const ANTHROPIC = "https://api.anthropic.com"; +const OPENAI = "https://api.openai.com/v1"; +const ZAI = "https://api.z.ai/api/paas/v4/"; +const MOONSHOT = "https://api.moonshot.cn/v1"; +const DEEPSEEK = "https://api.deepseek.com"; +const OPENROUTER = "https://openrouter.ai/api/v1"; +const SILICONFLOW = "https://api.siliconflow.cn/v1"; + +// Display convention shared with the AgentHub apps: prices are stored in USD +// (official CNY list prices pre-converted at 7 CNY/USD), so requesting CNY +// shows the vendor's numbers. +const CNY_PER_USD = 7.0; + +function usd(prompt: number, output: number, cached?: number): ModelPricing { + // thoughts and response tokens are both billed at the vendor's output price + return { + currency: "USD", + prompt_tokens: prompt, + thoughts_tokens: output, + response_tokens: output, + ...(cached !== undefined ? { cached_tokens: cached } : {}), + }; +} + +/** + * Declare a CNY-denominated official list price; storage stays USD (converted + * at 7 CNY/USD). + */ +function cny(prompt: number, output: number, cached?: number): ModelPricing { + const rate = (v: number): number => Math.round((v / CNY_PER_USD) * 1e6) / 1e6; + return usd(rate(prompt), rate(output), cached !== undefined ? rate(cached) : undefined); +} + +// Prices in USD per million tokens (official CNY prices pre-converted at +// 7 CNY/USD); platform data (context windows, OpenRouter USD prices, modality +// flags) verified against the live /models APIs on 2026-07-22, SiliconFlow CNY +// prices from the vendors' official price lists. +const SUPPORTED_MODELS: SupportedModel[] = [ + // official vendor endpoints + { + model: "gemini-3.6-flash", + base_url: GOOGLE, + client: "gemini-3.6", + input_modalities: ["Text", "Image", "Video", "Audio"], + output_modalities: ["Text"], + context_window: 1048576, + pricing: usd(1.5, 7.5, 0.15), + }, + { + model: "gemini-3.5-flash-lite", + base_url: GOOGLE, + client: "gemini-3.6", + input_modalities: ["Text", "Image", "Video", "Audio"], + output_modalities: ["Text"], + context_window: 1048576, + pricing: usd(0.3, 2.5, 0.03), + }, + { + model: "gemini-3.5-flash", + base_url: GOOGLE, + client: "gemini-3", + input_modalities: ["Text", "Image", "Video", "Audio"], + output_modalities: ["Text"], + context_window: 1048576, + pricing: usd(1.5, 9.0, 0.15), + }, + { + model: "gemini-3.1-flash-image", + base_url: GOOGLE, + client: "gemini-3", + input_modalities: ["Text", "Image"], + output_modalities: ["Image"], + }, + { + model: "gemini-3.1-flash-tts-preview", + base_url: GOOGLE, + client: "gemini-3", + input_modalities: ["Text"], + output_modalities: ["Audio"], + }, + { + model: "gemini-embedding-2", + base_url: GOOGLE, + client: "gemini-3", + input_modalities: ["Text"], + output_modalities: ["Embed"], + }, + { + model: "claude-fable-5", + base_url: ANTHROPIC, + client: "claude-5", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 1000000, + pricing: usd(10.0, 50.0, 1.0), + }, + { + model: "claude-sonnet-5", + base_url: ANTHROPIC, + client: "claude-5", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 1000000, + pricing: usd(2.0, 10.0, 0.2), + }, + { + model: "claude-opus-4-8", + base_url: ANTHROPIC, + client: "claude-5", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 1000000, + pricing: usd(5.0, 25.0, 0.5), + }, + { + model: "claude-sonnet-4-6", + base_url: ANTHROPIC, + client: "claude-4-6", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 1000000, + pricing: usd(3.0, 15.0, 0.3), + }, + { + model: "gpt-5.5", + base_url: OPENAI, + client: "gpt-5.5", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 1050000, + pricing: usd(5.0, 30.0, 0.5), + }, + { + model: "text-embedding-3-large", + base_url: OPENAI, + client: "openai-embedding", + input_modalities: ["Text"], + output_modalities: ["Embed"], + pricing: usd(0.13, 0.0), + }, + { + model: "glm-5.2", + base_url: ZAI, + client: "glm-5.2", + input_modalities: ["Text"], + output_modalities: ["Text"], + context_window: 1000000, + pricing: usd(1.4, 4.4, 0.26), + }, + { + model: "glm-5.1", + base_url: ZAI, + client: "glm-5.1", + input_modalities: ["Text"], + output_modalities: ["Text"], + context_window: 200000, + pricing: usd(1.4, 4.4, 0.26), + }, + { + model: "kimi-k3", + base_url: MOONSHOT, + client: "kimi-k3", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 1048576, + pricing: cny(20.0, 100.0, 2.0), + }, + { + model: "kimi-k2.6", + base_url: MOONSHOT, + client: "kimi-k2.6", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 262144, + pricing: cny(6.5, 27.0, 1.1), + }, + { + model: "deepseek-v4-flash", + base_url: DEEPSEEK, + client: "deepseek-v4", + input_modalities: ["Text"], + output_modalities: ["Text"], + context_window: 1000000, + pricing: cny(1.0, 2.0, 0.02), + }, + { + model: "deepseek-v4-pro", + base_url: DEEPSEEK, + client: "deepseek-v4", + input_modalities: ["Text"], + output_modalities: ["Text"], + context_window: 1000000, + pricing: cny(3.0, 6.0, 0.025), + }, + // OpenRouter (USD prices, context windows and modality flags from the live /models API) + { + model: "anthropic/claude-fable-5", + base_url: OPENROUTER, + client: "openai", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 1000000, + pricing: usd(10.0, 50.0, 1.0), + }, + { + model: "anthropic/claude-opus-4.8", + base_url: OPENROUTER, + client: "openai", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 1000000, + pricing: usd(5.0, 25.0, 0.5), + }, + { + model: "anthropic/claude-opus-4.7", + base_url: OPENROUTER, + client: "openai", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 1000000, + pricing: usd(5.0, 25.0, 0.5), + }, + { + model: "anthropic/claude-sonnet-5", + base_url: OPENROUTER, + client: "openai", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 1000000, + pricing: usd(2.0, 10.0, 0.2), + }, + { + model: "deepseek/deepseek-v4-flash", + base_url: OPENROUTER, + client: "deepseek-v4", + input_modalities: ["Text"], + output_modalities: ["Text"], + context_window: 1048576, + pricing: usd(0.098, 0.196, 0.0196), + }, + { + model: "deepseek/deepseek-v4-pro", + base_url: OPENROUTER, + client: "deepseek-v4", + input_modalities: ["Text"], + output_modalities: ["Text"], + context_window: 1048576, + pricing: usd(0.435, 0.87, 0.003625), + }, + { + model: "google/gemini-3.5-flash", + base_url: OPENROUTER, + client: "openai", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 1048576, + pricing: usd(1.5, 9.0, 0.15), + }, + { + model: "minimax/minimax-m3", + base_url: OPENROUTER, + client: "openai", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 1048576, + pricing: usd(0.3, 1.2, 0.06), + }, + { + model: "moonshotai/kimi-k3", + base_url: OPENROUTER, + client: "kimi-k3", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 1048576, + pricing: usd(3.0, 15.0, 0.3), + }, + { + model: "moonshotai/kimi-k2.6", + base_url: OPENROUTER, + client: "kimi-k2.6", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 262144, + pricing: usd(0.684, 3.42, 0.144), + }, + { + model: "nvidia/nemotron-3-ultra-550b-a55b:free", + base_url: OPENROUTER, + client: "openai", + input_modalities: ["Text"], + output_modalities: ["Text"], + context_window: 1000000, + pricing: usd(0.0, 0.0), + }, + { + model: "openai/gpt-5.6-sol", + base_url: OPENROUTER, + client: "openai", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 1050000, + pricing: usd(5.0, 30.0, 0.5), + }, + { + model: "openai/gpt-5.6-terra", + base_url: OPENROUTER, + client: "openai", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 1050000, + pricing: usd(2.5, 15.0, 0.25), + }, + { + model: "openai/gpt-5.5", + base_url: OPENROUTER, + client: "openai", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 1050000, + pricing: usd(5.0, 30.0, 0.5), + }, + { + model: "qwen/qwen3.6-35b-a3b", + base_url: OPENROUTER, + client: "openai", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 262144, + pricing: usd(0.14, 1.0), + }, + { + model: "qwen/qwen3-embedding-4b", + base_url: OPENROUTER, + client: "openai-embedding", + input_modalities: ["Text"], + output_modalities: ["Embed"], + context_window: 32768, + pricing: usd(0.02, 0.0), + }, + { + model: "stepfun/step-3.7-flash", + base_url: OPENROUTER, + client: "openai", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 262144, + pricing: usd(0.2, 1.15, 0.04), + }, + { + model: "tencent/hy3", + base_url: OPENROUTER, + client: "openai", + input_modalities: ["Text"], + output_modalities: ["Text"], + context_window: 262144, + pricing: usd(0.14, 0.58, 0.035), + }, + { + model: "x-ai/grok-4.5", + base_url: OPENROUTER, + client: "openai", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 500000, + pricing: usd(2.0, 6.0, 0.3), + }, + { + model: "xiaomi/mimo-v2.5", + base_url: OPENROUTER, + client: "openai", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 1050000, + pricing: usd(0.14, 0.28, 0.0028), + }, + { + model: "z-ai/glm-5.2", + base_url: OPENROUTER, + client: "glm-5.2", + input_modalities: ["Text"], + output_modalities: ["Text"], + context_window: 1048576, + pricing: usd(0.8204, 2.5784, 0.15236), + }, + { + model: "z-ai/glm-5.1", + base_url: OPENROUTER, + client: "glm-5.1", + input_modalities: ["Text"], + output_modalities: ["Text"], + context_window: 204800, + pricing: usd(0.966, 3.036, 0.1794), + }, + // SiliconFlow (official CNY price lists pre-converted to USD; no public pricing API) + { + model: "deepseek-ai/DeepSeek-V4-Flash", + base_url: SILICONFLOW, + client: "deepseek-v4", + input_modalities: ["Text"], + output_modalities: ["Text"], + context_window: 1000000, + pricing: cny(1.0, 2.0, 0.02), + }, + { + model: "deepseek-ai/DeepSeek-V4-Pro", + base_url: SILICONFLOW, + client: "deepseek-v4", + input_modalities: ["Text"], + output_modalities: ["Text"], + context_window: 1000000, + pricing: cny(12.0, 24.0, 0.1), + }, + { + model: "meituan-longcat/LongCat-2.0", + base_url: SILICONFLOW, + client: "openai", + input_modalities: ["Text"], + output_modalities: ["Text"], + context_window: 1000000, + pricing: cny(5.0, 20.0, 0.1), + }, + { + model: "moonshotai/Kimi-K2.7-Code", + base_url: SILICONFLOW, + client: "openai", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 262144, + pricing: cny(6.5, 27.0, 1.3), + }, + { + model: "zai-org/GLM-5.2", + base_url: SILICONFLOW, + client: "glm-5.2", + input_modalities: ["Text"], + output_modalities: ["Text"], + context_window: 1000000, + pricing: cny(8.0, 28.0, 2.0), + }, + { + model: "Pro/zai-org/GLM-5.1", + base_url: SILICONFLOW, + client: "glm-5.1", + input_modalities: ["Text"], + output_modalities: ["Text"], + context_window: 200000, + }, + { + model: "Pro/moonshotai/Kimi-K2.6", + base_url: SILICONFLOW, + client: "kimi-k2.6", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 262144, + }, + { + model: "Qwen/Qwen3.6-35B-A3B", + base_url: SILICONFLOW, + client: "openai", + input_modalities: ["Text", "Image"], + output_modalities: ["Text"], + context_window: 262144, + }, + { + model: "Qwen/Qwen3-Embedding-8B", + base_url: SILICONFLOW, + client: "openai-embedding", + input_modalities: ["Text"], + output_modalities: ["Embed"], + }, +]; + +function convertPricing(pricing: ModelPricing, currency: Currency): ModelPricing { + if (currency === "USD") { + return { ...pricing }; + } + + const round = (v: number): number => Math.round(v * CNY_PER_USD * 1e6) / 1e6; + return { + currency: "CNY", + prompt_tokens: round(pricing.prompt_tokens), + thoughts_tokens: round(pricing.thoughts_tokens), + response_tokens: round(pricing.response_tokens), + ...(pricing.cached_tokens !== undefined + ? { cached_tokens: round(pricing.cached_tokens) } + : {}), + }; +} + +/** + * List supported models with base URL, client, modalities, context window, and + * pricing. + * + * Covers the official vendor endpoints plus the OpenRouter and SiliconFlow + * platforms; `client` is the `clientType` token that routes the model to its + * protocol client. Prices are per million tokens for AgentHub's usage buckets + * (cached_tokens, prompt_tokens, thoughts_tokens, response_tokens), stored in + * USD and converted to `currency` at 7 CNY/USD on request. + */ +export function listSupportedModels(currency: Currency = "USD"): SupportedModel[] { + return SUPPORTED_MODELS.map((entry) => ({ + ...entry, + input_modalities: [...entry.input_modalities], + output_modalities: [...entry.output_modalities], + ...(entry.pricing ? { pricing: convertPricing(entry.pricing, currency) } : {}), + })); +} diff --git a/src_ts/tests/client.test.ts b/src_ts/tests/client.test.ts index 9d8ac9c7..fdbcbcde 100644 --- a/src_ts/tests/client.test.ts +++ b/src_ts/tests/client.test.ts @@ -13,6 +13,7 @@ // limitations under the License. import { AutoLLMClient } from "../src/autoClient"; +import { listSupportedModels } from "../src/registry"; import { ThinkingLevel, UniMessage, UniConfig, UniEvent } from "../src/types"; import { expect, describe, test } from "@jest/globals"; @@ -41,10 +42,11 @@ interface Model { const AVAILABLE_MODELS: Model[] = []; if (process.env.GEMINI_API_KEY) { + AVAILABLE_MODELS.push({ - name: "gemini-3.5-flash", + name: "gemini-3.6-flash", supportTextGeneration: true, - supportTemperature: true, + supportTemperature: false, supportImageUnderstanding: true, supportImageGeneration: false, supportAudioGeneration: false, @@ -52,8 +54,9 @@ if (process.env.GEMINI_API_KEY) { provider: "official", }); + AVAILABLE_MODELS.push({ - name: "gemini-3.1-flash-image-preview", + name: "gemini-3.1-flash-image", supportTextGeneration: false, supportTemperature: false, supportImageUnderstanding: false, @@ -126,7 +129,7 @@ if (process.env.OPENAI_API_KEY) { if (process.env.ZAI_API_KEY) { AVAILABLE_MODELS.push({ - name: "glm-5.1", + name: "glm-5.2", supportTextGeneration: true, supportTemperature: true, supportImageUnderstanding: false, @@ -139,7 +142,7 @@ if (process.env.ZAI_API_KEY) { if (process.env.MOONSHOT_API_KEY) { AVAILABLE_MODELS.push({ - name: "kimi-k2.6", + name: "kimi-k3", supportTextGeneration: true, supportTemperature: false, supportImageUnderstanding: true, @@ -178,9 +181,9 @@ if (process.env.BEDROCK_API_KEY) { if (process.env.VERTEX_API_KEY) { AVAILABLE_MODELS.push({ - name: "gemini-3.5-flash", + name: "gemini-3.6-flash", supportTextGeneration: true, - supportTemperature: true, + supportTemperature: false, supportImageUnderstanding: true, supportImageGeneration: false, supportAudioGeneration: false, @@ -189,7 +192,7 @@ if (process.env.VERTEX_API_KEY) { }); AVAILABLE_MODELS.push({ - name: "gemini-3.1-flash-image-preview", + name: "gemini-3.1-flash-image", supportTextGeneration: false, supportTemperature: false, supportImageUnderstanding: false, @@ -215,7 +218,7 @@ const RUN_SLOW_TEST = process.env.RUN_SLOW_TEST === "1"; if (process.env.OPENROUTER_API_KEY && RUN_SLOW_TEST) { AVAILABLE_MODELS.push({ - name: "z-ai/glm-5.1", + name: "z-ai/glm-5.2", supportTextGeneration: true, supportTemperature: true, supportImageUnderstanding: false, @@ -247,7 +250,7 @@ if (process.env.OPENROUTER_API_KEY && RUN_SLOW_TEST) { provider: "openrouter", }); AVAILABLE_MODELS.push({ - name: "moonshotai/kimi-k2.6", + name: "moonshotai/kimi-k3", supportTextGeneration: true, supportTemperature: false, supportImageUnderstanding: true, @@ -260,7 +263,7 @@ if (process.env.OPENROUTER_API_KEY && RUN_SLOW_TEST) { if (process.env.SILICONFLOW_API_KEY && RUN_SLOW_TEST) { AVAILABLE_MODELS.push({ - name: "Pro/zai-org/GLM-5.1", + name: "zai-org/GLM-5.2", supportTextGeneration: true, supportTemperature: true, supportImageUnderstanding: false, @@ -1048,6 +1051,51 @@ test("should reject unknown model", () => { ); }); +test("should list supported model entries", () => { + const entries = listSupportedModels(); + const kimi = entries.find((entry) => entry.model === "kimi-k3"); + expect(kimi).toBeDefined(); + expect(kimi?.base_url).toBe("https://api.moonshot.cn/v1"); + expect(kimi?.client).toBe("kimi-k3"); + expect(kimi?.context_window).toBe(1048576); + expect(kimi?.input_modalities).toEqual(["Text", "Image"]); + expect(kimi?.output_modalities).toEqual(["Text"]); + // stored in USD (official CNY prices pre-converted at 7 CNY/USD) + expect(kimi?.pricing).toEqual({ + currency: "USD", + prompt_tokens: 2.857143, + thoughts_tokens: 14.285714, + response_tokens: 14.285714, + cached_tokens: 0.285714, + }); + + const kimiCny = listSupportedModels("CNY").find( + (entry) => entry.model === "kimi-k3", + ); + expect(kimiCny?.pricing?.currency).toBe("CNY"); + expect(kimiCny?.pricing?.prompt_tokens).toBeCloseTo(20.0, 3); + expect(kimiCny?.pricing?.thoughts_tokens).toBeCloseTo(100.0, 3); + expect(kimiCny?.pricing?.response_tokens).toBeCloseTo(100.0, 3); + expect(kimiCny?.pricing?.cached_tokens).toBeCloseTo(2.0, 3); + + const glm52 = entries.find((entry) => entry.model === "z-ai/glm-5.2"); + expect(glm52?.base_url).toBe("https://openrouter.ai/api/v1"); + expect(glm52?.client).toBe("glm-5.2"); + + for (const entry of entries) { + expect(entry.input_modalities.length).toBeGreaterThan(0); + expect(entry.output_modalities.length).toBeGreaterThan(0); + const client = new AutoLLMClient({ + model: entry.model, + apiKey: "test-key", + baseUrl: entry.base_url, + clientType: entry.client, + }); + // eslint-disable-next-line @typescript-eslint/no-explicit-any + expect((client as any)._client).toBeDefined(); + } +}); + test.each([ ["openai-compatible", "OpenaiClient"], ["openai-embedding-compatible", "OpenaiEmbeddingClient"],