Skip to content

Commit d01cf20

Browse files
KochCFramewrk CI
andauthored
feat: add tool/function calling support (#52)
* feat: add tool/function calling support (closes #50) Adds OpenAI-style function tools, Anthropic tools, and Gemini function declarations across all four API surfaces (/v1/chat/completions, /v1/responses, /v1/messages, /v1beta/models/:model:generateContent), both streaming and non-streaming. OpenCode's own agent loop always executes tools itself, server-side, and has no concept of a 'client-executed' tool call to hand off to a caller. To bridge that gap: - When a request includes tools, the proxy dynamically registers a small local MCP server (mcp-tool-bridge.js) whose tool list is exactly the caller's declared tool schemas, reused from a small fixed-size pool of slot names (OpenCode's server API has no endpoint to deregister an MCP server once added). - Only those tools are enabled for that one prompt call via the existing tools enable/disable map; every built-in OpenCode tool stays disabled, same as before. - As soon as the model proposes calling one of the bridge tools, the full call (name + arguments) is already present on OpenCode's event stream (ToolStatePending includes the parsed input even before execution starts) - the proxy captures it and immediately aborts the session before the bridge's no-op tools/call handler would ever be consulted, then translates the call into the caller's expected tool_calls / tool_use / functionCall shape instead of a text answer. Also extends the OpenAI/Anthropic/Gemini/Responses message normalizers to render prior tool_calls/tool_use/functionCall and their results/tool_result/functionResponse as descriptive text when replaying conversation history, so multi-turn tool use works end-to-end even though sessions are stateless per-request. - index.js: parseOpenAITools/parseAnthropicTools/parseGeminiTools + applyOpenAIToolChoice/applyAnthropicToolChoice/applyGeminiToolChoice, sanitizeToolName, tool bridge pool + registerToolBridge, unified runAgentTurn (event-driven turn execution shared by executePrompt and executePromptStreaming when tools are present), tool-call branches in all four response builders and SSE emitters. - mcp-tool-bridge.js: minimal MCP stdio JSON-RPC server exposing caller-supplied tool schemas; tools/call is a harmless no-op since the proxy aborts the session before it would ever be consulted. - index.test.js: unit tests for the new parse/tool_choice helpers and history round-tripping, plus end-to-end tool-calling tests for all four API formats (stream + non-stream). - README.md: documents the new tools/tool_choice/toolConfig request fields, how the bridge mechanism works, its current limitations, and the new OPENCODE_LLM_PROXY_TOOL_BRIDGE_POOL_SIZE env var. Testing: - npm test — 138 passed (116 existing + 22 new) - npm run lint — clean * docs: feature tool calling prominently in README, expand discoverability keywords - Move the Tool calling section up (right after Configuration) and add a runnable curl request/response example, instead of burying it near the bottom after How it works. - Add a Contents section now that the README has grown to 10+ sections. - Call out tool calling in the top-level tagline, architecture diagram, supported-formats table, and Why section (coding agents are now a first-class use case, not just chat clients). - Note in Install that copying just index.js doesn't get you tool calling (needs mcp-tool-bridge.js alongside it) - use the npm plugin instead. - package.json: mention tool/function calling in the description and add tool-calling/function-calling/tools/mcp/model-context-protocol/ coding-agent/ai-agent/agentic keywords for npm search discoverability. --------- Co-authored-by: Framewrk CI <ci@framewrklabs.ai>
1 parent b7402ca commit d01cf20

5 files changed

Lines changed: 1307 additions & 119 deletions

File tree

README.md

Lines changed: 91 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,12 @@
55
[![CI](https://github.com/KochC/opencode-llm-proxy/actions/workflows/ci.yml/badge.svg)](https://github.com/KochC/opencode-llm-proxy/actions/workflows/ci.yml)
66
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
77

8-
**One local endpoint. Every model you have access to. Any API format.**
8+
**One local endpoint. Every model you have access to. Any API format. Tool calling included.**
99

1010
opencode-llm-proxy is an [OpenCode](https://opencode.ai) plugin that starts a local HTTP server on `http://127.0.0.1:4010`. It translates between the API format your tool speaks and whichever LLM provider OpenCode has configured — so you never reconfigure the same models twice.
1111

1212
```
13-
Your tool (OpenAI / Anthropic / Gemini SDK)
13+
Your tool (OpenAI / Anthropic / Gemini SDK, coding agent, etc.)
1414
1515
▼ http://127.0.0.1:4010
1616
opencode-llm-proxy
@@ -19,7 +19,7 @@ Your tool (OpenAI / Anthropic / Gemini SDK)
1919
GitHub Copilot · Anthropic · Gemini · Ollama · OpenRouter · Bedrock · …
2020
```
2121

22-
**Supported API formats — all with streaming:**
22+
**Supported API formats — all with streaming and [tool/function calling](#tool-calling):**
2323

2424
| Format | Endpoint |
2525
|---|---|
@@ -28,6 +28,24 @@ Your tool (OpenAI / Anthropic / Gemini SDK)
2828
| Anthropic Messages API | `POST /v1/messages` |
2929
| Google Gemini | `POST /v1beta/models/:model:generateContent` |
3030

31+
**✨ Tool calling works with all four formats** — point a coding agent (Claude Code, Cursor, Continue, Cline, your own agent loop, ...) at the proxy and its `tools`/`tool_choice` calls are translated through to whatever model OpenCode has configured, with a real `tool_calls` / `tool_use` / `functionCall` response handed back. See [Tool calling](#tool-calling).
32+
33+
---
34+
35+
## Contents
36+
37+
- [Why](#why)
38+
- [Quickstart](#quickstart)
39+
- [Install](#install)
40+
- [Configuration](#configuration)
41+
- [Tool calling](#tool-calling)
42+
- [Using with SDKs and tools](#using-with-sdks-and-tools)
43+
- [Finding model IDs](#finding-model-ids)
44+
- [API reference](#api-reference)
45+
- [How it works](#how-it-works)
46+
- [Limitations](#limitations)
47+
- [License](#license)
48+
3149
---
3250

3351
## Why
@@ -41,6 +59,7 @@ Most LLM tools speak exactly one API dialect. OpenCode already manages connectio
4159
- You want to **swap models without code changes**. Your app talks to the proxy; you change the model in OpenCode config.
4260
- You want to **share your models on a LAN**. Expose the proxy on `0.0.0.0` and give teammates the URL.
4361
- You use the **Anthropic SDK** but want to route through GitHub Copilot or Bedrock. No code change in the SDK — just point it at the proxy.
62+
- You're building or running a **coding agent** that needs real tool/function calling (read files, run shell commands, etc.) against whatever model OpenCode has configured. See [Tool calling](#tool-calling).
4463

4564
---
4665

@@ -110,6 +129,8 @@ curl -o .opencode/plugins/llm-proxy.js \
110129
https://raw.githubusercontent.com/KochC/opencode-llm-proxy/main/index.js
111130
```
112131

132+
> Copying just `index.js` works for everything except [tool calling](#tool-calling), which also needs `mcp-tool-bridge.js` alongside it. Use the npm plugin install method if you want tool calling.
133+
113134
---
114135

115136
## Configuration
@@ -120,6 +141,7 @@ curl -o .opencode/plugins/llm-proxy.js \
120141
| `OPENCODE_LLM_PROXY_PORT` | `4010` | TCP port. |
121142
| `OPENCODE_LLM_PROXY_TOKEN` | _(unset)_ | Bearer token required on every request. Unset = no auth. |
122143
| `OPENCODE_LLM_PROXY_CORS_ORIGIN` | `*` | `Access-Control-Allow-Origin` value for browser clients. |
144+
| `OPENCODE_LLM_PROXY_TOOL_BRIDGE_POOL_SIZE` | `8` | Max concurrent in-flight requests using [tool calling](#tool-calling). |
123145

124146
```bash
125147
OPENCODE_LLM_PROXY_HOST=0.0.0.0 \
@@ -129,6 +151,67 @@ opencode
129151

130152
---
131153

154+
## Tool calling
155+
156+
The proxy supports real tool/function calling on **all four API formats** — OpenAI function tools (`tools` on `/v1/chat/completions` and `/v1/responses`), Anthropic tools (`tools` on `/v1/messages`), and Gemini function declarations (`tools` on `:generateContent`/`:streamGenerateContent`). This is what lets coding agents and other tool-using clients work through the proxy, not just plain chat.
157+
158+
```bash
159+
curl http://127.0.0.1:4010/v1/chat/completions \
160+
-H "Content-Type: application/json" \
161+
-d '{
162+
"model": "github-copilot/claude-sonnet-4.6",
163+
"messages": [{"role": "user", "content": "What is the weather in NYC?"}],
164+
"tools": [{
165+
"type": "function",
166+
"function": {
167+
"name": "get_weather",
168+
"description": "Get the current weather for a city",
169+
"parameters": {
170+
"type": "object",
171+
"properties": { "city": { "type": "string" } },
172+
"required": ["city"]
173+
}
174+
}
175+
}]
176+
}'
177+
```
178+
179+
```json
180+
{
181+
"choices": [{
182+
"finish_reason": "tool_calls",
183+
"message": {
184+
"role": "assistant",
185+
"content": null,
186+
"tool_calls": [{
187+
"id": "call_...",
188+
"type": "function",
189+
"function": { "name": "get_weather", "arguments": "{\"city\":\"NYC\"}" }
190+
}]
191+
}
192+
}]
193+
}
194+
```
195+
196+
Send the tool's result back on your next request (`role: "tool"` / `tool_result` / `functionResponse`, per your API's convention) alongside the full conversation history, same as any other multi-turn request — the proxy is stateless between calls either way.
197+
198+
### How tool calling works under the hood
199+
200+
OpenCode's own agent loop always executes tools itself, server-side, so there's no native concept of a "client-executed" tool call to hand off to. To bridge that gap, when a request includes `tools`:
201+
202+
1. The proxy dynamically registers a small local [MCP](https://opencode.ai/docs/mcp-servers/) server whose tool list is exactly your declared tool schemas (see `mcp-tool-bridge.js`).
203+
2. Only those tools are enabled for that one prompt call — every built-in OpenCode tool stays disabled, same as always.
204+
3. As soon as the model proposes calling one of your tools, the proxy immediately aborts the OpenCode session (before the bridge's no-op handler is ever consulted) and translates the captured call name + arguments into your API's tool-call shape — `tool_calls` (OpenAI), `tool_use` (Anthropic), or a `functionCall` part (Gemini) — instead of a text answer.
205+
206+
### Notes and current limitations
207+
208+
- One tool call per turn — parallel/multiple simultaneous tool calls aren't supported.
209+
- `tool_choice: "none"` (OpenAI/Gemini `mode: "NONE"`/Anthropic `type: "none"`) disables tool calling for that request; forcing a specific named tool is supported.
210+
- Bridge servers are reused from a small fixed-size pool (`px_tools_0`, `px_tools_1`, ...) rather than registered fresh per request, since OpenCode's server API has no endpoint to deregister an MCP server once added. Configure the pool size with `OPENCODE_LLM_PROXY_TOOL_BRIDGE_POOL_SIZE` (default `8`) if you expect more than 8 concurrent in-flight tool-calling requests.
211+
- The bridge process is spawned with `node`, so `node` must be on `PATH` wherever OpenCode is running.
212+
213+
---
214+
132215
## Using with SDKs and tools
133216

134217
### OpenAI SDK (JS/TS)
@@ -310,18 +393,18 @@ x-opencode-provider: anthropic
310393
Returns all models from all configured providers in OpenAI list format.
311394

312395
### POST /v1/chat/completions
313-
OpenAI Chat Completions. Required fields: `model`, `messages`. Optional: `stream`, `temperature`, `max_tokens`.
396+
OpenAI Chat Completions. Required fields: `model`, `messages`. Optional: `stream`, `temperature`, `max_tokens`, `tools`, `tool_choice`.
314397

315398
### POST /v1/responses
316-
OpenAI Responses API. Required fields: `model`, `input`. Optional: `instructions`, `stream`, `max_output_tokens`.
399+
OpenAI Responses API. Required fields: `model`, `input`. Optional: `instructions`, `stream`, `max_output_tokens`, `tools`, `tool_choice`.
317400

318401
### POST /v1/messages
319-
Anthropic Messages API. Required fields: `model`, `messages`. Optional: `system` (string or array of `{type: "text", text: string}` content blocks), `max_tokens`, `stream`.
402+
Anthropic Messages API. Required fields: `model`, `messages`. Optional: `system` (string or array of `{type: "text", text: string}` content blocks), `max_tokens`, `stream`, `tools`, `tool_choice`.
320403

321404
Errors are returned in Anthropic format: `{ "type": "error", "error": { "type": "...", "message": "..." } }`.
322405

323406
### POST /v1beta/models/:model:generateContent
324-
Google Gemini non-streaming. Model name in URL path. Required field: `contents`. Optional: `systemInstruction`, `generationConfig`.
407+
Google Gemini non-streaming. Model name in URL path. Required field: `contents`. Optional: `systemInstruction`, `generationConfig`, `tools`, `toolConfig`.
325408

326409
### POST /v1beta/models/:model:streamGenerateContent
327410
Same as above, returns newline-delimited JSON stream.
@@ -345,9 +428,9 @@ Streaming uses OpenCode's `client.event.subscribe()` SSE stream. Text deltas are
345428
## Limitations
346429

347430
- Text only — image, audio, and file inputs are ignored
348-
- No tool/function calling — all OpenCode tools are disabled for proxy sessions
349431
- No cross-request session state — send full conversation history on every request
350432
- Temperature and max tokens are advisory (passed as system prompt hints)
433+
- Tool calling supports one call per turn — see [Tool calling](#tool-calling) above
351434

352435
---
353436

0 commit comments

Comments
 (0)