Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: Bug report
description: Something isn't working as expected
labels: ["bug"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to report a bug. Please fill in as much detail as you can.

- type: textarea
id: description
attributes:
label: What happened?
description: A clear description of the bug.
validations:
required: true

- type: textarea
id: reproduction
attributes:
label: Steps to reproduce
description: How do we reproduce the issue?
placeholder: |
1. Start opencode with the plugin loaded
2. Send a request to POST /v1/chat/completions with ...
3. See error
validations:
required: true

- type: textarea
id: expected
attributes:
label: Expected behaviour
description: What did you expect to happen?
validations:
required: true

- type: textarea
id: request
attributes:
label: Request / response (if applicable)
description: Paste the curl command or request body and the response you received.
render: bash

- type: input
id: version
attributes:
label: opencode-llm-proxy version
placeholder: "e.g. 1.6.1"
validations:
required: true

- type: input
id: runtime
attributes:
label: Runtime and OS
placeholder: "e.g. Node.js 22, macOS 14 / Bun 1.2, Ubuntu 24.04"
validations:
required: true

- type: input
id: provider
attributes:
label: Provider / model
placeholder: "e.g. github-copilot/claude-sonnet-4.6"
43 changes: 43 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Feature request
description: Suggest a new feature or improvement
labels: ["enhancement"]
body:
- type: markdown
attributes:
value: |
Thanks for the suggestion! Please describe the use case clearly so we can understand what you need.

- type: textarea
id: problem
attributes:
label: What problem does this solve?
description: Describe the situation where this would be useful.
placeholder: "e.g. I use the Vercel AI SDK and currently have to..."
validations:
required: true

- type: textarea
id: solution
attributes:
label: Proposed solution
description: What would you like to see added or changed?
validations:
required: true

- type: textarea
id: alternatives
attributes:
label: Alternatives you've considered
description: Any workarounds you're using today?

- type: dropdown
id: api_format
attributes:
label: Which API format does this relate to? (if any)
options:
- OpenAI Chat Completions
- OpenAI Responses API
- Anthropic Messages API
- Google Gemini
- All / general
- Not API-format specific
82 changes: 82 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Contributing

Thanks for your interest in contributing to opencode-llm-proxy.

## Getting started

```bash
git clone https://github.com/KochC/opencode-llm-proxy.git
cd opencode-llm-proxy
npm install
```

Run the tests:

```bash
npm test
```

Run the linter:

```bash
npm run lint
```

## How to contribute

### Reporting bugs

Open a [bug report](https://github.com/KochC/opencode-llm-proxy/issues/new?template=bug_report.yml). Include:

- What you did
- What you expected
- What actually happened
- Your Node.js / Bun version and OS

### Suggesting features

Open a [feature request](https://github.com/KochC/opencode-llm-proxy/issues/new?template=feature_request.yml) describing the use case.

### Submitting a pull request

1. Fork the repo and create a branch from `dev` (not `main`)
2. Make your changes
3. Add or update tests in `index.test.js` — all 112+ tests must pass
4. Lint passes: `npm run lint`
5. Commit using [Conventional Commits](https://www.conventionalcommits.org/):
- `fix:` for bug fixes (triggers a patch release)
- `feat:` for new features (triggers a minor release)
- `docs:` / `chore:` / `test:` for everything else (no release)
6. Open a PR against the `dev` branch

## Branch model

```
dev ──► main ──► npm (via Release Please)
```

- All work goes on `dev`
- `main` is release-only — only Release Please PRs merge directly there
- Do not open PRs against `main`

## Tests

Tests use the Node.js built-in test runner — no external framework needed.

```bash
node --test # run once
node --test --watch # watch mode
node --test --experimental-test-coverage # with coverage
```

Tests mock the OpenCode SDK client entirely — no real LLM calls are made.

## Code style

ESLint enforces style. Run `npm run lint` before pushing. The config is in `eslint.config.js`.

Key conventions in the codebase:

- Pure functions are exported for testability (`normalizeMessages`, `buildPrompt`, etc.)
- Each API format (OpenAI, Anthropic, Gemini) has its own section in `index.js`
- Error responses mirror the format of the target API (OpenAI errors for `/v1/*`, Anthropic errors for `/v1/messages`, etc.)
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2025 KochC

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
99 changes: 91 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@
[![CI](https://github.com/KochC/opencode-llm-proxy/actions/workflows/ci.yml/badge.svg)](https://github.com/KochC/opencode-llm-proxy/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

**One local endpoint. Every model you have access to. Any API format.**
**One local endpoint. Every model you have access to. Any API format. Tool calling included.**

opencode-llm-proxy is an [OpenCode](https://opencode.ai) plugin that starts a local HTTP server on `http://127.0.0.1:4010`. It translates between the API format your tool speaks and whichever LLM provider OpenCode has configured — so you never reconfigure the same models twice.

```
Your tool (OpenAI / Anthropic / Gemini SDK)
Your tool (OpenAI / Anthropic / Gemini SDK, coding agent, etc.)
▼ http://127.0.0.1:4010
opencode-llm-proxy
Expand All @@ -19,7 +19,7 @@ Your tool (OpenAI / Anthropic / Gemini SDK)
GitHub Copilot · Anthropic · Gemini · Ollama · OpenRouter · Bedrock · …
```

**Supported API formats — all with streaming:**
**Supported API formats — all with streaming and [tool/function calling](#tool-calling):**

| Format | Endpoint |
|---|---|
Expand All @@ -28,6 +28,24 @@ Your tool (OpenAI / Anthropic / Gemini SDK)
| Anthropic Messages API | `POST /v1/messages` |
| Google Gemini | `POST /v1beta/models/:model:generateContent` |

**✨ Tool calling works with all four formats** — point a coding agent (Claude Code, Cursor, Continue, Cline, your own agent loop, ...) at the proxy and its `tools`/`tool_choice` calls are translated through to whatever model OpenCode has configured, with a real `tool_calls` / `tool_use` / `functionCall` response handed back. See [Tool calling](#tool-calling).

---

## Contents

- [Why](#why)
- [Quickstart](#quickstart)
- [Install](#install)
- [Configuration](#configuration)
- [Tool calling](#tool-calling)
- [Using with SDKs and tools](#using-with-sdks-and-tools)
- [Finding model IDs](#finding-model-ids)
- [API reference](#api-reference)
- [How it works](#how-it-works)
- [Limitations](#limitations)
- [License](#license)

---

## Why
Expand All @@ -41,6 +59,7 @@ Most LLM tools speak exactly one API dialect. OpenCode already manages connectio
- You want to **swap models without code changes**. Your app talks to the proxy; you change the model in OpenCode config.
- You want to **share your models on a LAN**. Expose the proxy on `0.0.0.0` and give teammates the URL.
- You use the **Anthropic SDK** but want to route through GitHub Copilot or Bedrock. No code change in the SDK — just point it at the proxy.
- You're building or running a **coding agent** that needs real tool/function calling (read files, run shell commands, etc.) against whatever model OpenCode has configured. See [Tool calling](#tool-calling).

---

Expand Down Expand Up @@ -110,6 +129,8 @@ curl -o .opencode/plugins/llm-proxy.js \
https://raw.githubusercontent.com/KochC/opencode-llm-proxy/main/index.js
```

> Copying just `index.js` works for everything except [tool calling](#tool-calling), which also needs `mcp-tool-bridge.js` alongside it. Use the npm plugin install method if you want tool calling.

---

## Configuration
Expand All @@ -120,6 +141,7 @@ curl -o .opencode/plugins/llm-proxy.js \
| `OPENCODE_LLM_PROXY_PORT` | `4010` | TCP port. |
| `OPENCODE_LLM_PROXY_TOKEN` | _(unset)_ | Bearer token required on every request. Unset = no auth. |
| `OPENCODE_LLM_PROXY_CORS_ORIGIN` | `*` | `Access-Control-Allow-Origin` value for browser clients. |
| `OPENCODE_LLM_PROXY_TOOL_BRIDGE_POOL_SIZE` | `8` | Max concurrent in-flight requests using [tool calling](#tool-calling). |

```bash
OPENCODE_LLM_PROXY_HOST=0.0.0.0 \
Expand All @@ -129,6 +151,67 @@ opencode

---

## Tool calling

The proxy supports real tool/function calling on **all four API formats** — OpenAI function tools (`tools` on `/v1/chat/completions` and `/v1/responses`), Anthropic tools (`tools` on `/v1/messages`), and Gemini function declarations (`tools` on `:generateContent`/`:streamGenerateContent`). This is what lets coding agents and other tool-using clients work through the proxy, not just plain chat.

```bash
curl http://127.0.0.1:4010/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "github-copilot/claude-sonnet-4.6",
"messages": [{"role": "user", "content": "What is the weather in NYC?"}],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city",
"parameters": {
"type": "object",
"properties": { "city": { "type": "string" } },
"required": ["city"]
}
}
}]
}'
```

```json
{
"choices": [{
"finish_reason": "tool_calls",
"message": {
"role": "assistant",
"content": null,
"tool_calls": [{
"id": "call_...",
"type": "function",
"function": { "name": "get_weather", "arguments": "{\"city\":\"NYC\"}" }
}]
}
}]
}
```

Send the tool's result back on your next request (`role: "tool"` / `tool_result` / `functionResponse`, per your API's convention) alongside the full conversation history, same as any other multi-turn request — the proxy is stateless between calls either way.

### How tool calling works under the hood

OpenCode's own agent loop always executes tools itself, server-side, so there's no native concept of a "client-executed" tool call to hand off to. To bridge that gap, when a request includes `tools`:

1. The proxy dynamically registers a small local [MCP](https://opencode.ai/docs/mcp-servers/) server whose tool list is exactly your declared tool schemas (see `mcp-tool-bridge.js`).
2. Only those tools are enabled for that one prompt call — every built-in OpenCode tool stays disabled, same as always.
3. As soon as the model proposes calling one of your tools, the proxy immediately aborts the OpenCode session (before the bridge's no-op handler is ever consulted) and translates the captured call name + arguments into your API's tool-call shape — `tool_calls` (OpenAI), `tool_use` (Anthropic), or a `functionCall` part (Gemini) — instead of a text answer.

### Notes and current limitations

- One tool call per turn — parallel/multiple simultaneous tool calls aren't supported.
- `tool_choice: "none"` (OpenAI/Gemini `mode: "NONE"`/Anthropic `type: "none"`) disables tool calling for that request; forcing a specific named tool is supported.
- Bridge servers are reused from a small fixed-size pool (`px_tools_0`, `px_tools_1`, ...) rather than registered fresh per request, since OpenCode's server API has no endpoint to deregister an MCP server once added. Configure the pool size with `OPENCODE_LLM_PROXY_TOOL_BRIDGE_POOL_SIZE` (default `8`) if you expect more than 8 concurrent in-flight tool-calling requests.
- The bridge process is spawned with `node`, so `node` must be on `PATH` wherever OpenCode is running.

---

## Using with SDKs and tools

### OpenAI SDK (JS/TS)
Expand Down Expand Up @@ -310,18 +393,18 @@ x-opencode-provider: anthropic
Returns all models from all configured providers in OpenAI list format.

### POST /v1/chat/completions
OpenAI Chat Completions. Required fields: `model`, `messages`. Optional: `stream`, `temperature`, `max_tokens`.
OpenAI Chat Completions. Required fields: `model`, `messages`. Optional: `stream`, `temperature`, `max_tokens`, `tools`, `tool_choice`.

### POST /v1/responses
OpenAI Responses API. Required fields: `model`, `input`. Optional: `instructions`, `stream`, `max_output_tokens`.
OpenAI Responses API. Required fields: `model`, `input`. Optional: `instructions`, `stream`, `max_output_tokens`, `tools`, `tool_choice`.

### POST /v1/messages
Anthropic Messages API. Required fields: `model`, `messages`. Optional: `system`, `max_tokens`, `stream`.
Anthropic Messages API. Required fields: `model`, `messages`. Optional: `system` (string or array of `{type: "text", text: string}` content blocks), `max_tokens`, `stream`, `tools`, `tool_choice`.

Errors are returned in Anthropic format: `{ "type": "error", "error": { "type": "...", "message": "..." } }`.

### POST /v1beta/models/:model:generateContent
Google Gemini non-streaming. Model name in URL path. Required field: `contents`. Optional: `systemInstruction`, `generationConfig`.
Google Gemini non-streaming. Model name in URL path. Required field: `contents`. Optional: `systemInstruction`, `generationConfig`, `tools`, `toolConfig`.

### POST /v1beta/models/:model:streamGenerateContent
Same as above, returns newline-delimited JSON stream.
Expand All @@ -345,9 +428,9 @@ Streaming uses OpenCode's `client.event.subscribe()` SSE stream. Text deltas are
## Limitations

- Text only — image, audio, and file inputs are ignored
- No tool/function calling — all OpenCode tools are disabled for proxy sessions
- No cross-request session state — send full conversation history on every request
- Temperature and max tokens are advisory (passed as system prompt hints)
- Tool calling supports one call per turn — see [Tool calling](#tool-calling) above

---

Expand Down
Loading