Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Copy to .env and fill in. .env is gitignored.
# Your OpenRouter key (https://openrouter.ai/keys). Required for non-Claude routes.
# Your OpenRouter key (https://openrouter.ai/keys). Required for openrouter: routes.
OPENROUTER_API_KEY=sk-or-v1-xxxxxxxx
# Optional: Z.ai GLM Coding Plan key (https://z.ai). Required only for zai: routes.
# ZAI_API_KEY=your-zai-api-key
# Optional: only needed if the live test shows Claude Code does NOT forward its
# subscription/OAuth auth through a custom ANTHROPIC_BASE_URL (see README).
# ANTHROPIC_API_KEY=sk-ant-xxxxxxxx
Expand Down
34 changes: 31 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,11 +151,38 @@ claude-review = "anthropic:claude-sonnet-5"
The slugs above are illustrative — run `modelmux check-latest` to see which
models actually exist on OpenRouter right now.

## Flat-rate GLM: bring a Z.ai subscription

If you lean on GLM, OpenRouter's per-token pricing adds up fast. **Z.ai's GLM
Coding Plan** is a flat monthly subscription (see [z.ai](https://z.ai) for
current plans) with an Anthropic-compatible endpoint built for Claude Code — and
`zai` is a **built-in upstream**, so it's zero-config. Set your key and point an
alias at it:

```bash
export ZAI_API_KEY=<your z.ai api key>
```

```toml
[models]
orchestrator = "anthropic:passthrough" # brain stays on Claude
flagship = "zai:glm-5.2" # subagents run on GLM via your subscription
```

Your `flagship` subagents now ride your Z.ai subscription instead of OpenRouter's
per-token meter, while the orchestrator stays on Claude. It's sanctioned — your
own key, your own subscription, Z.ai's own documented endpoint. No impersonation.

Two things to know: use Z.ai's **bare** slug (`zai:glm-5.2`), not OpenRouter's
`z-ai/glm-5.2` prefix; and modelmux strips Claude Code's `anthropic-beta` headers
by default (safe). If you'd rather keep them, override with
`[upstreams]`: `zai = { base = "https://api.z.ai/api/anthropic", auth = "bearer:ZAI_API_KEY", stripBeta = false }`.

## Local & self-hosted models

`anthropic` and `openrouter` are built in, but you can point an alias at any
**Anthropic-Messages-compatible** endpoint by declaring it under `[upstreams]`,
then using it like any other model (`<name>:<slug>`).
`anthropic`, `openrouter`, and `zai` are built in, but you can point an alias at
any **Anthropic-Messages-compatible** endpoint by declaring it under
`[upstreams]`, then using it like any other model (`<name>:<slug>`).

The obvious use is a **local model**. Recent Ollama (v0.14+) speaks the Anthropic
Messages API natively, so no translation shim is needed — run your grunt-work
Expand Down Expand Up @@ -210,6 +237,7 @@ Everything is controlled by `routes.toml` and a few environment variables:
| Variable | Default | Purpose |
|----------|---------|---------|
| `OPENROUTER_API_KEY` | unset | Required for any `openrouter:` route. If a request routes to OpenRouter while it's unset, that request fails with HTTP 400. |
| `ZAI_API_KEY` | unset | Required for any `zai:` route (Z.ai GLM Coding Plan). Same 400 behavior when unset. |
| `PORT` | `8787` | Listen port. |
| `MUX_ROUTES` | `./routes.toml` | Path to the routes config (also the first-run bootstrap target). |
| `MUX_LOG` | `./decisions.jsonl` | Path to the JSONL decision log — one line per proxied request. |
Expand Down
1 change: 1 addition & 0 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,7 @@ These apply to both `bun run proxy` and the compiled binary.
| `MUX_ROUTES` | Path to the routes TOML config. | `./routes.toml` |
| `MUX_LOG` | Path to the JSONL decision log written per request. | `./decisions.jsonl` |
| `OPENROUTER_API_KEY` | Required for any `openrouter:` route. | unset |
| `ZAI_API_KEY` | Required for any `zai:` route (built-in Z.ai GLM Coding Plan upstream). | unset |
| `MUX_MODEL_<ALIAS>` | Per-run override of one alias's target (uppercase alias, hyphens to underscores; value is a full `upstream:slug`). | unset |
| `ANTHROPIC_API_KEY` | If set, used as `x-api-key` on the anthropic leg; otherwise Claude Code's inbound auth is forwarded. | unset |

Expand Down
12 changes: 8 additions & 4 deletions routes.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,14 @@ cheap = "openrouter:deepseek/deepseek-v4-flash"
claude-review = "anthropic:claude-sonnet-5"

# --- upstreams (optional) ---------------------------------------------------
# "anthropic" (passthrough) and "openrouter" (Bearer OPENROUTER_API_KEY) are
# built in. Add your own Anthropic-Messages-compatible endpoints here, then use
# them like any other alias: <name>:<slug>. auth = passthrough | passthrough:ENV
# | bearer:ENV | none.
# "anthropic" (passthrough), "openrouter" (Bearer OPENROUTER_API_KEY), and "zai"
# (Z.ai GLM Coding Plan, Bearer ZAI_API_KEY) are built in. Add your own
# Anthropic-Messages-compatible endpoints here, then use them like any other
# alias: <name>:<slug>. auth = passthrough | passthrough:ENV | bearer:ENV | none.
#
# Z.AI SUBSCRIPTION (flat-rate, not per-token): set ZAI_API_KEY and point an alias
# at the built-in zai upstream (no [upstreams] block needed). Use the BARE slug:
# flagship = "zai:glm-5.2" # in [models] above — not "z-ai/glm-5.2"
#
# LOCAL MODELS: recent Ollama (v0.14+) speaks the Anthropic Messages API
# natively, so no translation shim is needed — point an alias at it and run your
Expand Down
6 changes: 5 additions & 1 deletion scripts/check-latest.ts
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,11 @@ export async function run(routesPath = "routes.toml"): Promise<number> {
const configured = Object.entries(config.models)
.filter(([, ref]) => ref.upstream === "openrouter")
.map(([alias, ref]) => ({ alias, slug: ref.slug }));
const skipped = Object.keys(config.models).length - configured.length; // zai / anthropic / local aren't checkable here

if (configured.length === 0) {
console.log("No OpenRouter models configured in routes.toml — nothing to check.");
const note = skipped > 0 ? ` (${skipped} non-openrouter model(s) can't be verified here.)` : "";
console.log(`No OpenRouter models configured in routes.toml — nothing to check.${note}`);
return 0;
}

Expand Down Expand Up @@ -97,6 +99,8 @@ export async function run(routesPath = "routes.toml"): Promise<number> {
else {
console.log("\nAll configured OpenRouter slugs exist in the live catalog.");
}
if (skipped > 0)
console.log(`\n(${skipped} non-openrouter model(s) not checked — check-latest only verifies OpenRouter.)`);
return stale > 0 ? 1 : 0;
}

Expand Down
4 changes: 2 additions & 2 deletions src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ export function parseAuth(spec: string): AuthMode {
throw new Error(`unknown auth "${spec}" (use passthrough | passthrough:ENV | bearer:ENV | none)`);
}

// Merge any user-declared [upstreams] over the built-ins (anthropic, openrouter).
// Merge any user-declared [upstreams] over the built-ins (anthropic, openrouter, zai).
function buildUpstreams(raw: Record<string, RawUpstream> | undefined): Record<string, UpstreamDef> {
const out: Record<string, UpstreamDef> = { ...BUILTIN_UPSTREAMS };
for (const [name, u] of Object.entries(raw ?? {})) {
Expand Down Expand Up @@ -134,7 +134,7 @@ export function loadConfig(
// Every model's upstream must be built in or declared in [upstreams].
for (const [alias, ref] of Object.entries(resolved.models)) {
if (!resolved.upstreams?.[ref.upstream])
throw new Error(`model "${alias}" uses unknown upstream "${ref.upstream}" — built-ins are anthropic and openrouter; add others under [upstreams]`);
throw new Error(`model "${alias}" uses unknown upstream "${ref.upstream}" — built-ins are anthropic, openrouter, and zai; add others under [upstreams]`);
}
return resolved;
}
Expand Down
4 changes: 2 additions & 2 deletions src/types.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// An upstream name. "anthropic" and "openrouter" are built in; more can be
// defined in the [upstreams] table of routes.toml (e.g. a local model server).
// An upstream name. "anthropic", "openrouter", and "zai" are built in; more can
// be defined in the [upstreams] table of routes.toml (e.g. a local model server).
export type Upstream = string;

// How modelmux authenticates the outbound leg to an upstream.
Expand Down
8 changes: 8 additions & 0 deletions src/upstreams.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,14 @@ export const BUILTIN_UPSTREAMS: Record<string, UpstreamDef> = {
auth: { kind: "bearer", envKey: "OPENROUTER_API_KEY" },
stripBeta: true, // OpenRouter's Anthropic endpoint may reject Claude Code's betas
},
// Z.ai GLM Coding Plan — a flat-rate subscription via Z.ai's Anthropic endpoint.
// Betas stripped by default to avoid 400s; override with stripBeta = false in
// [upstreams] if you want to keep Claude Code's betas.
zai: {
base: "https://api.z.ai/api/anthropic",
auth: { kind: "bearer", envKey: "ZAI_API_KEY" },
stripBeta: true,
},
};

const HOP_BY_HOP = new Set(["host", "content-length", "connection", "accept-encoding"]);
Expand Down
9 changes: 9 additions & 0 deletions test/config.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,15 @@ test("loadConfig parses [upstreams] and accepts a model that targets one", () =>
rmSync(tmp, { force: true });
});

test("loadConfig accepts a built-in zai model with no [upstreams] block", () => {
const tmp = "test/.tmp-zai.toml";
writeFileSync(tmp, toml({ orchestrator: "anthropic:passthrough", flagship: "zai:glm-5.2" }, "orchestrator"));
const cfg = loadConfig(tmp, {});
expect(cfg.models.flagship).toEqual({ upstream: "zai", slug: "glm-5.2" });
expect(cfg.upstreams?.zai?.base).toBe("https://api.z.ai/api/anthropic");
rmSync(tmp, { force: true });
});

test("loadConfig rejects a model whose upstream is neither built in nor declared", () => {
const tmp = "test/.tmp-bad-upstream.toml";
writeFileSync(tmp, toml({ a: "local:qwen" }, "a")); // no [upstreams] table -> "local" is undefined
Expand Down
33 changes: 33 additions & 0 deletions test/integration.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,39 @@ test("missing OpenRouter key fails loud with 400 AND logs an error record", asyn
noKey.stop(true);
});

test("built-in zai upstream routes a subagent to Z.ai with Bearer ZAI_API_KEY, no Claude leak", async () => {
let seenAuth: string | null = "unset";
const zaiSrv = Bun.serve({
port: 0,
fetch: (req) => {
seenAuth = req.headers.get("authorization");
return sse("zai");
},
});
// No `upstreams` in the config → relies on the BUILT-IN zai for auth/base;
// baseOverride only redirects the URL to the fake server so we don't hit real Z.ai.
const cfg: Config = {
models: {
orchestrator: { upstream: "anthropic", slug: "passthrough" },
flagship: { upstream: "zai", slug: "glm-5.2" },
},
default: "orchestrator",
longContextThreshold: 200000,
routes: [{ when: { anySubagent: true }, use: "flagship" }],
};
const p = buildServer({ config: cfg, env: { ZAI_API_KEY: "zk-test" }, logPath: LOG, port: 0, baseOverride: { zai: zaiSrv.url.origin } });
const res = await fetch(`${p.url.origin}/v1/messages`, {
method: "POST",
headers: { "content-type": "application/json", "x-claude-code-agent-id": "a", "authorization": "Bearer real-claude-oauth" },
body: JSON.stringify({ model: "claude-x" }),
});
expect(await res.text()).toContain("\"upstream\":\"zai\"");
expect(seenAuth).toBe("Bearer zk-test"); // Z.ai received YOUR key, not Claude's oauth token
expect(readDecisions(LOG).at(-1)!.resolvedModel).toBe("glm-5.2");
p.stop(true);
zaiSrv.stop(true);
});

test("a custom local upstream routes there and leaks no Claude auth", async () => {
let seenAuth: string | null = "unset";
const localSrv = Bun.serve({
Expand Down
20 changes: 20 additions & 0 deletions test/upstreams.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -108,3 +108,23 @@ test("a config-defined bearer upstream injects its own env key", () => {
expect(rewriteHeaders(toGw, new Headers(), { GW_KEY: "secret" }, upstreams).get("authorization")).toBe("Bearer secret");
expect(() => rewriteHeaders(toGw, new Headers(), {}, upstreams)).toThrow(MissingKeyError);
});

const toZai: Decision = { alias: "flagship", upstream: "zai", model: "glm-5.2", matchedRule: "tag:flagship" };

test("built-in zai upstream targets Z.ai's Anthropic endpoint", () => {
expect(resolveUpstream("zai").base).toBe("https://api.z.ai/api/anthropic");
expect(forwardUrl("zai", "/v1/messages", "")).toBe("https://api.z.ai/api/anthropic/v1/messages");
});

test("built-in zai upstream sends Bearer ZAI_API_KEY, drops Claude auth, strips betas", () => {
const inbound = new Headers({ "authorization": "Bearer claude-oauth", "x-api-key": "sk-ant", "anthropic-beta": "x", "content-type": "application/json" });
const out = rewriteHeaders(toZai, inbound, { ZAI_API_KEY: "zk-secret" });
expect(out.get("authorization")).toBe("Bearer zk-secret"); // your Z.ai key, not Claude's
expect(out.get("x-api-key")).toBeNull(); // Claude auth never leaked to Z.ai
expect(out.get("anthropic-beta")).toBeNull(); // stripped by default (safe)
expect(out.get("content-type")).toBe("application/json");
});

test("built-in zai upstream throws MissingKeyError without ZAI_API_KEY", () => {
expect(() => rewriteHeaders(toZai, new Headers(), {})).toThrow(MissingKeyError);
});