TokenFuse

The runtime kill-switch for AI agents: cap their spend, stop runaway loops before they bill you.

A proxy you drop in front of every LLM call - it also blocks poisoned MCP tools and keeps secrets out of the model.

The kill-switch isn't a dashboard button you press after the fact - it's an HTTP 402 the gateway returns mid-run, before the provider bills you.

---

TokenFuse is a drop-in proxy between your AI agents and their LLM providers. It watches every call, adds up the real cost as it happens, and, the instant an agent goes rogue (burns through its budget, spins in a loop, or pastes a secret into a prompt), it cuts the circuit in real time, before the damage lands. You point your agent at it with a one-line base-URL change; no SDK, no rewrite. It also ships a free scanner that catches a poisoned or "rug-pulled" MCP tool before your agent ever calls it, and a broker that keeps the secrets that tool needs out of the model's context entirely.

⚡ Try it in one command, no signup, no account:

docker run -p 4100:4100 -e TOKENFUSE_ALLOW_STUB=1 ghcr.io/taipanbox/tokenfuse:v1.14.0

TOKENFUSE_ALLOW_STUB=1 is what makes this offline: with no provider behind it, the gateway answers from a built-in stub and meters a fixed 1000/500 tokens, so every figure it shows you is invented. That flag is how you say you know. Point it at a real provider and the numbers are real: see 🚀 Get started. Nothing to install at all: the live preview runs the fleet dashboard on sample data in your browser.

_{The same service as its room on it-rat.com draws it, where the diagram sits next to a simulation you can scrub back and forth.}

---

Where this fits in the stack

TokenFuse is the spend plane and the hot-path spine of the TAIPANBOX agent-governance stack: every agent LLM call passes through it, where it meters cost, routes to the cheapest model that meets the task tier, asks Wardryx for a per-request policy decision, and enforces the budget Breaker. The other services read the events it emits.

flowchart TB
  Agent["AI agent (any framework)"] -->|"LLM call (base-URL swap)"| TF["TokenFuse proxy: spend + enforcement"]
  TF -->|"POST /v1/decide (PEP)"| WX["Wardryx: policy PDP"]
  WX -.->|"allow / deny / hold"| TF
  TF -->|"cheapest model, budget OK"| LLM[("LLM provider")]
  TF -->|"CallRecords"| CL["TokenFuse Cloud: control plane, incidents, replay, evidence, kill-switch"]
  VCX["Vouchryx: delegation proved, and endable"] -->|"short-lived token: act + cnf"| TF
  TF -.->|"polls /v1/revocations"| VCX
  VCX ==>|"delegation_issued / denied / revoked"| BUS
  TF ==>|"agent-event NDJSON"| BUS{{"agent-event bus + Agent Passport"}}
  WX ==> BUS
  Agent -->|"web fetch"| SCX["Scopyx: governed web egress"]
  SCX -->|"POST /v1/decide"| WX
  SCX ==>|"web_fetch / web_blocked"| BUS
  ENG["Engram: memory"] -->|"reflect via base_url"| TF
  ENG ==> BUS
  BUS ==> IDX["Idryx: identity graph, detectors, Agent-BOM"]
  IDX ==>|"identity_finding"| BUS
  BUS ==> QX["Qryx: crypto / PQC, passport + hash-chain scan"]
  QX ==>|"crypto events"| BUS
  BUS ==> VX["Verdryx: quality / drift"]
  VX ==>|"quality events"| BUS
  TF -->|"outcome-tagged traces"| VX
  MX["Mockryx: pre-prod safety rehearsal"] -->|"hostile scenarios"| TF
  MX ==>|"sim events"| BUS
  BILL[("cloud, SaaS and model bills")] --> CC["CostCrew: the bill, worked by a crew of agents"]
  CC ==>|"spend_spike / budget_threshold / crew moves"| BUS
  BUS ==> TRX["Trailryx: the record plane, sealed and packed"]
  BUS ==> HX["reads the log, mails you (heraldyx)"]
  HX -->|"one mail, a view and never an action"| OPS["your mailbox"]
  HX ==>|"alert_sent"| HJ[("heraldyx's own hash-chained journal, not this bus")]
  YOU(["you, in a browser over your own tunnel"]) --> GX[["Genaryx: the console over all of it"]]
  GX -->|"signed commands: the kill, an approval, a policy"| CL
  GX -->|"signed commands"| WX
  GX ==>|"console_command"| BUS
  GX -.->|"reads it"| IDX
  GX -.->|"reads it"| QX
  GX -.->|"reads it"| VX
  GX -.->|"reads it"| MX
  GX -.->|"reads it"| ENG
  GX -.->|"reads it"| SCX
  GX -.->|"reads it"| CC
  TFP["terraform-provider-taipan"] -->|"budgets + passports as code"| CL
  ASG[["agent-stack-go: shared Go contract"]] -.->|imported by| IDX
  ASG -.->|imported by| WX
  ASG -.->|imported by| MX
  ASG -.->|imported by| TFP
  ASG -.->|imported by| HX
  ASG -.->|imported by| QX
  SPEC[["agent-passport: the spec"]] -.->|governs| BUS

Loading

• Consumes: agent LLM calls (one base-URL swap); Wardryx decisions (TokenFuse is the enforcement point, the PEP).
• Produces: priced and enforced upstream calls, agent-event NDJSON, CallRecords to its Cloud control plane, and outcome-tagged Parquet traces.
• Talks to: Wardryx (per-request policy), its own Cloud control plane, and every downstream consumer (Idryx, Qryx, Verdryx) via the event bus. Configured by terraform-provider-taipan; rehearsed against by Mockryx.

The full stack is TokenFuse (spend), Wardryx (policy), Vouchryx (delegation), Engram (memory), Idryx (access), Qryx (crypto), Verdryx (quality), Mockryx (pre-prod), scopyx (governed web egress), CostCrew (the bill), Trailryx (the record) and heraldyx (the mail out), on the shared Agent Passport + agent-event contract (agent-stack-go / agent-passport), configured via terraform-provider-taipan and driven from Genaryx, the console over all of it.

Run the whole open stack locally with one command via stack-up; the stack's home on the web is it-rat.com.

Live infrastructure validation

Before any public launch, TokenFuse was run on real Linux infrastructure with a real Anthropic key: a 4-node raft cluster across two datacenters (no double-spend, no split-brain), real enforcement under a 34-agent concurrent burst, and a matched-protocol cost-accounting run on Hetzner, AWS, and GCP.

The breaker fired 12 of 12 deliberate budget overruns on every one of the three clouds, which is the portability claim: what changes between them is the price of the machine, not the behaviour of the control.

Then the whole stack ran as a five-node Kubernetes cluster on each of those clouds (six clusters, 25 to 27 July 2026) to price the governance itself. Three findings, all counter-intuitive: the clouds differ by 108x on the shared RWX volume and by under 15% on compute; the newest CPU generation is 16% cheaper per unit of work despite costing 37% more per hour, so "take a smaller instance" quietly raises unit cost; and metering is cheap in CPU and expensive in gigabytes, at 426 bytes of audit per governed decision, every decision, which is 614 MB a day at a thousand calls a minute.

Full write-up, all numbers, the cluster-scale costs, and the real bugs live testing found (and fixed): VALIDATION.md.

---

📑 Table of contents

• The problem TokenFuse solves
• Why TokenFuse is different ← the differentiators
• How it works
• What's inside
• 🚀 Get started ← install & first run
• Safe by default ← what is on without configuration, and what it costs on upgrade
• Scan your MCP servers & gate CI ← free scanner + GitHub Action
• Architecture
• Project status
• The bigger picture
• Who is this for?
• Glossary · FAQ · Docs

---

🔥 The problem TokenFuse solves

A chatbot makes one call to an LLM. An agent makes hundreds: it thinks, calls a tool, reads the result, thinks again, retries, and loops. That loop is what makes agents powerful, and it's also what makes them dangerous in three specific ways:

1. Cost runs away silently

Agents burn tokens dramatically faster than a single chatbot turn: one study of agentic coding tasks found they can consume up to 1,000× more tokens than a single code-chat query - driven mostly by growing input context, not output - and that runs on the same task can vary by up to 30× in total tokens depending on how the loop unfolds (Bai et al., 2026). The failure mode that hurts most is that nothing looks wrong: a looping agent still returns 200 OK, so your APM stays green while the meter spins. The bill is the first and only symptom, and by then the money is spent.

2. "Per-key" limits don't understand agents

The unit that matters for an agent is the run: one whole task, start to finish, spanning many calls and often several sub-agents. Traditional controls cap a user or an API key. Neither can say "this one task has a $2 ceiling," neither notices that call #34 is identical to call #31 (a loop), and neither can stop a task mid-flight.

3. Agents are a new, live attack surface

Autonomous agents read untrusted web pages, call external MCP tools, and hold credentials. 65% of organizations reported an AI-agent security incident in the last year, and 82% discovered a shadow agent they didn't know was running (Cloud Security Alliance / Token Security, "Autonomous but Not Controlled," Apr. 2026). Prompt injection, secret exfiltration, and tool "rug-pulls" (a tool that changes behavior silently after a human already approved it) are runtime problems, and they can't be fixed by a code review before deploy.

TokenFuse addresses all three, in the request path, in real time, by enforcing per-run budgets, detecting loops, and acting as a security boundary for what agents can spend and leak.

flowchart LR
    A["🤖 Your AI agent"] -->|"just change base_url"| T["TokenFuse"]
    T -->|"forwards if OK"| P["☁️ LLM provider<br/>(Anthropic, OpenAI…)"]
    T -.->|"blocks if runaway"| X["🛑 402 stop"]
    T -.-> D["📊 Dashboard · alerts · reports"]
    classDef brand fill:#F6B740,stroke:#0A0E13,color:#0A0E13,font-weight:bold;
    class T brand;

Loading

A drop-in proxy. No SDK required, no rewrite of your agent.

---

🎯 Why TokenFuse is different

Most of the tooling around AI agents watches, logs, or filters. TokenFuse sits in the request path and can actually stop something, in real time, at the layer where the money and the secrets move. Four things about that we haven't found built together anywhere else:

1. Enforcement, not observability

That's the dashboard's own tagline, and it's literal, not marketing. TokenFuse doesn't just chart what an agent spent after the fact: it estimates cost before the call, checks it against the run's budget and a loop detector, and returns HTTP 402 the instant a run would go over - cutting the circuit before the provider bills you, not after. shadow → warn → enforce lets you prove this against real traffic before you ever let it block anything. A dashboard tells you the fire happened; TokenFuse is the breaker.

2. Budgets that survive a crash

A budget only means something if two gateways racing each other can't both spend it, and if it doesn't vanish the moment a process dies mid-run. TokenFuse's per-run budgets are hierarchical - a sub-agent's spend rolls up and is checked against every ancestor, all-or-nothing, and a sub-agent naming a parent the gateway has not opened is refused rather than checked against nothing in enforce (a would-block in shadow and warn) (the in-process ledger; the raft ledger's older walk is listed in CLAUDE.md invariant 49) - and, in cluster mode, are replicated across nodes through a raft state machine that can persist durably to disk (redb). The affordability check is linearized across the whole gateway fleet, so there's no cross-node double-spend, and - with durable storage enabled - a budget outlives not just a node crash but a full process restart.

3. Drop-in, fail-open, and fast

Point TOKENFUSE_UPSTREAM at Anthropic, or at an OpenAI-compatible endpoint - the gateway serves both /v1/messages and /v1/chat/completions from the same binary, and TOKENFUSE_WIRE (anthropic or openai) names which one a given process forwards to, inferred from the upstream URL when it isn't set, see docs/02 - and TokenFuse prices and enforces against all of it, with a fallback price for models it doesn't recognize rather than silently letting spend go untracked. It's a one-line base-URL swap, shadow mode first for the budget so it's risk-free to drop in, fail-open so it's never a single point of failure, runs offline against a built-in fake provider (TOKENFUSE_ALLOW_STUB=1, which exists so that nobody mistakes invented numbers for measured ones), and it's Rust: the enforcement decision itself adds well under a microsecond in-process (~0.4 µs p99 - see BENCHMARKS.md).

4. Catch a poisoned MCP tool for free, and gate CI on it

tokenfuse mcp-scan is a standalone, free CLI: point it at a live MCP server over Streamable HTTP or SSE, and it checks every tool and parameter description for injection phrases, credentials, instructions to chain another tool, paths to local secret files, text addressed to the model and hidden characters, then pins a fingerprint of every tool you approve and flags a rug pull the moment a tool's description or schema silently changes on a later fetch - exactly the supply-chain gap MCP's re-fetch-on-connect model opens up. It ships as a GitHub Action, so a rug pull fails the PR, not a future incident review; docs/17 has a runnable, self-contained demo of the whole catch. At runtime, the companion MCP credential-broker goes further and keeps secrets out of the model entirely: the agent only ever holds a handle like {{secret:github_token}}, and the real value is injected at the last hop, never in the prompt, the trace, or the model's memory.

Also worth knowing, in more detail under What's inside: a semantic cache that serves repeated questions for $0, budget/step policies you can backtest against real traffic before turning them on, eBPF-based shadow-agent discovery with zero application changes, and a hosted Cloud fleet view for when one gateway isn't enough.

Self-funding. The token blowouts above - a single task's usage swinging by up to 30× depending on how the loop unfolds - are exactly what a per-run budget is built to catch on call one, not on the invoice three days later. Most teams don't need an ROI deck for this: the first runaway TokenFuse blocks tends to cover the bill.

---

⚙️ How it works

_{The money diagram: every call is priced and gated in-line, before it reaches the provider. Enforcement, not observability.}

Every request flows through TokenFuse. It estimates the cost before the call, reserves it against the run's budget, forwards it only if it's safe, then reconciles the real cost from the streamed response.

sequenceDiagram
    participant A as 🤖 Agent
    participant T as TokenFuse
    participant L as ☁️ LLM provider
    A->>T: request (tagged with run id)
    T->>T: estimate cost + check budget & loops
    alt ✅ within budget, no loop
        T->>L: forward the call
        L-->>T: streamed answer
        T->>T: settle the real cost
        T-->>A: answer (passes straight through)
    else 🛑 over budget / loop detected
        T-->>A: 402 "budget_exceeded" → agent stops cleanly
        T->>T: raise an incident, export the event
    end

Loading

Three properties make this safe in production:

1. Shadow → Warn → Enforce. Start in shadow mode (observe only); flip to enforce when you trust it.
2. Fail-open by default. If TokenFuse itself has trouble, your traffic keeps flowing, so it never becomes a single point of failure. (And for the reverse, never losing a budget, it can run as a raft-replicated HA cluster.)
3. Metadata-only. It measures cost and behavior; it does not store prompt contents by default.

Because cost is estimated before the call and settled after it, TokenFuse's numbers are a fast pre-flight approximation reconciled against real usage, not a hard real-time guarantee - see the FAQ for what that means in practice.

One case is neither settled nor released: a call the gateway has already handed to the provider when the client disconnects, before any answer has come back. The provider may or may not have run it, so its reservation stays outstanding on the run and on the unit's monthly cap rather than being written off as free; GET /v1/runs lists it as retained / retained_usd, the log carries a warning naming the run and the amount, and the run's next call can widen its budget to absorb it. Nothing releases it on a timer: a person closes it with evidence through the operator console (POST /v1/retained/close, CLAUDE.md invariant 89; both routes are served only when TOKENFUSE_CLOSURE_SECRET is set). With TOKENFUSE_EVENTS_PATH set, the retained call is one call_retained event on the bus (medium) and its closure one retained_closed (high), so a person is told the money is held and told when it is released (invariant 90).

Latency: the enforcement decision adds ~0.4 µs p99 in-process; on the wire the gateway adds ~0.8 ms p50 / ~2 ms p99 over a direct provider call, negligible next to an LLM response measured in hundreds of ms to seconds. Method + numbers: BENCHMARKS.md.

---

🧩 What's inside

_{One shared core, enabled as config-gated capability packs.}

Everything below is implemented on main and tested in CI (see PROGRESS.md for the per-component status and tests); see Project status for exactly which of it is in the tagged v1.14.0 release versus landed on main since.

_{tokenfuse top: a live htop-style view of every run's spend against its budget; press k to kill a runaway.}

Cost & control

• 💰 Per-run budgets: a hard cap for a whole task, with hierarchical roll-up so a sub-agent's spend counts against its parent.
• 🔁 Loop / runaway detection: identical-call, ping-pong, and context-growth detectors.
• 🛑 Breaker: hard-stop a run (kill-switch) from the API (POST /v1/runs/{id}/kill), the tokenfuse top TUI, or the Cloud dashboard. There is no chat integration in this repository: anything that can call that endpoint can drive it, and nothing here calls Slack.
• 🧩 Policies as code (WASM): custom rules in any language, sandboxed.
• 🕰️ Backtesting: replay a candidate budget/step policy over past (Parquet) traffic to see what it would have blocked and saved, before enforcing it.
• ⚡ Semantic cache: repeated questions served for $0.
• 🧾 FOCUS export: tokenfuse focus-export --traces <dir> --out focus.csv turns the Parquet trace into a FinOps FOCUS-format CSV, one row per call - blocked calls stay in as BilledCost=0 / x_blocked=true rows rather than being dropped, so the enforcement savings show up in the same FinOps tooling a bank already points at its cloud spend. Every row carries x_key_id, the client credential the call was made with, and a blocked row names its Breaker reason in x_block_reason; a call refused for identity (identity_mismatch) is filed under key:<key_id>, never under the agent id it claimed, so an impersonation's refusals never land on its victim. The Cloud files the same refusals the same way, in /v1/agents, /v1/spend and /v1/owners, and never names the victim on an incident for them. A call the identity check would refuse but TOKENFUSE_IDENTITY_STRICT=warn forwarded stays an ordinary spent row under the id it claimed, and says so: its x_identity_reason names the check that failed (also set on a refusal for identity), the Cloud counts it on the run and the fleet, and the dashboard marks the run identity · warn. The column list is published in contracts/tokenfuse-constants.json (focus_export.columns).
• 🛠️ Tool-run metric: counts the tool calls (tool_use blocks / tool_calls arrays) the model emits per LLM call, for both Anthropic and OpenAI shapes, streaming and non-streaming alike. Rides the trace as a new nullable tool_calls column (schema-evolution safe, like every prior addition), as x_tool_calls in FOCUS export, and rolls up into the Cloud dashboard's Runs table and summary tile. Observed only in this release: no budget, no enforcement, just a count (see docs/21).

Also hardens your agents

• 🔒 Agent firewall (taint): block risky actions after an agent touches untrusted data. A tool the policy names nowhere is one of those risky actions: after untrusted input the starter policy refuses it as capability unclassified rather than presuming it harmless. A tool call the model asks for is judged whether the answer is buffered or streamed: a streamed answer's text flows as it arrives, its tool calls are held to the end of the stream and judged, and a denied one is replaced by an error event the client library raises on (see docs/26 section 8).
• 🕵️ DLP: catches a recognisable secret pasted into a prompt and refuses the call before it leaves (TOKENFUSE_DLP, block by default). It reads contiguous text and matches patterns, so it catches carelessness, not intent: a secret with no distinctive prefix, or one split across the text, goes through. See Safe by default for the measured cases.
• 🔑 MCP credential-broker + a free tool-poisoning / rug-pull scanner, CI-gated - see Scan your MCP servers & gate CI.
• 📡 eBPF Radar: discover shadow agents on a host, zero config (Linux).

Ops & platform

• 🧬 HA raft cluster: replicated budgets, durable storage, runtime membership, token auth + TLS.
• ☁️ Hosted Cloud: Rust control plane + Next.js dashboard: fleet-wide spend, kill-switch, and central budgets across many gateways. Binds to loopback by default; a wider bind is an explicit opt-in (TOKENFUSE_CLOUD_HOST) meant to sit behind your own TLS or tunnel, never on a raw public IP. Each site gets its own ingest key, secret:org:ingest:site-name (the ingest role can push telemetry and read like a viewer, nothing more), and the Cloud names the site from that key on GET /v1/gateways - never from anything a gateway's own request could claim. A gateway with nothing to report still pushes an empty heartbeat every 30 s by default, so a quiet but healthy site is not indistinguishable from a dead one; set TOKENFUSE_CLOUD_HEARTBEAT_SECONDS=0 to turn it off. A run whose latest call was refused reads refused with the Breaker reason on the dashboard's Runs table, with its refused calls counted and drawn under the burn-rate line; each gateway names the basis it charged every admitted call at (known or fallback, the same word as x-fuse-price), and the dashboard marks a model charged at the fallback and counts those calls across the fleet, alongside the calls from gateways older than v1.8.0 that name no basis.
• 📋 Compliance evidence pack + audit trail: tokenfuse compliance (CLI, free) and the Cloud /v1/compliance / /v1/compliance/evidence endpoints project real decision, incident, and MCP-scan evidence onto EU AI Act, US Fed SR 11-7, and SOC 2 controls, each graded enforced / partial / documented rather than over-claimed (a green catalog is not a certification). Every control-plane mutation (kill, budget change, incident ack) lands in a hash-chained, ES256-signed audit trail (/v1/audit, /v1/audit/verify, /v1/audit/manifest).
• 🗄️ Zero-DB analytics: telemetry in open Parquet, queried with tokenfuse sql "..."; OTel export (TOKENFUSE_OTLP_ENDPOINT, unset by default); a separate opt-in NDJSON event stream sits alongside it (next bullet).
• 📨 Agent-event NDJSON export (opt-in): set TOKENFUSE_EVENTS_PATH=<file> and every breaker trip, DLP/taint block, DLP finding let through in shadow or mask (kinds and counts, never the text), MCP rug-pull, call retained with its outcome unknown, and closure of one is appended as one NDJSON line in the shared Agent Passport event envelope (taipanbox.dev/agent-event/v0.2). Unset by default: zero hot-path cost, no file handle opened. A path that cannot be opened at startup (a missing directory, a directory the process may not write to) costs one line at warn naming the path and the error and saying the export is off; the process keeps running, in both the gateway and the control plane. Writes are fail-open (a write error is logged, never a request failure), an event with no identity is skipped and counted, never fabricated, and a chain a delegation token PROVED carries the token that proved it (SPEC 5.2 delegation_proof) and names the agent the record is filed under when no x-fuse-agent-id header was sent. Every line carries the spec's §6.5 prev_hash integrity chain (one file = one chain, resumed across restarts); verify a stream with agent-conform -chain <file>. Tamper-evidence, not tamper-proof: a partial edit or truncation no longer passes silently.
• 🐍 Python SDK, sub-µs decision path, four public container images, and SDK packages for npm, PyPI and crates.io, each released by a manual publish workflow in this repository.

Agent Passport. TokenFuse's x-fuse-agent-id / x-fuse-run-id / x-fuse-parent-run-id headers, the new x-fuse-on-behalf-of delegation-chain header (a comma-separated, ordered, root-first list of agent:// / user:// URIs, captured to the trace verbatim and never truncated when forwarded; two caps, and they bound different things: a raw header over 4 KiB is ignored as if absent, warned and counted, and a chain of more than 32 entries, the spec's maximum depth, is refused with 400 and one identity_mismatch event whether or not a delegation issuer is configured, since the shared event envelope refuses a longer chain; the run-ancestry walk over x-fuse-parent-run-id has its own, separate cap of 64 ancestors across requests, refused rather than truncated, which bounds which runs a call rolls up into, not whom it acts for), parent_run_id (now its own persisted Parquet trace column, not just an in-memory budget-hierarchy key), and the agent-event envelope above all follow one shared spec - Agent Passport - used across the TAIPANBOX agent-governance stack (TokenFuse for spend, Wardryx for policy, Engram for memory, Idryx for access, Qryx for crypto, Verdryx for quality, Mockryx for pre-prod, heraldyx for the mail), so the same agent identifier and delegation chain read the same way in every product's traces and events. Idryx, specifically, ingests TokenFuse's agent-events as a behavioral source for identity-graph correlation.

---

🚀 Get started

TokenFuse is a proxy: start it, then point your agent at it instead of the provider. Three steps, ~2 minutes.

Step 1. Start TokenFuse

Published to GitHub Container Registry, so it runs anywhere with Docker, nothing to compile:

docker run -p 4100:4100 -e TOKENFUSE_ALLOW_STUB=1 ghcr.io/taipanbox/tokenfuse:v1.14.0

A working gateway on http://localhost:4100, answering from a built-in fake provider so you can try it offline.

Why that flag is not optional. Without a provider the gateway would answer every call itself and meter a fixed 1000 input / 500 output tokens as real spend, so both the model's answers and the money would be invented, and both would look plausible from either end. That happened on a live cluster whose manifest simply had not set TOKENFUSE_UPSTREAM: every call returned 200, each was billed $0.0035, and nothing warned. So the stub is opt-in, and a gateway with neither a provider nor this flag refuses to start and prints both ways forward.

Prefer to build from source? (needs Rust)

git clone https://github.com/TAIPANBOX/tokenfuse.git
cd tokenfuse
TOKENFUSE_ALLOW_STUB=1 cargo run -p tokenfuse-gateway   # gateway on http://localhost:4100

Or skip the build: grab a prebuilt binary from the Releases page and run tokenfuse --version to confirm which one you downloaded, with no provider configured and nothing else set up.

Step 2. Point it at your real LLM provider

Tell TokenFuse where the provider is with TOKENFUSE_UPSTREAM, then send your agent's traffic to localhost:4100. Your provider API key is passed straight through, so TokenFuse never needs it.

docker run -p 4100:4100 \
  -e TOKENFUSE_UPSTREAM=https://api.anthropic.com/v1/messages \
  ghcr.io/taipanbox/tokenfuse:v1.14.0

Then change one line in your app, the base URL:

export ANTHROPIC_BASE_URL=http://localhost:4100   # Anthropic SDK

The same one line in nine frameworks, each with the place its per-run headers go and a script that runs until the 402: examples/recipes/ (LangChain, OpenAI Agents SDK, PydanticAI, CrewAI, AutoGen, Semantic Kernel, LiteLLM, Haystack, Claude Code).

Your agent runs exactly as before, with one thing to know before you point production traffic at it: a call that carries no x-fuse-run-id is refused (400 metering_required), because a call this gateway cannot account for is one it does not make. Add the header (step 3), or set TOKENFUSE_REQUIRE_RUN_ID=0 to restore the unmetered pass-through while you wire the header up. See Safe by default for both.

The gateway serves two front doors from the same binary, through the same enforcement pipeline: the Anthropic Messages API (/v1/messages) and an OpenAI-compatible /v1/chat/completions (docs/02), so OpenAI-style SDKs (and Ollama / vLLM clients) can point at it too. One gateway process forwards to one upstream, and TOKENFUSE_WIRE says which shape it speaks; unset, it's inferred from TOKENFUSE_UPSTREAM's path, and a caller at the door this process doesn't serve is refused before anything is reserved. On a streamed /v1/chat/completions call, the gateway adds stream_options: {"include_usage": true} when the request didn't set it itself, so the caller's stream ends with one extra chunk after the model's own content: choices is empty on it and usage carries the token totals for the whole request. Setting stream_options.include_usage yourself, including to false, is left exactly as you wrote it, and this chunk is not added; a stream that then reports no usage is charged the pre-flight estimate, not measured and not free (invariant 43).

Step 3. Give a run a budget

Add two headers: a run id (a name for the whole task) and a budget. TokenFuse tallies the real cost live and returns HTTP 402 the moment the task would blow past its cap.

curl http://localhost:4100/v1/messages \
  -H "content-type: application/json" \
  -H "x-fuse-run-id: my-agent-task-1" \
  -H "x-fuse-budget-usd: 0.50" \
  -d '{"model":"claude-sonnet","max_tokens":100,"messages":[{"role":"user","content":"hi"}]}'

• No x-fuse-run-id? The call is refused with 400 metering_required, and TOKENFUSE_REQUIRE_RUN_ID=0 brings back the old unmetered pass-through.
• Live view: docker exec <container> tokenfuse top shows every run and its $/min.

Observe first, then enforce. The BUDGET starts in shadow mode: it records what it would block (an x-fuse-would-block response header on the call, and, with TOKENFUSE_EVENTS_PATH set and an agent id on the request, a breaker_shadow event on the bus) but changes nothing, so a cap you are still tuning cannot stop a run. Flip to enforce when you trust it:

docker run -p 4100:4100 -e TOKENFUSE_MODE=enforce \
  -e TOKENFUSE_UPSTREAM=https://api.anthropic.com/v1/messages \
  ghcr.io/taipanbox/tokenfuse:v1.14.0

TOKENFUSE_MODE = shadow (default) · warn · enforce.

Cap what an agent may declare. The budget in x-fuse-budget-usd is whatever the calling agent sends, and an agent that sends a larger one on its next call widens its own run. With no client keys configured, that header is the only per-run ceiling there is. Set TOKENFUSE_MAX_RUN_BUDGET_USD to a positive decimal number of dollars (at most six decimals, for example 25 or 2.50) and the gateway clamps every budget the caller declared, and the built-in USD 5 default used when it declared none, to that figure before the run is opened:

docker run -p 4100:4100 -e TOKENFUSE_MODE=enforce -e TOKENFUSE_MAX_RUN_BUDGET_USD=25 \
  -e TOKENFUSE_UPSTREAM=https://api.anthropic.com/v1/messages \
  ghcr.io/taipanbox/tokenfuse:v1.14.0

A clamped call's response carries x-fuse-budget-clamped: <ceiling> (absent when nothing was clamped) and the first clamp of each run is logged once. A caller may still ask for less than the ceiling, widening an open run still works up to the ceiling and never past it, and a budget an operator set in the Cloud is the operator's own word and is not clamped. Unset (the default) changes nothing. A value the gateway cannot read (0, -1, 1e9, abc) exits 2 at startup rather than starting with no ceiling. It is one figure per run, not a per-agent limit: tying a budget to an agent identity needs client keys.

Which price a call is charged at. Every metered answer, streamed or not, carries x-fuse-price: known when the price book (built-in rows plus any TOKENFUSE_PRICE_BOOK rows) has a row for the model id, fallback when it did not and the call was priced at the conservative fallback of USD 15 / 75 per million tokens. That is the safe direction for a cap: it is at or above the input and output base rate of every row in the book (the dearest there is o1, at 15 / 60), and about 150 times the list price of Claude Haiku 5.5 (0.10 / 0.50). The first fallback-priced call of each model id is also logged at warn. x-fuse-cost-usd is this call's settled cost and x-fuse-spent-usd the run's running total, so a reader recording per-call charges wants the first.

The book has a row for every Claude model id Anthropic lists, under the Claude API, Amazon Bedrock, Google Cloud and OpenRouter, each rate read from the vendor's page on a date named beside it in pricebook.rs, plus OpenAI's current models, DeepSeek's (at its peak-hours rate, the dearer of its two), Moonshot's Kimi K3 and Z.ai's GLM-5.3 and GLM-5.1, read on 2026-10-13. Bedrock and Google Cloud bill a regional endpoint 10 percent above a global one, and the model id does not always say which endpoint served the call: an id that may be regional (a bare Bedrock anthropic.* id, a us./eu./jp./apac. profile, a Google @-dated id) is priced with the premium, which over-charges a global call by 10 percent, and a Google Cloud dateless id is the Claude API's id and is priced at list, which under-charges a regional Google call by 10 percent. Rates move; a row is right on the date it was read.

Gemini on Vertex AI's OpenAI-compatible endpoint has a row for each current text model under the id that endpoint takes (google/gemini-2.5-flash, google/gemini-3.5-flash, ...), read from Google's Vertex AI pricing page on 2026-10-08, and the Gemini 3.1 Pro Preview and Gemini 2.5 Pro rows on 2026-10-13. The output rate is the one Google bills for the response and the reasoning together. Each row errs toward over-charging where the page leaves a choice: the Gemini 3 family at the non-global rate (the id does not say which endpoint served the call), and Gemini 3.6, 3.7 and 3.8 Flash at the standard rate rather than the introductory one Google pays back as a credit until the end of 2026. Rows price text input; an audio prompt on a model that lists a higher audio rate is under-charged, which TOKENFUSE_PRICE_BOOK corrects.

A long prompt is priced at its long rate. Some models are sold at two rates by prompt length: Claude Haiku 5.5 above 100,000 prompt tokens, OpenAI's tiered models above 272,000, Gemini 2.5 Pro and 3.1 Pro Preview above 200,000. The book holds both. A call whose prompt (input, cache reads and cache writes together) is above the threshold is priced at the long rate for every token, output included, as the vendors bill it; a shorter call pays the base rate. The published book lists the long rates under price_book.long_context, and GET /v1/price-book lists the ones in effect. An OpenAI cache write (GPT-5.6 and later bill it at 1.25 times input) is read from the usage and priced as a write, not as plain input.

For anything the book does not know, or a rate that differs for you (a regional endpoint, a negotiated discount, a local model that costs nothing), set TOKENFUSE_PRICE_BOOK to a JSON file of rows in the published book's own shape (contracts/tokenfuse-constants.json, price_book.models), read once at startup. A row replaces the built-in row of the same id, long-prompt rate included (the file's row is the whole price for that id at every prompt length), or adds a new one; the fallback cannot be changed:

{"models": [{"model": "claude-sonnet-5",
             "input_per_mtok_microusd": 2200000, "output_per_mtok_microusd": 11000000,
             "cache_read_per_mtok_microusd": 220000, "cache_write_per_mtok_microusd": 2750000,
             "cache_write_1h_per_mtok_microusd": 4400000}]}

Rates are whole micro-USD per million tokens, every field is required and no other key is accepted. A file that is missing, not JSON, misspells a key, carries a negative, fractional or absurd rate (over USD 1,000 per million tokens), repeats a model id, has no rows, or is over 1 MiB or 4,096 rows makes the gateway exit 2 naming the variable, rather than start on the built-in book while you believe your rates are live.

GET /v1/price-book (behind the admin keys below) says which book the running gateway prices with: the rows in all and in the built-in book, whether a TOKENFUSE_PRICE_BOOK file is in effect and which model ids it replaced and added, the fallback rates, and the model ids callers have sent that the book has no row for and that were therefore charged at the fallback since the process started (at most 100 listed, each cut at 256 bytes, with the total beside them):

curl -s localhost:4100/v1/price-book
# {"rows":..,"built_in_rows":..,"operator_file":true,"overridden":1,"added":1,
#  "overridden_models":["claude-sonnet-5"],"added_models":["my-local-model"],
#  "fallback":{"input_per_mtok_microusd":15000000,"output_per_mtok_microusd":75000000,...},
#  "fallback_models_seen":["claude-sonnet-6"],"fallback_models_seen_total":1,"detail":"..."}

---

Verify what you downloaded

Every release from v0.5.0 on is signed keyless with Sigstore and carries a build-provenance attestation and an SBOM. The image tag is pinned on purpose: there is no :latest, so what you run is what a release page describes. With cosign and gh installed:

tag=v1.14.0
cosign verify-blob --bundle SHA256SUMS.sigstore.json \
  --certificate-identity "https://github.com/TAIPANBOX/tokenfuse/.github/workflows/release.yml@refs/tags/${tag}" \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com SHA256SUMS
sha256sum -c SHA256SUMS
gh attestation verify tokenfuse-x86_64-unknown-linux-musl -R TAIPANBOX/tokenfuse

The image:

cosign verify ghcr.io/taipanbox/tokenfuse:v1.14.0 \
  --certificate-identity-regexp '^https://github.com/TAIPANBOX/tokenfuse/\.github/workflows/release\.yml@refs/tags/v' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com
gh attestation verify oci://ghcr.io/taipanbox/tokenfuse:v1.14.0 -R TAIPANBOX/tokenfuse

Releases before v0.5.0 carry checksums only. Verified on 2026-09-13 from a machine that built nothing: sha256sum -c OK on a downloaded binary, and gh attestation verify on the binary and on the image both returned a SLSA provenance v1 statement signed by this workflow at refs/tags/v0.5.0. The release assets are plain binaries, not archives: tokenfuse-<target> and tokenfuse-cloud-<target>.

---

🔒 Safe by default

A cloud range on 2026-08-04 ran this stack against a real provider and found that its guarantees were all opt-in: secret scanning was off unless you set a variable, a call with no run id reached the provider and was recorded nowhere, and the check for "is the policy plane on the data path" read environment variables rather than asking whether a decision had ever come back. Separately each is defensible. Together they mean a deployment can pass every check it has and be governed on paper. Three things changed, and the old behaviour is one explicit variable away in each case.

Setting	Default	The old behaviour
TOKENFUSE_DLP	block: prompts are scanned and a call carrying a recognised secret is refused with 403 dlp_blocked	TOKENFUSE_DLP=off
TOKENFUSE_REQUIRE_RUN_ID	on: a call with no x-fuse-run-id is refused with 400 metering_required rather than reaching the provider unmetered	TOKENFUSE_REQUIRE_RUN_ID=0
GET /v1/policy-plane	new read-only report: what the policy PDP has actually answered	n/a, there was no such fact

What this costs you on upgrade. Both are behaviour changes on a live path. If your prompts legitimately carry things that match a secret pattern, those calls now get a 403 where they used to succeed; TOKENFUSE_DLP=mask redacts instead of refusing, and off is unchanged from before. If any of your traffic reaches the gateway without a run id, it now gets a 400; add the header or set TOKENFUSE_REQUIRE_RUN_ID=0. Neither default changes what TOKENFUSE_MODE governs: the budget still starts in shadow.

What the secret scanner does, stated plainly. It reads contiguous text and matches patterns, so it catches carelessness: an agent that pasted a config file, a key, or a .env into a prompt. Measured against a real provider on 2026-08-04, a whole AKIAIOSFODNN7EXAMPLE was refused before it left. Two things went through, and both are inherent rather than bugs: the 40-character AWS secret key on its own, which has no distinctive prefix to match, and the same access key split across the text. A pattern scanner cannot stop somebody who intends to leak a secret, only somebody who did not mean to, and nothing here should be read as though it did. Keeping the secret out of the model's context entirely is a different control, and that one is the MCP credential-broker.

Whether the policy plane is really there. GET /v1/policy-plane reports what the Wardryx PDP has ANSWERED, not what is configured: counts of real allow / deny / hold verdicts, the fallbacks this gateway synthesized when the PDP could not be reached (which are deliberately not counted as verdicts, since fail-open turns an outage into an allow), and two facts a deployment check can act on.

curl -s localhost:4100/v1/policy-plane?window_ms=3600000

on_data_path is true when a real verdict came back inside the window; allow_and_deny_seen is true only when a real allow and a real deny both did. The second is deliberately hard: it stays false until a deployment drill sends one call the policy must refuse, which is the point, because a check that cannot fail reports zero forever. The endpoint carries no prompt content. It sits behind the admin keys described below: on a loopback bind it needs nothing, and on a wide bind it is refused with 403 admin_keys_required until TOKENFUSE_ADMIN_KEYS is set, so set that before you publish the port.

---

🔍 Scan your MCP servers & gate CI

MCP servers are a new, live attack surface: a client typically re-fetches tools/list on every connection and trusts whatever comes back, so a tool a human already approved can silently change behavior later, without anyone re-reviewing it. tokenfuse mcp-scan is a free, standalone CLI - no TokenFuse account, gateway, or Cloud connection required.

The scan is deterministic and offline: it reads the shape of every description (a credential, an order to chain another tool, a path to a local secret file, text addressed to the model, hidden characters), not its meaning. A description that tells the model in calm plain prose to make something up passes, and so does a chain of tools whose names are single ordinary words. Treat a clean scan as "none of these shapes", not as "safe".

Pin the tool set you trust, then diff every later scan against it:

# First run: pin the approved tool set (writes a fingerprint lockfile)
tokenfuse mcp-scan --url https://mcp.example.com/rpc \
  --lock .mcp-scan.lock.json --write-lock

# Every later run: diff against the pin, flag poisoning + rug-pulls,
# write a machine-readable report, and set an exit code CI can act on
tokenfuse mcp-scan --url https://mcp.example.com/rpc \
  --lock .mcp-scan.lock.json \
  --json-out scan.json \
  --fail-on high      # critical | high | medium | low | none

It runs over Streamable HTTP or SSE, scans tool descriptions for injection phrases and hidden Unicode, and (against a server you own, via --attempt-call) can also check for unauthenticated exposure, plaintext transport, wildcard CORS, and SSRF-capable tools.

Gate it in CI with the repo's own composite GitHub Action - no self-hosted scanning infra:

- uses: TAIPANBOX/tokenfuse@main   # pin to a tag/SHA in production
  with:
    url: https://mcp.example.com/rpc
    fail-on: high                  # critical|high|medium|low|none
    # lock-path: .mcp-scan.lock.json   # rug-pull baseline, if you keep one

It uploads the full ScanReport JSON as a build artifact on every run, pass or fail, so a finding is easy to inspect straight from the PR check. A full copy-pasteable workflow_dispatch template lives at .github/workflows/mcp-scan-example.yml.

See it catch a live rug pull in under a minute - no external server, safe to run anywhere including CI:

cargo run --example rugpull_demo -p tokenfuse-gateway

It pins a benign tool, mutates the tool's own description to look poisoned, rescans, and prints ⛔ RUG PULL: tool 'weather' description/schema changed at Critical severity. Full walkthrough: docs/17 · Rug-pull demo.

Scanning catches a poisoned tool before it's approved. The runtime MCP credential-broker (tokenfuse mcp-broker) is the complementary live control: point your agent's MCP client at it, and it swaps {{secret:name}} handles for real values only at the last hop, so even a tool that slips past review can't exfiltrate a credential it was never given.

Since 2026-08-06, a non-loopback bind with nothing configured on the broker's door refuses to start rather than only warn, because anything that reaches an open, unauthenticated port could have {{secret:name}} handles resolved against the whole vault. Upgrade consequence, stated plainly: a deployment that binds the broker to a non-loopback address with no credential configured will not start after this change. Fix it by configuring TOKENFUSE_MCP_KEYS="secret:key_id,...", by keeping the default loopback bind, or, if the open bind is deliberate, by setting TOKENFUSE_MCP_ALLOW_OPEN_BIND=1 (full detail, including what still only warns: docs/12).

There are two ways to put something on that door, and the second is newer and stronger. TOKENFUSE_MCP_KEYS is a shared secret in a header: whoever captures it holds it. TOKENFUSE_MCP_CLIENT_IDS instead names client metadata documents in the shape CIMD defines, each published by a client at its own https client_id URL and naming that client's public keys; a caller then proves possession of one of those keys per request with an RFC 9449 DPoP proof, single-use. Nothing an operator holds is worth stealing, and a captured header is worth nothing after a minute. Both doors can be configured at once while clients move across, and TOKENFUSE_MCP_REQUIRE_PROOF=1 is how that ends: docs/24.

A third, experimental since 2026-09-24: an access token that vouchryx issued through Cross App Access, where an enterprise identity provider's assertion about a person (an ID-JAG, draft-ietf-oauth-identity-assertion-authz-grant-04) is redeemed for a five-minute token scoped to this broker, presented as Authorization: Bearer. Set TOKENFUSE_MCP_ACCEPT_XAA=on and TOKENFUSE_MCP_RESOURCE (this broker's own resource URL, https, which the token's audience must equal exactly), with TOKENFUSE_DELEGATION_ISSUER and TOKENFUSE_DELEGATION_JWKS naming vouchryx and TOKENFUSE_DELEGATION_URL the broker's public origin (the delegation door in the same process requires it). The broker then publishes RFC 9728 metadata at /.well-known/oauth-protected-resource, every refusal points at it in WWW-Authenticate, a call is attributed to the agent the token names with the person as the root of its chain, and a token for another resource, a token bound to a key, a delegation token presented as bearer, or a token naming a revoked person or agent is refused. With XAA on, a caller presenting no credential is refused even when no other door is configured. The grant is an IETF draft, so both settings are experimental in compat/1.0.json; CLAUDE.md invariant 62 holds the rules.

---

🏗️ Architecture

_{One Rust workspace: a hot-path gateway, a dependency-minimal core, an optional HA cluster, and the hosted Cloud.}

One fast Rust binary in the request path, a Rust control plane for the Cloud, a Next.js dashboard. Telemetry lives in open Parquet files instead of a heavy database.

flowchart TB
    subgraph packs["🧩 Capability packs (enabled by config)"]
        direction LR
        F["💰 FinOps<br/>budgets, kill, forecast"]
        C["⚡ Cache<br/>semantic cache"]
        S["🔒 Security<br/>taint · MCP · DLP"]
        DA["🗄️ Data / RAG<br/>ingest scan · ledger"]
    end
    subgraph core["🦀 Shared core · Rust (single binary)"]
        direction LR
        I["Interception<br/>(the proxy)"]
        LE["Ledger + traces"]
        PE["Policy engine"]
        TD["Taint domain"]
    end
    packs --> core
    core --> ST["📦 Parquet + DataFusion · OTel export"]
    core -.-> HA["🧬 raft HA cluster"]
    core -.-> CL["☁️ Cloud control plane + dashboard"]

Loading

Design decisions and the data model: docs/02-architecture.md.

---

📋 Project status

v1.14.0: the current release, functional and shipped, young and not yet battle-tested.

The full request path (budget enforcement, SSE passthrough, loop detection, hierarchical budgets), the intelligence/ops layer (semantic cache, WASM policies, backtesting, Parquet + tokenfuse sql, OTel, tokenfuse top, Python SDK), the security packs (agent firewall/taint, DLP, MCP scanner + credential-broker, CI Action), eBPF Radar, the raft HA cluster (durable storage, membership, auth, TLS), and the hosted Cloud (control plane + dashboard, telemetry, fleet-wide kill-switch, central budgets) are all implemented, tested in CI, and published as container images.

The newest tag, v1.14.0, is a minor release: the gateway admits nothing a budget governs until it has read its control plane's money state, judges streamed tool calls as it judges buffered ones, puts a refused client key on the event bus, records the PII it lets through on each trace row and in the fleet view, and the Cloud names every model a run used. v1.13.2 before it is a security patch: rustls 0.23.46 for four RustSec advisories (RUSTSEC-2026-0346 to 0349), and the control-plane page now renders run fields as text, so a run id or model name a caller chose can never become markup on that page. v1.13.1 before it caps a tool call's arguments at the policy plane's own 12 KiB bound, so a larger call reaches Wardryx flagged as truncated and gets a verdict instead of being left to the failmode, and moves datafusion to 55, which drops the thrift crate. v1.13.0 before it holds the built-in price book as the vendors' rates read on 2026-10-13, and a model sold at two rates by prompt length (Claude Haiku 5.5, OpenAI's tiered models, Gemini 2.5 Pro and 3.1 Pro Preview) is priced by the prompt each call sent. Each release's notes also say what is not proven.

v0.4.0 ("live-validation fixes, fail-closed hardening", 2026-07-15) shipped everything built since v0.3.0: the web dashboard restyled around the "fuse" identity, the MCP scanner's live --url mode, JSON reports, --fail-on exit codes and composite GitHub Action, tokenfuse focus-export (Parquet traces → a FinOps FOCUS-format CSV, blocked calls included as $0 rows), the opt-in agent-event NDJSON exporter (TOKENFUSE_EVENTS_PATH) and the x-fuse-on-behalf-of delegation-chain header (the shared Agent Passport spec), plus the fail-closed fixes a real-infrastructure validation campaign shook out (VALIDATION.md).

Since v0.4.0, on main and in the releases after it (see docs/releases/): TokenFuse is now free end to end (the last plan-entitlement gating was removed from Cloud; there is no paid TokenFuse tier); the Cloud control plane binds to loopback by default, with TOKENFUSE_CLOUD_HOST as the explicit opt-in for a wider bind; the gateway refuses to start rather than answer from a stub and meter invented usage as spend (TOKENFUSE_ALLOW_STUB=1 keeps the offline loop); the MCP credential-broker now refuses to start too, on a non-loopback bind with no TOKENFUSE_MCP_KEYS configured (TOKENFUSE_MCP_ALLOW_OPEN_BIND=1 is the opt-out for a deliberately open bind); the gateway now serves an OpenAI-compatible /v1/chat/completions door beside /v1/messages, through the same enforcement pipeline, selected by TOKENFUSE_WIRE (docs/26); and the dashboard gained a no-install live preview with sample data. Landed after a live cloud range on 2026-08-04: safe defaults (secret scanning on, unmetered pass-through off), GET /v1/policy-plane, and incident severity that comes from the magnitude a detector measured instead of from its name.

It has not yet had a production hardening pass or a security audit; treat it as an early, capable release you can evaluate today, not a turnkey enterprise product. Run it in shadow mode first.

docker run -p 4100:4100 -e TOKENFUSE_ALLOW_STUB=1 ghcr.io/taipanbox/tokenfuse:v1.14.0   # gateway, offline
cd cloud && docker compose up                                                    # + Cloud dashboard (:3000)

Images on GHCR: tokenfuse · tokenfuse:<tag>-cluster · tokenfuse-control-plane · tokenfuse-dashboard, every one pinned by tag (no :latest).

---

🧭 The bigger picture: a runtime firewall

TokenFuse starts as a cost tool and grows into an agent runtime firewall: one brand, one install. The parts reinforce each other (that's the moat): a single taint domain follows data from the web → RAG → memory → tool calls, so the thing that catches a prompt injection is the same thing that enforces a budget.

flowchart LR
    M["💰 Stop burning money"] --> D["🗄️ Stop leaking data"] --> FW["🔒 Control everything agents do"]

Loading

Rationale ("one product, not three"): docs/09-product-strategy.md.

---

👥 Who is this for?

• AI / ML engineers shipping agents to production who've been surprised by a bill.
• Platform / DevOps teams who need guardrails and cost visibility across many agents.
• Security teams worried about what autonomous agents can do: prompt injection, data exfiltration, shadow agents, poisoned MCP tools.
• Solo builders who want a safety net that installs in one command.

---

📖 Glossary for newcomers

Term	Plain-English meaning
LLM	The AI model behind the scenes (Claude, GPT…). You pay per "token" it reads and writes.
Token	A chunk of text (~¾ of a word). Billing is per token.
Agent	An AI that works in a loop: think → act → observe → repeat. Powerful, but can spiral.
Run	One complete agent task, start to finish, possibly hundreds of LLM calls.
Runaway	An agent stuck looping or exploding in cost; the thing TokenFuse stops.
Breaker	The mechanism that trips and stops a run when it crosses a budget, loop, taint, DLP, or policy limit - exposed as a kill-switch across the API, the TUI, and the Cloud dashboard.
Proxy	A middleman in the request path. You point your agent at it instead of the provider.
MCP	A standard for agents to call external tools/servers; powerful, and a new security surface.
Rug pull	A previously approved MCP tool that silently changes its description or schema on a later fetch.
Prompt injection	A hidden instruction smuggled into data the agent reads, hijacking its behavior.
FOCUS	The FinOps Open Cost & Usage Specification - a standard CSV shape for cloud/service spend; tokenfuse focus-export puts LLM spend into it.
Agent Passport	A shared spec (identifier format + delegation chain + event envelope) that TokenFuse and the rest of the TAIPANBOX stack use so an agent's identity reads the same way everywhere.

---

❓ FAQ

Will it slow my agent down? Negligibly. ~0.8 ms p50 added on the wire, and responses stream straight through (no buffering). See BENCHMARKS.md.

Do I have to change my code? No. Change one base-URL env var so calls go through TokenFuse. An optional Python SDK adds nicer error handling.

Does it read or store my prompts? No. Metadata-only by default. It measures cost and behavior, not content.

Is the budget enforcement a hard real-time guarantee? No - be precise about this. TokenFuse estimates cost before forwarding a call and settles the real cost from the response afterward; it's a fast pre-flight approximation reconciled against actual usage, not a guarantee that not one extra cent can ever be spent. It's also fail-open by default: if TokenFuse itself has trouble, traffic keeps flowing rather than stalling your agents. That's a deliberate trade-off for availability - run the raft HA cluster if you need the opposite guarantee (never losing a budget).

Is it free? Yes, all of it. TokenFuse is open source (Apache-2.0) and free to self-host, with no seat limits and no time limit: the CLI, the local proxy, tokenfuse mcp-scan and its GitHub Action, and the Cloud control plane and dashboard (fleet spend, alerts, central budgets, the kill-switch). There is no paid TokenFuse tier. A separate commercial product provides the secured, managed enterprise control room over the whole stack (authenticated remote access over a tunnel, unified fleet control, hardware-signed actions); TokenFuse itself stays free and open.

Is it production-ready? It is 1.14.0: the surface is written and held (COMPATIBILITY.md), it is CI-tested and it was run on real boxes, clusters and providers before the 1.0.0 tag (docs/releases/v1.0.0.md), but it is not audited or battle-hardened. Start in shadow mode and evaluate.

---

⚙️ Wave-2 configuration (router, policy, incidents)

Wave 2 added three opt-in integrations, each off by default and each a true no-op until you set its env var. Full design notes: docs/19-wave2-governance.md.

Model router (route each call to the cheapest model that still clears the task's quality tier):

Env var	Values / default	Meaning
TOKENFUSE_ROUTER	off (default) · shadow · on	shadow reports the route it would take without rewriting the request; on rewrites the model.
TOKENFUSE_ROUTER_RULES	path to a JSON rules file (optional)	Task-class → required tier + candidate models. Unset, unreadable, or malformed falls open to the built-in defaults (logged).

Request header x-fuse-task-type names the task class (e.g. cheap, hard). Response header x-fuse-router reports the decision: <model>=kept, <from>-><to> when a route was applied, or would-<from>-><to> in shadow mode. Router savings are booked as their own dimension, separate from cache savings.

Wardryx policy hook (a PEP calling an external Wardryx PDP for allow / deny / hold decisions):

Env var	Values / default	Meaning
TOKENFUSE_WARDRYX_URL	URL (unset ⇒ hook off)	The PDP's /v1/decide base URL. Unset forces off regardless of mode.
TOKENFUSE_WARDRYX_MODE	off (default) · shadow · enforce	shadow reports would-allow/would-deny/would-hold and never blocks; enforce blocks.
TOKENFUSE_WARDRYX_FAILMODE	closed (default under enforce) · open (default otherwise)	Behaviour when the PDP is unreachable: fail-open allows, fail-closed denies. Unset under enforce it is closed, so an outage of the policy plane refuses what the policy would judge; an explicit open still works and is said once at startup.
TOKENFUSE_WARDRYX_KEY	bearer token (optional)	Sent to the PDP.
TOKENFUSE_WARDRYX_TIMEOUT_MS	default 50	Per-decision timeout on the model path. When set, it is also the MCP broker's timeout on a tools/call unless the next variable is set.
TOKENFUSE_MCP_WARDRYX_TIMEOUT_MS	default: TOKENFUSE_WARDRYX_TIMEOUT_MS if set, else 1000	Timeout for the MCP broker's decision on a tools/call only. A tool call is not latency-critical the way a model call is, and a PDP behind a proxy (such as the typed-signal proxy, which waits up to 150 ms) needs more than 50 ms; a deny that arrives after the timeout is lost to the failmode, which is closed under enforce unless set to open. A value that is not a positive integer is ignored with one warning.
TOKENFUSE_WARDRYX_CACHE_TTL_MS	default 3000 (0 disables)	Short-TTL decision cache; only decisions the PDP marks cacheable are ever cached (a hold never is).

The attestation_method the PDP is told comes from the door that authenticated the call, in agent-passport's words: dpop-key when the request proved possession of a key (a DPoP-bound delegation, or the MCP broker's proof door), none otherwise, a client key and an XAA bearer token included. A caller's x-fuse-attestation-method header is ignored, and the gateway says so once per process, so a deny_if_unattested policy cannot be satisfied by a string the caller wrote.

Shadow tool-pruning measurement (how many input tokens go to schemas of tools the wardryx policy would deny, measured, never enforced):

Env var	Values / default	Meaning
TOKENFUSE_TOOLS_PRUNE	off (default) · shadow	shadow calls wardryx's /v1/filter-tools (with the wardryx hook on, at least one tool declared and the agent named in x-fuse-agent-id), records how many of the declared tools its policy would deny and their estimated input-token cost, and sets x-fuse-tools-would-prune. Never changes the forwarded request in any mode. An unrecognised value is treated as off, logged once.

The MCP broker's decision request also carries the call it is about to make, tool_call: {name, arguments, target}: arguments is params.arguments exactly as the agent sent it, taken before any {{secret:NAME}} handle is replaced (the PDP sees the handle text, never the secret), target is the upstream server the call routes to (the x-fuse-mcp-upstream name, else the default upstream's host and port). Arguments over 12 KiB serialized (the policy plane's own bound, which it refuses above that size) are not sent at all: the object arrives without arguments and with arguments_truncated: true, never a cut-off copy. The model path sends no tool_call, because a model call only offers tools.

A deny returns 403 with x-fuse-wardryx: deny. A hold returns 403 with x-fuse-wardryx: hold plus x-fuse-approval-id; obtain an approval out of band, then resubmit the identical request carrying x-fuse-approval-token.

Cloud incident thresholds (tokenfuse-cloud; the gateway itself needs none of these):

Env var	Default	Trips when
TOKENFUSE_CLOUD_REPLAY_EVENTS	unset	Path to an agent-event NDJSON file the control plane reads (never writes) to reconstruct a run for /v1/replay/{run}. Unset ⇒ replay reports configured:false.
TOKENFUSE_CLOUD_INCIDENT_BUDGET_BLOCKS	3	A run hits ≥ this many budget-protection blocks (budget_exhausted).
TOKENFUSE_CLOUD_INCIDENT_LOOP_REPEATS	3	≥ this many loop_detected decisions for one run (sustained_loop).
TOKENFUSE_CLOUD_INCIDENT_SPEND_PER_MIN_USD	5.0	The floor for spend_spike: an org's last-minute burn below this rate is never a spike, whatever its own baseline.
TOKENFUSE_CLOUD_INCIDENT_SPIKE_MULTIPLE	4 (floor 2)	How many times its own recent baseline an org's last minute has to be before spend_spike calls it a spike.
TOKENFUSE_CLOUD_INCIDENT_FANOUT_RUNS	20	The floor for fanout_explosion: one agent_id driving fewer distinct runs than this in the window is never an explosion.
TOKENFUSE_CLOUD_INCIDENT_FANOUT_MULTIPLE	4 (floor 2)	How many times its own habit an agent's window has to be before fanout_explosion calls it an explosion.
TOKENFUSE_CLOUD_STALL_MINUTES	5 (0 off)	A run this control plane has seen calling at least twice has made no call for this many minutes, and for longer than its own longest gap between calls (run_stalled, raised once per run by a sweep every 10 s; a kill, a refused last call or an outcome tag on the last call count as the run ending).

The gateway's side of the Cloud (read by the gateway, not by the control plane above; it connects only when both are set and not empty):

Env var	Default	Meaning
TOKENFUSE_CLOUD_URL	unset ⇒ no Cloud sink	The base URL of the control plane.
TOKENFUSE_CLOUD_KEY	unset ⇒ no Cloud sink	The site's ingest key, secret:org:ingest:site-name (see the Hosted Cloud bullet under What's inside).

Shipped in the same change as replay: the Cloud regulator evidence pack (/v1/compliance/evidence: EU AI Act / SR 11-7 / SOC 2 sections, each control graded from this org's live decision + incident data, not the replay file itself). Full picture, including the hash-chained audit trail and the free CLI: What's inside.

Trace + unit-economics (already present, documented here for completeness): set TOKENFUSE_DATA_DIR to write Parquet trace segments (read back by focus-export / outcomes / sql). A segment reaches disk at 256 buffered calls, on a two-second flush tick, or on shutdown (SIGINT or SIGTERM), whichever comes first, so it is safe to look in the directory a few seconds after any call. Request header x-fuse-outcome tags a call's result with an opaque string, captured verbatim and never validated against a fixed vocabulary; the illustrative values TokenFuse and Verdryx both use are case_resolved, escalated, abandoned.

Other gateway settings:

Env var	Default	Meaning
TOKENFUSE_OTLP_ENDPOINT	unset or empty ⇒ no OTel export	The OTLP endpoint the gateway sends its telemetry to, alongside the Parquet traces.
TOKENFUSE_MAX_BODY_BYTES	16777216 (16 MiB)	The most request-body bytes the gateway accepts; a larger body is refused. A value that is not a number is ignored.
TOKENFUSE_AGENT_ID_MODE	off	What to do with an agent_id the agent-event envelope would reject: off records and counts it, warn also logs a warning naming it, enforce refuses the request with 400 before it reaches a provider. Any other value exits at startup.
TOKENFUSE_WASM_POLICY	unset ⇒ no WASM policy	Path to a WASM policy file. Only a binary built with --features wasm reads it, and the default release build is not one.
TOKENFUSE_CLOSURE_SECRET	unset ⇒ /v1/retained and /v1/retained/close are not served	The secret the operator console shares with the gateway, which a retained-reservation closure is checked against. At least 32 bytes; a shorter value stops startup.

Client credentials (opt-in, off by default):

Env var	Default	Meaning
TOKENFUSE_CLIENT_KEYS	unset ⇒ off	secret:key_id,…. Set it and every call to /v1/messages or /v1/chat/completions must present a known secret in the x-fuse-key header or get 401; each call's trace then records the resolved key_id.

Until now the gateway authenticated nobody, and every identity on a call was a header the caller wrote (x-fuse-run-id, x-fuse-agent-id). That is honest for attribution a cooperating fleet reports about itself, and it is why agent_id is documented as attribution-only. It is not enough to key a budget on: anything a caller can choose, a caller can change, so a per-agent cap keyed on x-fuse-agent-id is bypassed by sending a different one, and someone else's agent id can be burned on purpose. key_id is the first identity on the trace the caller cannot choose.

Notes, because the details matter more than the flag:

• Unset changes nothing. No credential is required and key_id is empty. A drop-in proxy has to stay drop-in on upgrade.
• Set-but-unusable refuses to start. A typo, a stray quote or an empty interpolated variable would otherwise be read as "off", leaving the gateway open at exactly the moment you believed you had closed it. It exits with a message instead.
• x-fuse-key, not Authorization. Authorization on an inbound call is your provider's credential and is deliberately forwarded upstream; x-fuse-* headers never are.
• A missing and an unknown credential are refused identically, and the presented secret is never echoed into the error body. The operator's record does tell them apart: with TOKENFUSE_EVENTS_PATH set, each refusal is an identity_mismatch event whose data.reason is missing_key or unknown_key, with data.key_fingerprint (sha256: and 16 hex digits of the presented value, never the value) for an unknown one, filed under the x-fuse-agent-id the call claimed (or the reserved agent://unidentified.invalid/caller when it names none, which is what a scan looks like); one event per key and claimed agent a minute and at most sixty a minute per gateway, so a key scan is seen without flooding the bus (invariant 105).
• An open bind with no keys says so. Unset on a bind that is not loopback (the shipped image binds 0.0.0.0:4100), anyone who can reach the port spends through the gateway. The gateway prints a warning at startup naming TOKENFUSE_CLIENT_KEYS, and GET /healthz?detail=1 answers client_auth, bind and the same warning as JSON. Plain GET /healthz still answers ok.
• The tool-call check needs a key too. With keys set, POST /v1/fuse/check-tool-call refuses a missing or unknown key with the same 401, and answers a caller only about runs its own key opened: a question about a run another key opened is answered deny with nothing of that run in it. Without keys it answers anyone, as every door of a gateway with no keys does.
• Scope, stated plainly: both model doors, /v1/messages and /v1/chat/completions, and the tool-call check share this check. This adds identity; the budget enforced against it is the identity map, next.

Admin keys: /v1/runs, /v1/runs/{id}/kill, /v1/keys, /v1/policy-plane, /v1/agent-ids, /v1/price-book (loopback-safe by default; a wide bind needs a key):

Env var	Default	Meaning
TOKENFUSE_ADMIN_KEYS	unset ⇒ off on loopback	Comma-separated bearer keys, same trimming rules as TOKENFUSE_CLIENT_KEYS. Set it and every one of the six routes above needs Authorization: Bearer <key> matching one, or 401.
TOKENFUSE_ALLOW_OPEN_OBS	unset ⇒ off	Set to 1 to keep the six routes open on a non-loopback bind with no TOKENFUSE_ADMIN_KEYS configured (see below). Logged as a warning either way.

These routes list every run's budget and spend, list key ids, enumerate agent identities, name the model ids callers sent, and kill any run, and until now they carried no authentication at all: the comment beside them said the gateway binds loopback by default, which is true and was not the whole picture, because TOKENFUSE_ADDR=0.0.0.0:... (the shipped Docker image's default) makes them reachable from the network. The posture now mirrors the MCP broker's door: nothing configured on a loopback bind changes nothing; nothing configured on a wide bind refuses every request to these six with 403 admin_keys_required until TOKENFUSE_ADMIN_KEYS is set or TOKENFUSE_ALLOW_OPEN_OBS=1 opts back into the old behaviour; a configured key is required regardless of the bind. /healthz and /v1/messages are never behind this gate.

Identity map: key ↔ agent ↔ business unit, plus monthly unit budgets (opt-in, off by default; design notes in docs/20):

Env var	Default	Meaning
TOKENFUSE_IDENTITY_MAP	unset ⇒ off	Path to a JSON map with three sections: units (each optionally carrying budget_usd_month), keys (which key_id belongs to which unit, and which agent:// ids it may present), prefixes (attribution fallback for unkeyed traffic; a caller that DID present a known key with no keys entry also lands here, and under strict that is refused rather than letting it pick its own unit by header). Set-but-unusable refuses to start, same posture as TOKENFUSE_CLIENT_KEYS.
TOKENFUSE_IDENTITY_STRICT	enforce	off | warn | enforce, governing the key↔agent binding check AND whether a header may contradict a chain a delegation token proved: warn lets the call through with x-fuse-identity: would-block=<reason> and the same reason on its trace row (identity_reason), its FOCUS row (x_identity_reason) and in the Cloud; enforce returns 403 with "type": "identity_mismatch". Unit budgets follow TOKENFUSE_MODE like every other budget.

{
  "units":    [{ "id": "treasury", "budget_usd_month": 2000.0 }],
  "keys":     [{ "key_id": "treasury-bots", "unit": "treasury",
                 "agents": ["agent://bank.example/treasury/*"] }],
  "prefixes": [{ "match": "agent://bank.example/treasury/*", "unit": "treasury" }]
}

This closes the loop the client-credential slice opened: a credential (key_id) listed in keys is bound to the agent ids it may present, and under TOKENFUSE_IDENTITY_STRICT one that is not listed there cannot make up the difference by choosing an agent id (it used to reach the prefix fallback, where an id matching nothing skipped the monthly cap and an id matching another unit's prefix charged that unit). enforce is the default since 2026-08-27, and off restores the old behaviour in one variable. A deployment that configured neither client keys nor a delegation issuer is unaffected either way: a mismatch needs something to mismatch with, and both sources are opt-in. Agents roll up into a business unit, and the unit gets the first budget above the run - a UTC-calendar-month cap enforced with the same reserve-then-settle discipline as run budgets (402, "type": "unit_budget_exceeded", with the unit's numbers). Every trace row now carries a server-resolved unit column (nullable-evolution, old files keep reading), focus-export grows an x_unit column so per-unit chargeback is a spreadsheet filter, and the Cloud aggregates per-unit spend (GET /v1/units, all-time plus a month-to-date rollup over the same UTC-month window the caps enforce - the figure the dashboard's Business units card compares against the monthly caps) with central per-unit cap overrides (POST /v1/units/{id}/budget, polled by every gateway of the org). Unmapped spend stays visible as the unassigned bucket, never silently dropped.

Limits, stated plainly: unit counters are in-process and per-gateway - they reset on restart, and a gateway with a control plane configured seeds the current month's tally from GET /v1/units before it listens (one log line says what was seeded, or that it could not be), then counts its own spend on top; a control plane that is not up yet does not open a gap: until the gateway has read run spend, the unit month, the Cloud budgets and kills, a call carrying x-fuse-run-id is answered 503 seed_pending (retryable, retry-after: 1) in enforce and marked in x-fuse-would-block in shadow, while the gateway keeps asking in the background (invariant 103); they are not fleet-consistent across gateways (the replicated raft ledger deliberately does not grow this dimension in this change; the durable cross-fleet view of unit spend is the Cloud aggregation). Budgets remain estimate-then-settle. With client keys off, strict has nothing authenticated to check: binding checks stay idle and only prefix attribution applies.

---

🔗 Constants other repositories read

Anything downstream that has to agree with this gateway on a literal value reads contracts/tokenfuse-constants.json rather than retyping it. One versioned file carries the Breaker block-decision wire strings with their HTTP statuses, the flat blocked-decision set (whether a trace row's cost_microusd is avoided spend or real spend), the agent-event types with the severity each one always carries, both Parquet trace schemas (write and read: the difference between them is the backward-compatibility contract), and the default price book in microdollars per Mtok.

It is generated from the Rust, never hand-maintained: tokenfuse constants prints it, ./scripts/constants.sh --write regenerates the committed copy, and CI fails when the file and the source disagree. A hand-written constants file is the same defect one level up, a file that can drift from the constants it names, which is exactly what happened at the far end of a retyped copy: a downstream mirror carried seven block reasons while this repository had nine, for eleven days, so avoided estimates were counted as real spend.

Consume it by pinning a tag and reading the path, from a checkout or over raw HTTP. schema_version is the compatibility signal; the path deliberately carries no version, because a versioned filename is how a consumer keeps reading the old file forever without noticing there was a new one.

What is not here: the stack's fixed local port map. This repository owns only its own defaults (TOKENFUSE_ADDR, TOKENFUSE_MCP_ADDR); the assignments that make services agree with each other are decided by the local orchestrators (taipan, stack-up), which is also where collisions get resolved.

---

📚 Documentation

Document	What's inside
contracts/tokenfuse-constants.json	The generated, versioned constants above: wire strings, event severities, trace columns, prices. Read this instead of retyping them
PROGRESS.md	Live component-by-component build status & tests
BENCHMARKS.md	Latency methodology + numbers
01 · Research	The pain points and hard numbers behind the idea
02 · Architecture	Rust core, ADRs, data model, policy language
03 · Roadmap	Phases, demo script, metrics, risks
06 · Semantic cache · 07 · Taint model	Detailed subsystem designs
08 · Security extensions	MCP broker, RAG scanning, agent identity
10 · HA cluster · 11 · Hosted Cloud · 12 · MCP credential-broker	The distributed + cloud + security layers
13 · Security model & hardening	Trust boundaries, implemented controls, cargo audit gate
16 · Design system	The "fuse" visual identity: tokens, components, and how the dashboard is put together
17 · Rug-pull demo	cargo run --example rugpull_demo - a self-contained, lab-only demo of the live rug-pull scanner catching a tool that mutates post-approval
19 · Wave-2 governance	Model router, the Wardryx policy hook (PEP/PDP split), Cloud replay + the regulator evidence pack, and per-instance Parquet trace segments: the design notes behind the "Wave-2 configuration" section above
20 · Identity map	Key ↔ agent ↔ business-unit binding, strict mode, and monthly unit budgets: the design + build plan behind the "Identity map" section above
21 · Tool runs	What counts as a tool run (Anthropic/OpenAI, streaming/non-streaming), the nullable tool_calls trace column, and why v1 is observed-only with no budgets

---

📜 License

Apache License 2.0.

_{Built in the open. Diagrams render natively on GitHub (Mermaid).}