feat(gateway): add the LLM lane — governed, brokered, metered model egress

[Ar9av]

Why

The gateway has governed the tool lane since the MCP aggregator shipped: one connector in front of every MCP server, every tools/call policy-evaluated before it forwards, every result scanned before the model sees it.

Model traffic had no equivalent. An agent could be fully governed on the tool side and still paste a live credential straight into a prompt, and there was no attribution for what any of it cost.

This adds the model lane, and makes prismor gateway the umbrella command over both.

prismor gateway mcp    # tools  — the existing aggregator, unchanged
prismor gateway llm    # models — new
prismor mcp-gateway    # still works, alias for `gateway mcp`

Both lanes share one policy engine and one session id, so a session's tool calls and model calls land on the same trace.

What the model lane does

A local HTTP proxy an SDK points at instead of the provider. Per request:

1. Policy — evaluated as a network event carrying the outbound payload. Reusing the existing event type is the whole trick: the egress allowlist, secret-in-payload rules, and taint escalation apply to model calls with no new policy surface to configure. A block is refused before egress, so the provider never receives the body.
2. Credential brokering — the real key is resolved server-side, from a Cloak placeholder or the gateway's own environment. Clients can hold a dummy value or nothing at all.
3. Metering — tokens and cost come from the provider's own usage accounting, attributed per session/agent, flushed as llm_usage records through the shared uploader, inheriting its offline spool and at-least-once dedupe.
4. Response scan — the completion is re-evaluated as untrusted content before being returned.

Only the base URL changes on the client:

client = Anthropic(base_url="http://127.0.0.1:8787", api_key="unused")

Design notes worth reviewing

Streaming is a first-class path. Agent traffic is overwhelmingly SSE; a proxy that only handled buffered JSON would be useless. Bytes relay unbuffered (time-to-first-token unchanged) while a tap recovers the usage block and assistant text. Usage is only knowable at end-of-stream, so the response scan on a streamed call is post-hoc by construction — the pre-call check has already run and the scan still reaches telemetry. Flagged in the code and docs rather than papered over.

Fail-closed in enforce. A policy engine that errors denies the call rather than waving model traffic through — same stance as the MCP lane.

Denials wear the provider's error shape (HTTP 403, permission_error), so SDKs surface a readable refusal instead of crashing on an unexpected body.

Cost is attribution, not invoicing. Built-in USD/1M-token table matched by longest model-id prefix, cache reads discounted. An unknown model falls back to a conservative non-zero rate — a silent $0 reads as "this model is free". Override with --pricing.

Bind is loopback-only by default. The gateway is a credential-bearing egress point; --host is available but documented as needing an authenticating proxy in front.

Second commit: a real bug this surfaced

cloaked-secret-in-mcp-args fires on any event with an outbound_payload. That was only ever remote MCP calls, so the hardcoded title "outbound MCP tool arguments" was accurate. The LLM lane reaches the same check with a prompt — so an operator blocked while pasting a token into a chat message was being pointed at the wrong subsystem.

Fixed the title to name the surface it saw. Deliberately did not rename the ruleId — the enforcement floor and existing RuleExemptions key on it, so renaming would silently drop both protections and admin-granted exemptions.

Found by running the real gateway against an enrolled secret on a real box, not in unit tests.

Testing

48 unit + transport tests (tests/test_llm_gateway.py). Transport tests run the real handler against a fake upstream on loopback, so the streaming relay, header rewriting, and credential swap are exercised rather than mocked.

End to end on st3ve (Ubuntu, Python 3.12 — vs 3.14 locally), real CLI process, real sockets, isolated Cloak store via PRISMOR_SECRETS_DIR:

=== enforce: enrolled secret in a model prompt ===
  PASS  call refused (403)
  PASS  payload never reached the provider
  PASS  provider-shaped error
  PASS  names the secret, never the value
    -> Blocked by Prismor: [CRITICAL] Enrolled secret (placeholder for
       E2E_TEST_TOKEN) detected in an outbound model prompt
       (rule: cloaked-secret-in-mcp-args)

=== control: same prompt without the secret ===
  PASS  benign call still allowed (200)
  PASS  benign call reached the provider
  PASS  both calls metered (blocked + allowed)

Note the finding names the secret but never its value — asserted as an invariant in the unit tests too.

Also verified on st3ve: credential brokered server-side (the client's key is never forwarded), tokens metered from provider usage, cost attributed, blocked attempts still metered so they appear in spend rather than vanishing.

Regression: 231 passed / 2 skipped across test_policy_engine, test_enforcement_floor, test_egress, test_mcp_gateway, test_cloak_secret_guard.

The 4 failures in test_cli.py (3 x analyze, 1 x cloak env import) pre-exist on clean origin/main — verified by stashing and re-running.

Not in this PR

• Bedrock provider (PrivateLink deployment) — next
• Wiring llm_usage into the control plane's ingest + spend dashboards — separate prismor-web PR
• Joining hook and gateway events into one rendered trace — depends on the above