feat: routing table, per-key cost cap, and single-hop provider fallback (#4)

.env.example

@@ -17,5 +17,11 @@ REDIS_URL=redis://localhost:6379/0
 # Providers
 OPENAI_API_KEY=sk-your-openai-key
 
+# Routes (issue #4): JSON array of {model, primary, fallback,
+# input_price_per_1k, output_price_per_1k, max_output_tokens}.
+# Empty/unset ⇒ prefix-based fallback (gpt-* → openai, claude-* → anthropic, ...).
+# Example:
+# ROUTES_JSON=[{"model":"gpt-4o-mini","primary":"openai","input_price_per_1k":0.00015,"output_price_per_1k":0.0006,"max_output_tokens":16384},{"model":"gpt-4o","primary":"openai","fallback":"anthropic","input_price_per_1k":0.0025,"output_price_per_1k":0.01,"max_output_tokens":4096}]
+
 # Observability
 LOG_LEVEL=DEBUG

.gitignore

@@ -3,6 +3,9 @@ __pycache__/
 *.py[cod]
 *$py.class
 
+# Vendored flare source (managed by pixi, see pixi.toml)
+flare/
+
 # C extensions
 *.so
 
@@ -169,3 +172,4 @@ dist-mojo/
 # Personal research notes (not shipped with the project)
 .tmp/
 
+uv.lock

adr/003-mojo-default.md

@@ -161,7 +161,7 @@ Each follow-up is a one-page issue. None blocks this ADR.
 
 3. ~~**Rate limiting.**~~ **Shipped (per-process):** `RateLimit[Inner]` token bucket tuned by `RATE_LIMIT_RPS` / `RATE_LIMIT_BURST` (0 = disabled). Approximately global across workers via a shared atomic cell. Distributed rate limiting still needs Redis and stays open.
 
-4. **TLS termination.** **Shipped via [PR #36](https://github.com/echohello-dev/opengateway/pull/36) on the flare#7 commit (`edf3f22`).** Bumped `pixi.toml` from `v0.9.0` to `edf3f22`; rebuilt `libflare_tls.so` against the new flare source (one-line `-Wl,-u,_flare_ssl_read_ex` linker flag because clang treats unreferenced extern C as dead-code; gcc includes them by default). `main.mojo` binds via `HttpServer.bind_tls(addr, cert, key, alpn=["http/1.1"])` + `serve_tls(stack)` when `TLS_CERT_FILE`/`TLS_KEY_FILE` are set. e2e: `openssl s_client` handshake completes, ALPN negotiates `http/1.1`, non-streaming + streaming `/v1/chat/completions` round-trip works over HTTPS, structured logger emits JSON per line. The flare reactor-side TLS reactor is real and working. Self-signed cert hostname gotcha documented in README ("Self-signed cert gotcha") — curl needs `--resolve localhost:port:127.0.0.1` when the cert is `CN=localhost`. Production terminates TLS at the edge LB; the in-binary path is for the edge-binary deployment story (Fly.io machines, bare metal, Lambda-ish). The stdlib-ssl proxy (`opengateway/mojo_bridge/tls_proxy.py` + `tests/test_tls_proxy.py` + `opengateway/mojo/repro_tls_thread.mojo`) stays in-tree as a vendored fallback for the v0.9.0 pin and as a reproduction of the early daemon-thread diagnostic; the wrapper usage in `main.mojo` was reverted. The flare issue I filed earlier (#8) was closed because the only "stall" I observed was curl's cert hostname verification failing silently under `-s`, not a reactor bug.
+4. **TLS termination.** **Shipped via [PR #36](https://github.com/echohello-dev/opengateway/pull/36) on the flare#7 commit (`edf3f22`).** Bumped `pixi.toml` from `v0.9.0` to `edf3f22`; rebuilt `libflare_tls.so` against the new flare source (one-line `-Wl,-u,_flare_ssl_read_ex` linker flag because clang treats unreferenced extern C as dead-code; gcc includes them by default). `main.mojo` binds via `HttpServer.bind_tls(addr, cert, key, alpn=["http/1.1"])` + `serve_tls(stack)` when `TLS_CERT_FILE`/`TLS_KEY_FILE` are set. e2e: `openssl s_client` handshake completes, ALPN negotiates `http/1.1`, non-streaming + streaming `/v1/chat/completions` round-trip works over HTTPS, structured logger emits JSON per line. The flare reactor-side TLS reactor is real and working. Self-signed cert hostname gotcha documented in README ("Self-signed cert gotcha") — curl needs `--resolve localhost:port:127.0.0.1` when the cert is `CN=localhost`. Production terminates TLS at the edge LB; the in-binary path is for the edge-binary deployment story (Fly.io machines, bare metal, Lambda-ish). The flare issue I filed earlier (#8) was closed because the only "stall" I observed was curl's cert hostname verification failing silently under `-s`, not a reactor bug.
 
 5. ~~**DB-backed virtual keys.**~~ **Shipped via the Python bridge** (no Mojo Postgres driver required): `opengateway/mojo_bridge/db.py` defines a `VirtualKeyStore` protocol with an asyncpg-backed implementation, and `authenticate_authorization` consults it after the root-key short-circuit with a 60 s in-process TTL cache. One connection per lookup — the bridge's one-shot `asyncio.run` model means a pool cannot outlive its event loop; at gateway latencies the connect cost is noise. `database_url` is now unset-by-default; when unset the store seam returns `None` and auth is root-key-only. Schema is created by `PostgresVirtualKeyStore.ensure_schema` (`virtual_keys` table with models / budget / tpm / rpm columns). **Spend recording shipped with it:** the bridge increments `budget_used` by `usage.total_tokens` after each unary completion, and the streaming pump parses the terminal usage chunk (the bridge injects `stream_options: {"include_usage": true}` so upstreams always send one). Budgets are token-denominated; a per-model dollar pricing table is a deliberate product decision left open.
 

opengateway/config.py

@@ -1,11 +1,39 @@
 from __future__ import annotations
 
+import json
 from functools import lru_cache
 
-from pydantic import Field
+from pydantic import AliasChoices, Field, field_validator
 from pydantic_settings import BaseSettings, SettingsConfigDict
 
 
+class RouteRule(BaseSettings):
+    """One per-model routing rule.
+
+    Each rule pins a primary provider and (optionally) a fallback
+    provider that the bridge retries on retryable failures. Pricing
+    fields drive the per-virtual-key dollar cost cap (``max_cost_usd``).
+
+    Sourced from the ``ROUTES_JSON`` environment variable as a JSON
+    array. Empty list ⇒ prefix-based fallback (``gpt-*`` → OpenAI,
+    ``claude-*`` → Anthropic, etc.) is used, preserving pre-#4
+    behaviour.
+    """
+
+    model_config = SettingsConfigDict(
+        env_file=".env",
+        env_file_encoding="utf-8",
+        extra="ignore",
+    )
+
+    model: str
+    primary: str
+    fallback: str | None = None
+    input_price_per_1k: float = 0.0
+    output_price_per_1k: float = 0.0
+    max_output_tokens: int | None = None
+
+
 class Settings(BaseSettings):
     """Gateway configuration. All values can be overridden via environment variables."""
 
@@ -45,10 +73,36 @@ class Settings(BaseSettings):
     max_retries: int = Field(default=2)
     retry_backoff: float = Field(default=1.0)
 
+    # Routes (empty list = prefix-based fallback; see RouteRule docstring).
+    # Aliases ``ROUTES_JSON`` for the env var name, falls back to the
+    # field name ``routes`` so tests can also construct ``Settings``
+    # directly with ``routes=[...]``.
+    routes: list[RouteRule] = Field(
+        default_factory=list,
+        validation_alias=AliasChoices("ROUTES_JSON", "routes"),
+    )
+
     # Observability
     enable_metrics: bool = Field(default=True)
     log_level: str = Field(default="INFO")
 
+    @field_validator("routes", mode="before")
+    @classmethod
+    def _parse_routes(cls, value: object) -> object:
+        """Accept ``ROUTES_JSON`` as a JSON string or an already-parsed list."""
+        if isinstance(value, str):
+            stripped = value.strip()
+            if not stripped:
+                return []
+            try:
+                parsed = json.loads(stripped)
+            except json.JSONDecodeError as exc:
+                raise ValueError(f"ROUTES_JSON is not valid JSON: {exc}") from exc
+            if not isinstance(parsed, list):
+                raise ValueError("ROUTES_JSON must be a JSON array of route objects")
+            return parsed
+        return value
+
     @property
     def database_url_str(self) -> str | None:
         return str(self.database_url) if self.database_url else None

opengateway/mojo_bridge/__init__.py

@@ -27,15 +27,13 @@
 from opengateway.mojo_bridge.auth import AuthResult, authenticate_authorization
 from opengateway.mojo_bridge.chat import chat_completion, health_check
 from opengateway.mojo_bridge.stream import start_streaming_chat
-from opengateway.mojo_bridge.tls_proxy import start_tls_proxy
 
 __all__ = [
     "handle_chat",
     "handle_chat_stream",
     "health_check",
     "authenticate_authorization",
     "AuthResult",
-    "start_tls_proxy",
 ]
 
 logger = logging.getLogger("opengateway.mojo_bridge")

opengateway/mojo_bridge/auth.py

@@ -35,6 +35,11 @@ class AuthResult:
     budget_used: float
     tpm_limit: int | None
     rpm_limit: int | None
+    # Dollar cost cap (USD). Independent from ``max_budget`` which is
+    # token-denominated (ADR-003 §166). When set, the bridge estimates
+    # the upstream cost of each request and rejects before the call when
+    # the estimate would exceed this cap. ``None`` = no dollar cap.
+    max_cost_usd: float | None = None
 
 
 # key_hash -> (AuthResult, expires_at). In-process only; revocation
@@ -101,6 +106,7 @@ def _lookup_virtual_key(key_hash: str) -> AuthResult | None:
         models=record.models,
         max_budget=record.max_budget,
         budget_used=record.budget_used,
+        max_cost_usd=record.max_cost_usd,
         tpm_limit=record.tpm_limit,
         rpm_limit=record.rpm_limit,
     )