Reliability: stable host + container-per-call scaling story

[teetangh]

Context

The current deployment story is a cloudflared quick-tunnel (URL churn broke Twilio webhooks for ~30 s — engineering-log #10) with a single uvicorn process running every concurrent call's Pipecat pipeline in one event loop. The practical concurrency ceiling is the per-call local Smart Turn ONNX inference (~30 ms CPU each) contending in that loop. Pipecat's production guidance is one container per session (or Pipecat Cloud); Redis/Postgres are in docker-compose.yml but unused by the voice paths.

Implementation plan

1. Stable host: deploy the API behind a stable HTTPS/WSS hostname (Fly.io / Cloud Run / a VM + Caddy; or a named cloudflared tunnel with a reserved hostname as the minimal step). WEBHOOK_BASE_URL becomes a stable config value.
2. Concurrency benchmark first: measure max concurrent simulator calls on one container (CPU-bound Smart Turn is the suspected ceiling) to size the next step honestly.
3. Scale-out pattern — pick one and document as an ADR:
   • Container-per-call: a lightweight dispatcher receives dial-init/TwiML webhooks and spawns/routes each call's WS to a dedicated worker container (Cerebrium/Fly machines-style), or
   • Pipecat Cloud: move the pipeline to Pipecat Cloud and keep this API for dial-init/business logic.
4. docker-compose truth: with the P0 Docker fixes landed, make compose actually wire Redis (session store issue) and Postgres (persistence issues) into the api/worker services; add healthchecks.
5. Load-test: N concurrent simulator calls (target ≥ 10) with per-call TTFB metrics recorded; document results in the roadmap doc.

Acceptance criteria

• Two (then ≥10) concurrent calls on the deployed host complete cleanly with per-stage TTFB p95 within 1.5× single-call baseline.
• Twilio webhooks survive an app restart/redeploy without changing configured URLs.
• ADR documenting the chosen scaling pattern with the measured ceiling.

Dependencies

• P0 fixes PR (Docker images must build/run voice).
• Reliability: Redis-backed session store (replace in-memory _SESSIONS/_PENDING) #4 Redis-backed session store (cross-container session lookup).