U9: Prometheus metrics endpoint#13
Conversation
Adds GET /api/metrics exposing system health in Prometheus text format for scraping by Prometheus/Grafana. Collects 7 metric families (models, exams, gateway requests, routing latency p50/p99, learned provider limits, active fail streaks, Redis memory) in parallel with per-query error isolation so one failure can't kill the endpoint. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds a new Next.js route handler at GET /api/metrics to expose operational data in Prometheus text exposition format for scraping/visualization.
Changes:
- Introduces
/api/metricsendpoint returningtext/plain; version=0.0.4with multiple metric families. - Collects metrics from Postgres + Redis in parallel with per-query failure isolation.
- Forces dynamic evaluation and disables caching for live scrapes.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const labels = s.labels | ||
| ? "{" + | ||
| Object.entries(s.labels) | ||
| .map(([k, v]) => `${k}="${String(v).replace(/"/g, '\\"')}"`) | ||
| .join(",") + |
There was a problem hiding this comment.
Prometheus label values must escape backslashes and newlines as well as quotes (per text exposition format). Current rendering only escapes ", so a provider/model id containing \ or a newline can produce invalid metrics and break scraping. Consider a dedicated escape helper that replaces \\ -> \\\\, \n -> \\n, then " -> \\" (and apply similar escaping to HELP strings if they ever become dynamic).
| async function safe<T>(fn: () => Promise<T[]>): Promise<T[]> { | ||
| try { | ||
| return await fn(); | ||
| } catch { | ||
| return []; | ||
| } |
There was a problem hiding this comment.
safe() swallows all query errors and returns [] without logging. For a metrics endpoint this makes failures invisible during incident response (you'll just see missing metrics). Consider logging the error with a short query/metric identifier (or returning a structured { ok, rows } result) so partial failures are discoverable.
| metrics.push({ | ||
| name: "bcproxy_active_fail_streaks", | ||
| help: "Number of models currently in a fail streak (streak_count > 0)", | ||
| type: "gauge", | ||
| samples: [{ value: failStreakRows[0]?.active ?? 0 }], | ||
| }); |
There was a problem hiding this comment.
bcproxy_active_fail_streaks will report 0 if the underlying DB query fails (since safe() returns [] and the code falls back to ?? 0). This can mask outages and trigger false "all clear" dashboards/alerts. Prefer omitting the metric when the query fails, or have safe() surface an explicit failure signal so you don't emit a synthetic zero.
| metrics.push({ | |
| name: "bcproxy_active_fail_streaks", | |
| help: "Number of models currently in a fail streak (streak_count > 0)", | |
| type: "gauge", | |
| samples: [{ value: failStreakRows[0]?.active ?? 0 }], | |
| }); | |
| const activeFailStreaks = failStreakRows[0]?.active; | |
| if (activeFailStreaks != null) { | |
| metrics.push({ | |
| name: "bcproxy_active_fail_streaks", | |
| help: "Number of models currently in a fail streak (streak_count > 0)", | |
| type: "gauge", | |
| samples: [{ value: activeFailStreaks }], | |
| }); | |
| } |
| }, | ||
| }); | ||
| } catch (err) { | ||
| return new NextResponse(`# metrics error: ${String(err)}\n`, { |
There was a problem hiding this comment.
The 500 response body includes String(err) which may leak internal details (SQL, connection errors, stack/message content) to anyone who can hit /api/metrics. Safer pattern is to log the error server-side and return a generic # metrics error line without embedding the raw error text.
| return new NextResponse(`# metrics error: ${String(err)}\n`, { | |
| console.error("Failed to collect metrics", err); | |
| return new NextResponse(`# metrics error\n`, { |
| async function collectMetrics(): Promise<Line[]> { | ||
| const sql = getSqlClient(); | ||
|
|
||
| const [ | ||
| modelsRows, | ||
| examRows, | ||
| gatewayRows, | ||
| latencyRows, | ||
| limitRows, | ||
| failStreakRows, | ||
| redisUsed, | ||
| ] = await Promise.all([ | ||
| safe( | ||
| () => sql<{ provider: string; total: number; active: number; cooldown: number }[]>` |
There was a problem hiding this comment.
This handler runs several potentially expensive DB queries (e.g., percentile over 24h) on every scrape and exports per-model limit series (high-cardinality). With typical Prometheus scrape intervals (e.g., 15s), this can create avoidable load on Postgres/Redis. Consider adding a short in-memory memo/TTL (similar to /api/infra) and/or reducing query cost/cardinality (pre-aggregation, longer scrape interval guidance, or gating high-cardinality metrics behind a flag).
- Add HuggingFace as provider jaturapornchai#13 (scanner, providers, api-keys, shared, SpeedRace) - Add Setup Modal: save API keys via web UI to DB, auto-scan without restart - Add /api/providers + /api/setup endpoints, api_keys table in DB - Rewrite benchmark: 10 questions across 8 categories (thai/code/math/instruction/creative/knowledge/vision/audio) - Category-aware smart routing: prompt category → benchmark score → model selection - Vision routing: priority providers, Ollama URL→base64 conversion, fake vision detection - Fix routing: exclude h.status='error' models, add HTTP 410 cooldown (7 days) - Redesign nav menu: 2-row layout with emoji icons, wrappable - Add provider status cards on dashboard with key status - Rewrite README for 13 providers + all new features Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
/guide Hermes snippet was based on old CLI (hermes model add …, config.toml) that doesn't match today's Hermes v0.10.0. Replaced with the exact commands verified on a fresh Ubuntu 24.04 (WSL2) install — including the provider=custom trick that redirects Hermes away from openrouter.ai. Also bumps README: - Smart Routing: 13 → 15 steps. Added #12 Mistral message-order auto-patch + #13 Client-400 short-circuit (both shipped today as fixes for Hermes/OpenClaw compatibility). - Feature table: new 🤖 Agent-ready row. - Integration: new "Hermes Agent (Nous Research)" subsection with install + config + one-liner demo. Verified in production: 13/16 built-in Hermes tools pass cleanly through SMLGateway (terminal, file RW, code_execution, todo, vision, tts, clarify, session_search, skills, delegation, moa). 3 require extra config (memory-persist must be explicit, cronjob needs gateway service, delegation subagent self-corrects unknown tool names — all non-gateway issues). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summary
GET /api/metricsendpoint emitting Prometheus text format (text/plain; version=0.0.4) for scraping by Prometheus/GrafanaPromise.allwith per-query error isolation so a single failure returns an empty section rather than killing the endpointforce-dynamic+no-storeso scrapes always hit live dataTest plan
rtk npx next build— Errors: 0 | Warnings: 0http://localhost:3334/api/metricsafter deploy, confirm 200 and# HELP bcproxy_models_total ...in body/api/metricsand verify metrics land in GrafanaNote:
npm testhas pre-existing failures inscanner.test.tsunrelated to this change (DB mocking issue in a different file).