Skip to content

cost.md

Latest commit

 

History

History
61 lines (43 loc) · 2.1 KB

cost.md

File metadata and controls

61 lines (43 loc) · 2.1 KB
Raw

Layer 5 · Cost observability

Per-query Claude spend is recorded in Langfuse (cost_usd on each insightiq_query trace) and in the query_audit table.

Langfuse cost dashboard

  1. Open Langfuse → your InsightIQ project.
  2. Go to Traces and filter by name insightiq_query.
  3. Open any trace → check metadata cost_usd, input_tokens, output_tokens.
  4. For aggregate cost: Analytics → group by trace name or use the cost column if enabled on your plan.

Example trace (production)

Langfuse trace — MRR trend query with one retry

Real insightiq_query trace (claude-sonnet-4-5, June 2026):

Metric Value
Question “What is our MRR trend for the last 6 months?”
Total cost $0.118
Tokens (prompt / completion) 38,488 / 201
Retries 1
Total latency 17.45s
DB execution 62ms · 6 rows

Each generate_sql span in the trace is ~19k prompt tokens and ~$0.059 on its own.

Typical cost per query

At claude-sonnet-4-5 rates ($3/MTok in, $15/MTok out):

Scenario Tokens (in / out) Cost
Measured — one generate_sql call ~19,250 / ~100 ~$0.06
Measured — with 1 retry (trace above) ~38,500 / ~200 ~$0.12
Optimized (caching / slim schema) ~1,000 / ~100 ~$0.004

Retries add linearly — each extra generate_sql costs another ~$0.06 at the current schema-injection size.

Daily budget alert

Set in production:

Variable Default Purpose
BUDGET_USD_DAILY 2.00 In-app daily spend cap (alert threshold)
COST_ALERT_WEBHOOK_URL (empty) Slack-compatible webhook; one alert per UTC day

The GitHub cost-monitor.yml workflow still checks Anthropic’s billing API at 09:00 UTC as a second line of defense.

SQL: average cost from audit log

SELECT
    ROUND(AVG(cost_usd)::numeric, 6) AS avg_cost_usd,
    COUNT(*) AS queries,
    ROUND(SUM(cost_usd)::numeric, 4) AS total_usd
FROM query_audit
WHERE verdict = 'allowed' AND cost_usd > 0;