Add cost and latency tracking per model/query

[rchojn]

Problem

SafeReader uses multiple models (gemma2, llama3, qwen) but we don't track which models are cost-effective or fast enough.

Proposed Solution

Add cost and latency dashboard showing:

• Total cost per model (based on token usage)
• Average latency per model
• Cost per query type (simple vs complex)
• Trending over time

Competitive Context

• Langfuse: Full cost tracking with provider-specific pricing
• Helicone: Native cost tracking, saves up to 95% with caching
• Phoenix: Latency metrics and performance monitoring

GoEvals approach: Lightweight cost/latency aggregation from JSONL, no provider integration needed.

Implementation

1. Extend JSONL to include cost/latency:

{
  "timestamp": "2025-10-26T14:30:00Z",
  "model": "gemma2:2b",
  "scores": {"combined": 0.85},
  "response_time_ms": 1234,
  "tokens": {
    "prompt": 450,
    "completion": 120,
    "total": 570
  },
  "cost_usd": 0.0002
}

2. Add dashboard metrics:
   • Per-model summary table: Model, Avg Latency, Total Cost, Queries
   • Time-series chart: Cost/latency trending over days/weeks
   • Cost breakdown: Which queries are most expensive?

Value for SafeReader

• Optimize model choice: Is gemma2:2b fast enough? Should we use llama3:8b?
• Budget forecasting: At 100 queries/day, what's monthly cost?
• Performance regression: Did latest Ollama update slow down responses?

Philosophy

• Manual cost input - user provides pricing (Ollama is free, OpenAI has rates)
• Stdlib only - aggregation in Go, no external DB
• Local-first - all calculations from JSONL files
• Progressive enhancement - cost tracking is optional

Success Metrics

• SafeReader can identify cheapest model in < 30 seconds
• Dashboard shows cost trends without external API calls
• Zero cloud dependencies