Add optional WebSocket support for real-time updates

[rchojn]

Problem

GoEvals currently requires manual refresh to see new eval results. Competitors offer real-time updates through WebSockets or server-sent events.

Proposed Solution

Add optional WebSocket support for real-time dashboard updates:

• Server pushes new results as they're appended to JSONL
• Dashboard auto-refreshes without page reload
• Configurable (can disable for simplicity)
• Graceful fallback to HTTP polling

Competitive Context

• Langfuse: Real-time tracing via WebSockets
• Helicone: Live dashboard updates
• Phoenix: Real-time span updates

GoEvals approach: WebSocket as progressive enhancement, HTTP polling remains default.

Implementation

1. Add WebSocket endpoint:

// Optional WebSocket support
func handleWebSocket(w http.ResponseWriter, r *http.Request) {
    upgrader := websocket.Upgrader{}
    conn, _ := upgrader.Upgrade(w, r, nil)

    // Watch JSONL files for changes
    watcher := watchFiles(filePaths)
    for update := range watcher {
        conn.WriteJSON(update)  // Push to client
    }
}

2. Client-side connection:

// Try WebSocket first, fallback to polling
const ws = new WebSocket('ws://localhost:3000/ws')
ws.onmessage = (e) => {
    const newResults = JSON.parse(e.data)
    updateDashboard(newResults)  // Live update
}
ws.onerror = () => {
    startPolling()  // Fallback to HTTP polling
}

Value for SafeReader

• Live feedback: See eval results appear in real-time as SafeReader runs tests
• No manual refresh: Dashboard updates automatically
• Better UX: Feels more responsive during long eval runs

Philosophy

• Progressive enhancement - WebSocket is optional, HTTP polling works without it
• Stdlib preferred - use gorilla/websocket if needed (minimal external dep)
• Local-first - WebSocket for localhost only
• Fallback support - HTTP polling remains default for compatibility

Trade-offs

Pros:

• Real-time updates during eval runs
• Better developer experience
• Modern dashboard feel

Cons:

• Adds WebSocket dependency (gorilla/websocket)
• More complex than HTTP polling
• Requires file watching (inotify/fsnotify)

Decision: Make it optional via flag:

# With WebSocket (real-time)
./goevals --realtime evals.jsonl

# Without WebSocket (HTTP polling - default)
./goevals evals.jsonl

Success Metrics

• Real-time updates appear within 500ms of JSONL append
• Graceful fallback to polling if WebSocket fails
• No breaking changes to existing HTTP polling

Labels