Claude Code Flight Recorder (Phase 1): sessions, hooks, SQLite, redaction, timelines - #3
Claude Code Flight Recorder (Phase 1): sessions, hooks, SQLite, redaction, timelines#3juancgarza wants to merge 5 commits into
Conversation
…timelines
Pivot the proxy from request-centric to session-centric (Phase 1).
- Ingest Claude Code lifecycle hooks at POST /events/claude-code and
reconstruct durable coding sessions.
- Persist sessions, normalized session events, and API requests to SQLite
(modernc.org/sqlite, pure-Go) behind a Repository interface.
- Correlate API traffic to sessions best-effort (prompt content match +
most-recent-active fallback) since the API carries no session id.
- Redact secrets (API keys, tokens, AWS keys, PEM blocks, bearer tokens)
from bodies, stream events, and hook payloads before persistence.
- Control plane: /sessions, /sessions/{id}, /sessions/{id}/timeline, plus
optional bearer auth and localhost-only WebSocket origin + CORS.
- Session-first dashboard (Sessions | Requests) with live timelines.
- Portable curl-based hook installer (idempotent jq merge).
Hardening from an adversarial review pass: deep-copy records out of the
store (data race), redact derived prompt/summary fields, bound WebSocket
writes, guard hook endpoint with the dashboard token, and degrade to a
no-op repository if the database cannot be opened.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…manager Turn raw hook events into signal-ready structured data (Phase 3-lite). - commands.go: ClassifyCommand categorizes shell commands (package_manager|test|lint|typecheck|build|git|migration|dangerous) and detects the JS package manager invoked; extractToolOutcome pulls exit code / success from a PostToolUse tool_response across its several shapes. - Detect the repo package manager from lockfiles at SessionStart. - A failing PostToolUse becomes a distinct tool.failure timeline event. - SessionEvent gains command/commandCategory/packageManager/exitCode/success; Session gains packageManager. Commands are redacted before persistence. - Additive SQLite columns with an idempotent ALTER TABLE migration so existing databases upgrade in place. - Dashboard timeline renders command text, category/pm badges, exit codes, and failure styling; session header shows the package manager. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ssues view
Turn the recorder into an insight engine (Phase 4).
- signals.go: deterministic detectors over enriched session events —
repeated_failed_command, package_manager_mismatch, edited_generated_file,
human_correction, token_bloat. Each yields an evidence-linked Signal with a
severity and a suggestedFixType (rule|skill|hook|claude_md) for Phase 5.
- AggregateSignals groups per-session signals by type for a "top issues" view.
- API: GET /sessions/{id}/signals (per session) and GET /signals (aggregate
across recent sessions), reusing the existing auth + CORS wrappers.
- Dashboard: a third "Signals" view (top recurring issues with severity,
counts, suggested fix, and example sessions) plus a per-session Signals
section in the detail pane.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adversarial review of the C/B changes surfaced six confirmed defects:
- tool.failure was effectively never emitted: the Bash PostToolUse payload is
{interrupted,stderr,stdout,...} with no exit code, so extractToolOutcome
returned unknown and detectRepeatedFailedCommand/the failure UI were dead.
Register PostToolUseFailure in the installer and derive failure from the real
`interrupted` field. (Full exit-code fidelity needs transcript ingestion.)
- A leading env-var assignment (NODE_ENV=production npm run build) fell through
primarySegment and classified as unknown with no package manager; now strip
leading VAR=val assignments and env/sudo/nice wrappers before classifying.
- `curl … | bash` and other dangerous patterns in piped/chained portions were
unreachable (primarySegment stripped the pipe); scan the full command for
dangerous patterns first. Also catch plain `chmod 777` (optional flag).
- GET /signals is O(sessions × events) and was refetched on every live
SESSION_UPDATE; add a 5s server-side cache and switch the dashboard to a
fixed-interval refresh keyed on new sessions only.
- token_bloat no longer overloads Signal.Count with the token total.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Tier 1 of subagent/newer-feature awareness — persist fields the hook payload already carries and start capturing subagent lifecycle. - SessionEvent/TimelineEvent gain tool_use_id (pairs PreToolUse<->PostToolUse) and prompt_id (groups a user turn); additive columns + migration. - Register SubagentStop and PreCompact hooks in the installer (already mapped in normalizeHookType), so Task-subagent completion and context compaction are recorded. - Dashboard timeline groups events into turns by prompt_id and renders Task spawns as "Subagent" spans plus subagent.stop / context-compaction events. Groundwork for Tier 2 (transcript ingestion), which will use the transcript's requestId/isSidechain/toolUseResult for exact correlation, true subagent hierarchy, and real tool outcomes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| if [[ -n "$TOKEN" ]]; then | ||
| out="${out//-H 'content-type: application/json'/-H 'content-type: application/json' -H 'authorization: Bearer ${TOKEN}'}" | ||
| fi |
There was a problem hiding this comment.
🔴 Hook installer never adds the auth token, so session recording silently stops when the proxy requires one
The bearer token is never inserted into the generated hook commands (the ${out//...} substitution at install/install-hooks.sh:29) because the replacement expression never matches, so every hook the CLI sends is rejected and nothing is recorded.
Impact: Anyone who protects the proxy with a token installs hooks that are silently refused, so no coding sessions, prompts or timelines ever appear in the dashboard.
Why the bash pattern substitution silently no-ops
In ${var//pattern/string} bash terminates the pattern at the first unquoted /. The pattern here contains application/json, so bash parses the pattern as -H 'content-type: application (quotes removed → -H content-type: application) and treats the rest as the replacement. That pattern never matches the rendered template text, so out is returned unchanged.
Verified locally:
$ out="curl -H 'content-type: application/json' --data-binary @-"; TOKEN=abc
$ echo "${out//-H 'content-type: application/json'/-H 'content-type: application/json' -H 'authorization: Bearer ${TOKEN}'}"
curl -H 'content-type: application/json' --data-binary @- # unchanged
Because the hook commands end in >/dev/null 2>&1 || true, the resulting 401 from RequireToken (proxy/main.go:55) is invisible to the user.
| if [[ -n "$TOKEN" ]]; then | |
| out="${out//-H 'content-type: application/json'/-H 'content-type: application/json' -H 'authorization: Bearer ${TOKEN}'}" | |
| fi | |
| if [[ -n "$TOKEN" ]]; then | |
| out="$(printf '%s' "$out" | sed "s#-H 'content-type: application/json'#-H 'content-type: application/json' -H 'authorization: Bearer ${TOKEN//#/\\#}'#g")" | |
| fi |
Was this helpful? React with 👍 or 👎 to provide feedback.
| sess, err := m.repo.GetSession(hook.SessionID) | ||
| if err != nil { | ||
| log.Printf("hook: get session: %v", err) | ||
| } | ||
| isNew := sess == nil | ||
| if isNew { | ||
| sess = &Session{ | ||
| ID: hook.SessionID, | ||
| Provider: ProviderAnthropic, | ||
| CLI: "claude-code", | ||
| StartedAt: t, | ||
| Status: "active", | ||
| } | ||
| } |
There was a problem hiding this comment.
🟡 A momentary database read failure erases a session's accumulated token and request totals
A session that cannot be read back is treated as brand new and written out with zeroed counters (GetSession error path at proxy/hooks.go:100-113) instead of being left alone, so the previously accumulated totals for that session are permanently overwritten.
Impact: After a transient storage hiccup, an in-progress session's request count and token usage reset to zero in the dashboard and stay wrong for the rest of the session.
Read-modify-write cycle loses data on read error
ingest logs the error from m.repo.GetSession but then falls through to isNew := sess == nil, builds a fresh Session{} with InputTokens/OutputTokens/RequestCount == 0, and calls UpsertSession. The SQL upsert's ON CONFLICT(id) DO UPDATE (proxy/repository.go:79-96) unconditionally sets input_tokens, output_tokens and request_count from the new zeroed row, clobbering the stored values. It also re-broadcasts SESSION_START for an already-known session.
A safer shape is to bail out of the ingest (or skip the upsert of aggregate columns) when the read failed, distinguishing "not found" from "error".
Prompt for agents
In proxy/hooks.go, SessionManager.ingest calls m.repo.GetSession and, on error, only logs it before treating sess == nil as "new session". Because SQLiteRepository.UpsertSession overwrites input_tokens, output_tokens and request_count from the passed struct, a failed read causes the existing row's accumulated counters to be reset to zero, and a spurious SESSION_START is broadcast. Distinguish a genuine "not found" (nil, nil) from an error: on error, either abort the ingest for that hook (still returning 200 to the CLI) or avoid the upsert of aggregate columns. The same pattern exists in OnRequestStart/OnRequestComplete, which discard the error from GetSession entirely.
Was this helpful? React with 👍 or 👎 to provide feedback.
| func jsPackageManager(head string) string { | ||
| switch head { | ||
| case "npm", "npx": | ||
| return "npm" | ||
| case "pnpm", "pnpx": | ||
| return "pnpm" | ||
| case "yarn": | ||
| return "yarn" | ||
| case "bun", "bunx": | ||
| return "bun" | ||
| } | ||
| return "" | ||
| } |
There was a problem hiding this comment.
🟡 Running npx in a pnpm or yarn repo is reported as a high-severity wrong-package-manager problem
Commands run through npx are recorded as if they used npm (jsPackageManager at proxy/commands.go:100-101), so in a repository whose lockfile says pnpm/yarn/bun every such command is counted as using the wrong package manager.
Impact: The Signals view shows false high-severity "wrong package manager" issues for perfectly normal commands like npx tsc, drowning out real findings.
Detector path and a related substring-matching issue
jsPackageManager maps npx→npm, pnpx→pnpm, bunx→bun. detectPackageManagerMismatch (proxy/signals.go:94-121) then flags any event whose PackageManager differs from the session's lockfile-derived manager, with severity high. npx <bin> is the conventional binary runner even in pnpm/yarn repos, so it should not count as an install-manager mismatch (only real manager subcommands such as install/add/run should).
Separately, classifyPMSub (proxy/commands.go:113-132) matches bare substrings against the whole command, so pnpm add @testing-library/react matches test and is categorized as a test run rather than a package install (latest, rebuild, etc. have the same problem). That only affects timeline labels, but it comes from the same classification code.
Prompt for agents
proxy/commands.go maps npx→npm, pnpx→pnpm and bunx→bun in jsPackageManager, and proxy/signals.go detectPackageManagerMismatch flags any event whose PackageManager differs from the session's lockfile-derived manager as a high-severity signal. Because npx is the standard binary runner regardless of the repo's package manager, this produces false positives. Consider distinguishing 'runner' invocations (npx/pnpx/bunx/`yarn dlx`) from real manager invocations — e.g. record the runner separately, or only set CommandInfo.PackageManager for manager subcommands (install/add/remove/run/test/...), so the mismatch detector only fires on genuine manager usage.
Was this helpful? React with 👍 or 👎 to provide feedback.
| func tokenAuthorized(r *http.Request, token string) bool { | ||
| if constantTimeEqual(r.URL.Query().Get("token"), token) { | ||
| return true | ||
| } | ||
| const prefix = "Bearer " | ||
| auth := r.Header.Get("Authorization") | ||
| if !strings.HasPrefix(auth, prefix) { | ||
| return false | ||
| } | ||
| return constantTimeEqual(strings.TrimSpace(auth[len(prefix):]), token) | ||
| } |
There was a problem hiding this comment.
🟨 Control-plane token can be leaked via URL query parameter
tokenAuthorized accepts the dashboard bearer token from the token query parameter as well as the Authorization header (proxy/api.go:77-87). The dashboard puts the token into the WebSocket URL (dashboard/src/app/lib/api.ts:8-11), and the same check guards the REST endpoints and the hook ingestion endpoint. Query-string credentials end up in browser history, referrers, and any HTTP access logs, and are more likely to be shoulder-surfed or copied into bug reports than a header value.
Was this helpful? React with 👍 or 👎 to provide feedback.
| cfg := Config{ | ||
| ProxyAddr: envOr("CCPROXY_PROXY_ADDR", ":8080"), | ||
| ControlAddr: envOr("CCPROXY_CONTROL_ADDR", ":8081"), | ||
| DBPath: envOr("CCPROXY_DB_PATH", defaultDBPath()), | ||
| DashboardToken: os.Getenv("CCPROXY_DASHBOARD_TOKEN"), | ||
| MaxLiveItems: 1000, | ||
| } |
There was a problem hiding this comment.
🟨 Control plane is unauthenticated by default and binds to all interfaces
CCPROXY_DASHBOARD_TOKEN is unset by default (proxy/config.go:23), so RequireToken passes every request through unguarded, and the control plane listens on :8081 (all interfaces, proxy/config.go:21) rather than 127.0.0.1:8081. On a shared or untrusted network anyone who can reach the port can read /sessions, /sessions/{id}/timeline and the stored request records — which contain user prompts, file paths and source code — and can POST forged hook events to /events/claude-code. The WebSocket origin check only stops browsers, not direct clients (proxy/websocket.go:23-28 returns true when no Origin header is present).
Was this helpful? React with 👍 or 👎 to provide feedback.
| local out | ||
| out="$(sed "s#http://localhost:8081/events/claude-code#${URL//#/\\#}#g" "$TEMPLATE")" | ||
| if [[ -n "$TOKEN" ]]; then | ||
| out="${out//-H 'content-type: application/json'/-H 'content-type: application/json' -H 'authorization: Bearer ${TOKEN}'}" | ||
| fi | ||
| printf '%s\n' "$out" | ||
| } |
There was a problem hiding this comment.
🟨 Hook installer writes the auth token in cleartext into the user's Claude settings file
When CCPROXY_DASHBOARD_TOKEN/CCPROXY_TOKEN is set, the installer is intended to embed -H 'authorization: Bearer <token>' directly into the hook command strings persisted in ~/.claude/settings.json (install/install-hooks.sh:28-30). The settings file is created with default permissions and is commonly committed or shared, so the control-plane token would be stored in cleartext alongside ordinary configuration.
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
Summary
Evolves the proxy from a request inspector into a session-level flight recorder for Claude Code / Codex. It ingests Claude Code lifecycle hooks, reconstructs durable coding sessions, correlates API traffic to them, redacts secrets before persistence, and adds a session-first dashboard with normalized timelines.
What's new
POST /events/claude-codereconstructs sessions fromSessionStart/UserPromptSubmit/PreToolUse/PostToolUse/SessionEnd.modernc.org/sqlite, pure-Go, no cgo) behind aRepositoryinterface; survives restarts./sessions,/sessions/{id},/sessions/{id}/timeline; optional bearer auth; localhost-only WebSocket origin + CORS for the dashboard.curlhooks, idempotent jq merge that preserves existing user hooks.Review & hardening
An adversarial multi-agent review produced 10 verified findings, all applied: deep-copy records out of the store (data race, clean under
-race), redact derived prompt/summary fields, bound WebSocket writes + single-writer-per-conn, token-guard the hook endpoint, constant-time token compare, string-encodeseq, correlation eligibility fixes, and degrade to a no-op repository on DB-open failure. A late CORS fix resolved the dashboard timeline not populating.Verification
go build/vet/test/test -race/gofmtclean (Go 1.18); dashboardtsc --noEmit+next buildgreen. Runtime-smoked end to end: hooks → session → timeline, token auth, prompt redaction, DB-failure fallback, CORS.Not in scope (later phases)
Richer command/test/failure timeline enrichment (Phase 3) and deterministic signal detection (Phase 4) land as follow-up commits on this branch.
🤖 Generated with Claude Code