Visual debugging, execution traces, and observability for AI agents.
When your AI agent makes a bad decision at step 7 of a 12-step workflow, Pathlight shows you exactly what happened — what went in, what came out, how long it took, and where in your code it was called. Then it lets you edit that step's prompt and re-run it against the real model, diff two runs side-by-side, pause the agent mid-execution, or block a PR when latency regresses.
No more debugging agents with console.log.
| Feature | Why you'd use it | Docs |
|---|---|---|
| Waterfall timeline | Visual answer to "where is the time going?" | overview |
| Span inspector | Full input/output/metadata on click, side-by-side with timeline | overview |
| Real-time stream | New traces and status changes land without refresh | docs/realtime.md |
| Trace diff | Side-by-side compare of two traces; "did my prompt change break anything?" | docs/trace-diff.md |
| Git-linked regressions | SDK auto-captures commit SHA; /commits page flags >25% cost/latency regressions |
docs/git-regressions.md |
| Live breakpoints | await tl.breakpoint(...) pauses your agent; edit state on the dashboard and resume |
docs/breakpoints.md |
| LLM replay | Edit messages on any LLM span and re-run against the real provider | docs/replay.md |
| Eval-as-code + CI | expect(trace).toCostLessThan(0.10) — assert over recent traces, exit nonzero in CI |
packages/eval/README.md |
| Code-fixing agent | BYOK LLM reads your failing trace + source and proposes a unified diff. CLI, dashboard button, bisect across commits. |
docs/fix.md |
| BYOK key storage | Encrypted-at-rest LLM keys + git tokens per project. Dashboard picker uses stored IDs; plaintext never touches the browser. | docs/byok-keys.md |
| OpenClaw plugin | @pathlight/openclaw captures OpenClaw agent runs, LLM calls, tool execution, and sub-agent delegation with git provenance. |
docs/openclaw-plugin.md |
| Eventloom visualizer | Dedicated Capture, Replay, and Handoff panels for Eventloom-exported event logs | docs/eventloom.md |
pathlight share |
Single-file HTML snapshot of a trace, zero deps to open | packages/cli/README.md |
| OpenTelemetry interop | Collector accepts OTLP/HTTP; any OTel-instrumented app can ship to Pathlight | docs/opentelemetry.md |
| Python SDK | pip install pathlight — same dashboard features, Pythonic API, sync + async |
docs/python.md |
| One-command deploy | docker compose up -d pulls prebuilt images from GHCR; SQLite persists in a named volume |
docs/docker.md |
| Automatic source mapping | Every span records the file:line where it was created | overview |
| Issue detection | Failed spans + error-pattern matches flag traces in the list | overview |
Full chronological list in CHANGELOG.md.
git clone https://github.com/syndicalt/pathlight.git
cd pathlight
docker compose up -dThat's it. Dashboard at http://localhost:3100, collector at
http://localhost:4100. Trace data lives in the pathlight_data named
volume — survives restarts; docker compose down -v wipes it.
Prebuilt images are published to GHCR so the first up pulls rather
than builds.
git clone https://github.com/syndicalt/pathlight.git
cd pathlight
npm install
# Generate and apply database migrations
npm run db:generate -w packages/db
DATABASE_URL="file:$(pwd)/packages/collector/pathlight.db" \
npm run db:migrate -w packages/db
# Start collector (4100) + dashboard (3100) together
npx turbo dev- Collector: http://localhost:4100 — receives trace data from your agent
- Dashboard: http://localhost:3100 — view, debug, diff, replay
npm install @pathlight/sdkimport { Pathlight } from "@pathlight/sdk";
const tl = new Pathlight({ baseUrl: "http://localhost:4100" });
const trace = tl.trace("research-agent", { query: "What is WebAssembly?" });
// Each step is a span
const llm = trace.span("classify", "llm", { model: "gpt-4o" });
const result = await openai.chat(/* … */);
await llm.end({ output: result, inputTokens: 50, outputTokens: 10 });
const search = trace.span("web-search", "tool", { toolName: "search" });
const results = await searchTool(result.query);
await search.end({ toolResult: results });
await trace.end({ output: finalAnswer });That's it. Open the dashboard and every trace appears in real time with full waterfall, source locations, token counts, and git provenance.
| Surface | Shot |
|---|---|
| Dashboard "Fix this" dialog with streaming diff | site/assets/fix.png |
BYOK key store at /settings/keys (masked previews) |
site/assets/byok.png |
| OpenClaw-generated trace with nested agent / llm / tool spans | site/assets/openclaw.png |
| Trace compare view (diff two runs side-by-side) | site/assets/diff.png |
| LLM replay editor on a span | site/assets/replay.png |
| Live breakpoint editor floating over the dashboard | site/assets/breakpoint.png |
All shots are captured against the running stack via scripts/seed-screenshots.mjs
examples/quote-agent— re-runnable for re-shooting after a UI change.
- Open the trace list. Commit badge on each row shows which SHA produced it.
- Pick the last good run and your new run; click Compare.
- See per-span duration delta, input/output JSON diff, and new/missing spans.
- Drop
state = await tl.breakpoint({ label: "post-retrieval", state: { docs, query } }). - Run the agent. When execution reaches the breakpoint, the dashboard's floating pulse badge lights up.
- Edit the JSON state, click Resume with edits — your agent continues with the modified value.
- Open a trace detail, click any LLM span.
- Edit the system prompt, messages, or model right in the inspector.
- Enter your provider API key (saved in
localStorageper-provider) and click Run replay.
- Write
specs/estimate.mjswithexpect(trace).toCostLessThan(0.05)etc. - Add a GitHub Actions step:
npx pathlight-eval specs/estimate.mjs --base-url …. - Merge gate fails when any trace violates the assertion.
pathlight share <trace-id> --out /tmp/bad-run.html- Attach the HTML to your GitHub issue / Slack thread. They open it in any browser — no dashboard, no server, no dependencies.
- Export the Eventloom log to Pathlight with
eventloom export pathlight. - Open the trace detail page.
- Use the Eventloom panel for Capture, Replay, and Handoff state; use the waterfall below it for spans, timing, and span-level inspection.
- Open the failing trace, click the broken span.
- Click Fix this in the inspector header. Pick a source (local path or remote git URL), a provider key from your BYOK store, and a mode (span / trace / bisect).
- SSE streams progress. Diff preview renders per-file with add/remove colorization. Click Apply to working tree (path mode) or Download .patch. See docs/fix.md.
- From the CLI:
pathlight fix <trace-id> --bisect --from <good-sha> --to <bad-sha> --git-url <url> - Binary-search walks the commit range in O(log N) probes and returns the regression SHA plus a proposed fix against that commit.
- In the dashboard: pick mode Bisect in the fix dialog — the result
screen shows a banner with the regression SHA linking to
/commits.
All agent runs at a glance. Status, duration, tokens, tags, commit badge. Unreviewed runs get a subtle left accent. Real-time — new traces and status updates appear without refresh. Multi-select two rows to compare.
Every span as a proportional bar showing when it started and how long it took relative to the total trace. Sequential steps cascade down. Issue spans get an amber highlight. Clicking a span splits the view: timeline shrinks left, inspector fills the right.
Side-by-side with the timeline. Shows:
- Status, duration, model, provider, tokens
- Source location — file:line automatically captured from the call stack
- Input / output / tool args / tool result / metadata (formatted JSON)
- Error details
- For LLM spans: full replay editor — edit messages and re-run
When a trace has metadata.visualizer.version = "eventloom.pathlight.visualizer.v1" and output.visualizer, the trace
detail page renders a dedicated Eventloom panel above the generic trace
input/output section. The panel has three tabs:
- Capture — ordered Eventloom facts and event type counts.
- Replay — integrity status, projection hash, and replay projection.
- Handoff — task state, telemetry, verification evidence, observability gaps, and next actions.
See docs/eventloom.md for the export contract and local smoke flow.
Groups recent traces by commit SHA. Per-commit aggregates (trace count, avg duration/tokens/cost, failure count) with deltas vs. the previous commit. Rows where a metric got ≥25% worse are tinted red so regressions can't hide.
Floating amber pulse badge appears the moment any agent hits a breakpoint. Click it to open a per-breakpoint editor: label, trace link, live JSON state editor, Resume / Resume with edits / Cancel.
| Type | Color | Use for |
|---|---|---|
llm |
Blue | LLM API calls (chat completions, embeddings) |
tool |
Green | Tool invocations (search, code exec, API calls) |
retrieval |
Violet | RAG retrieval, document fetches, knowledge base lookups |
agent |
Orange | Sub-agent invocations, delegation |
chain |
Cyan | Chain/pipeline steps, sequential processing |
custom |
Gray | Anything else |
pathlight/
├── apps/
│ └── web/ # Next.js 15 dashboard (port 3100)
│ └── src/
│ ├── app/
│ │ ├── page.tsx # Trace list + real-time stream
│ │ ├── traces/[id]/page.tsx # Trace detail, waterfall, inspector
│ │ ├── traces/compare/page.tsx # Side-by-side trace diff
│ │ └── commits/page.tsx # Per-commit regression view
│ ├── components/
│ │ ├── TopNav.tsx # Sticky horizontal nav
│ │ ├── BreakpointsPanel.tsx # Floating badge + slide-out editor
│ │ └── Fix/ # "Fix this" dialog — button, form, key picker,
│ │ # SSE stream, diff preview, apply/download/copy,
│ │ # bisect banner
│ └── lib/
│ ├── api.ts # Collector fetch helpers
│ ├── diff.ts # LCS line-diff utility
│ └── format.ts # Duration / token / timestamp formatters
├── packages/
│ ├── collector/ # Hono-based trace collector (port 4100)
│ │ └── src/
│ │ ├── routes/
│ │ │ ├── traces.ts # CRUD + SSE stream + /commits aggregate
│ │ │ ├── spans.ts # Span CRUD
│ │ │ ├── projects.ts # Project CRUD
│ │ │ ├── breakpoints.ts # Live breakpoint register/wait/resume/SSE
│ │ │ ├── replay.ts # LLM replay proxy (OpenAI + Anthropic)
│ │ │ ├── otlp.ts # OTLP/HTTP ingest (gen_ai.* mapping)
│ │ │ ├── fix.ts # POST /v1/fix — SSE wrapper around @pathlight/fix
│ │ │ ├── fix-apply.ts # POST /v1/fix-apply — git apply a diff locally
│ │ │ └── keys.ts # BYOK key CRUD (mounted only when PATHLIGHT_SEAL_KEY is set)
│ │ ├── events.ts # Trace EventEmitter for SSE fan-out
│ │ ├── breakpoints.ts # In-memory breakpoint registry
│ │ └── router.ts # CORS + route mounting
│ ├── db/ # Drizzle ORM + SQLite (builds to dist/)
│ │ ├── drizzle/ # Migrations (shipped with runtime)
│ │ └── src/
│ │ ├── schema.ts # traces, spans, events, projects, scores
│ │ ├── retire.ts # db:retire command
│ │ └── index.ts
│ ├── sdk/ # TypeScript SDK — @pathlight/sdk
│ │ └── src/index.ts # Pathlight, Trace, Span, breakpoint()
│ ├── sdk-python/ # Python SDK — pathlight on PyPI
│ │ ├── src/pathlight/
│ │ │ ├── client.py # Pathlight / AsyncPathlight classes
│ │ │ ├── git.py # Auto git-context capture
│ │ │ └── _source.py # Stack-walk source location
│ │ └── tests/ # pytest + pytest-httpx
│ ├── eval/ # Assertion DSL + pathlight-eval CLI
│ │ ├── bin/pathlight-eval.js
│ │ ├── examples/
│ │ └── src/index.ts # expect() matchers, evaluate()
│ ├── cli/ # pathlight CLI (subcommand router)
│ │ ├── bin/pathlight.js
│ │ └── src/
│ │ ├── commands/share.ts # pathlight share <trace-id>
│ │ ├── commands/fix.ts # pathlight fix <trace-id> — path/git/bisect modes
│ │ └── viewer-template.ts # Self-contained HTML viewer
│ ├── fix/ # Code-fixing agent core — @pathlight/fix
│ │ └── src/
│ │ ├── index.ts # fix() entry + composition
│ │ ├── types.ts # FixOptions / FixResult / FixMode / FixProgress
│ │ ├── source/{path,git}.ts # SourceReader implementations
│ │ ├── llm/{anthropic,openai}.ts # BYOK adapters behind a shared interface
│ │ ├── bisect.ts # O(log N) regression search
│ │ ├── prompt.ts # Trace + source → LLM messages; PROPOSE_FIX_TOOL schema
│ │ ├── diff-parser.ts # Tool-call → parsed unified diff
│ │ └── secrets.ts # Token/key redaction for error paths
│ ├── keys/ # BYOK encrypted key storage — @pathlight/keys (internal)
│ │ ├── LEAK-AUDIT.md # Per-release security checklist
│ │ └── src/
│ │ ├── seal.ts # libsodium crypto_secretbox primitives
│ │ ├── seal-key.ts # PATHLIGHT_SEAL_KEY loader (fail-stop on missing)
│ │ ├── store.ts # KeyStore: create/list/rotate/revoke/resolveSecret
│ │ └── resolver.ts # SecretResolver adapter for /v1/fix
│ └── openclaw-plugin/ # OpenClaw tracing plugin — @pathlight/openclaw
│ └── src/
│ ├── index.ts # definePluginEntry wiring
│ ├── hooks/ # trace-envelope, llm, tool, delegation
│ └── state.ts # Per-run state (trace + in-flight spans)
├── Dockerfile.collector # Multi-stage build; migrations run on boot
├── Dockerfile.web # Next.js 15 standalone runtime
├── docker-compose.yml # Collector + dashboard + SQLite volume
├── .github/workflows/
│ ├── ci.yml # Node + Python tests on every PR
│ ├── docker.yml # Publish GHCR images on push to master
│ ├── publish-python.yml # PyPI release on py-v* tag
│ └── ...
└── CHANGELOG.md
| Entity | Description |
|---|---|
| Trace | A complete agent execution. Status, input/output, duration, tokens, cost, git_commit/git_branch/git_dirty, reviewed_at. |
| Span | A single step within a trace. Types: llm, tool, retrieval, agent, chain, custom. Nests via parent_span_id. |
| Event | Point-in-time annotation within a span (logs, decisions, errors). Severity levels. |
| Project | Groups traces; has an API key for SDK auth. |
| Score | Quality annotation on a trace or span (human or auto). |
| Endpoint | Method | Description |
|---|---|---|
/v1/traces |
GET | List traces; filter by status, name, projectId; paginate with limit, offset |
/v1/traces |
POST | Create a trace. Accepts gitCommit, gitBranch, gitDirty |
/v1/traces/stream |
GET | SSE stream of trace.created / trace.updated / ping |
/v1/traces/commits |
GET | Per-commit aggregate stats (see git-regressions docs) |
/v1/traces/:id |
GET | Get trace with all spans, events, scores |
/v1/traces/:id |
PATCH | Update (status, output, duration, reviewedAt) |
/v1/traces/:id |
DELETE | Delete trace and all related rows |
| Endpoint | Method | Description |
|---|---|---|
/v1/spans |
POST | Create a span |
/v1/spans/:id |
PATCH | Update (status, output, tokens, cost, toolResult) |
/v1/spans/:id/events |
POST | Log an event |
| Endpoint | Method | Description |
|---|---|---|
/v1/breakpoints |
POST | Register + block until resumed or timed out (default 15m). Returns { state } |
/v1/breakpoints |
GET | List active breakpoints |
/v1/breakpoints/:id/resume |
POST | Resume with optional { state } override |
/v1/breakpoints/:id/cancel |
POST | Reject the waiting SDK call with 408 |
/v1/breakpoints/stream |
GET | SSE: snapshot, added, resolved, cancelled, ping |
| Endpoint | Method | Description |
|---|---|---|
/v1/replay/llm |
POST | Proxy LLM call. Body: { provider, model, messages, system?, apiKey?, baseUrl?, temperature?, maxTokens? } |
| Endpoint | Method | Description |
|---|---|---|
/v1/otlp/traces |
POST | OTLP/HTTP JSON ingest. Accepts resourceSpans[...]. Maps gen_ai.* attributes to Pathlight fields. See OTel docs. |
| Endpoint | Method | Description |
|---|---|---|
/v1/fix |
POST | Run @pathlight/fix against a failing trace. Response is text/event-stream with progress / chunk / result / error / done events. Resolves keyId/tokenId via the BYOK key store. See docs/fix.md. |
/v1/fix-apply |
POST | Write a unified diff to a local working tree via git apply. Body: { sourceDir, diff }. Runs git apply --check first. Restrict writable roots with PATHLIGHT_FIX_APPLY_ROOTS. |
Only mounted when PATHLIGHT_SEAL_KEY is set. See docs/byok-keys.md.
| Endpoint | Method | Description |
|---|---|---|
/v1/projects/:id/keys |
GET | List keys — masked metadata only |
/v1/projects/:id/keys |
POST | Create a key: { kind, provider, label, value } |
/v1/projects/:id/keys/:keyId |
PUT | Atomically rotate |
/v1/projects/:id/keys/:keyId |
DELETE | Revoke immediately |
| Endpoint | Method | Description |
|---|---|---|
/v1/projects |
GET | List projects |
/v1/projects |
POST | Create a project (returns API key) |
| Endpoint | Method | Description |
|---|---|---|
/health |
GET | Liveness probe |
const tl = new Pathlight({
baseUrl: "http://localhost:4100",
projectId: "my-project", // optional
apiKey: "tl_…", // optional
disableGitContext: false, // opt out of auto git capture
});const trace = tl.trace("agent-name", inputData, {
tags: ["production", "v2"],
metadata: { userId: "123" },
});
await trace.end({ output: result });
// or
await trace.end({ status: "failed", error: "…" });const span = trace.span("step-name", "llm", {
model: "gpt-4o",
provider: "openai",
input: { prompt: "…" },
toolName: "search", // for tool spans
toolArgs: { query: "…" }, // for tool spans
});
await span.end({
output: result,
inputTokens: 100,
outputTokens: 200,
cost: 0.003,
toolResult: { …: … }, // for tool spans
});
await span.event("decision", { choice: "retry" }, "info");Source location (file, line, function) is captured automatically.
const state = await tl.breakpoint({
label: "post-retrieval",
state: { docs, query },
timeoutMs: 15 * 60_000, // default: 15 minutes
});
// If the dashboard edited `state`, the returned value reflects the edit.See docs/breakpoints.md.
The SDK walks the stack on every trace.span(...) call and captures the
first frame outside @pathlight/sdk and node_modules. The file, line,
column, and function name get stored in span.metadata._source and render
as a clickable breadcrumb in the dashboard.
This means zero instrumentation config — the waterfall always knows exactly where in your code each step was created.
The collector enriches every trace list response with per-row issue flags:
- Span failed —
span.status === "failed" - Span error —
span.erroris set - Suspicious output — regex matches
\bfail\b,failed,error,exception,timeout,invalid,denied,refused,rejected,incomplete,truncat
Traces with any issue get an amber Issues detected badge on the list and individual amber-highlighted rows in the waterfall.
| Variable | Default | Purpose |
|---|---|---|
DATABASE_URL |
file:pathlight.db |
SQLite or Turso URL |
DATABASE_AUTH_TOKEN |
— | Turso auth token |
PORT |
4100 |
Collector port |
PATHLIGHT_HOST |
127.0.0.1 |
Collector bind host. Docker sets 0.0.0.0; local npm runs stay localhost-only by default. |
PATHLIGHT_ACCESS_TOKEN |
— | Optional shared token required for /v1/* collector routes. Set this before exposing the collector beyond localhost. |
NEXT_PUBLIC_COLLECTOR_URL |
http://localhost:4100 |
Collector URL (browser) |
NEXT_PUBLIC_PATHLIGHT_ACCESS_TOKEN |
— | Dashboard build-time token sent to collectors protected by PATHLIGHT_ACCESS_TOKEN. |
PATHLIGHT_URL |
http://localhost:4100 |
Collector URL for CLI (pathlight, pathlight-eval) |
REPLAY_API_KEY |
— | Generic server-side key for /v1/replay/llm — works with OpenAI, Anthropic, or any OpenAI-compatible gateway |
REPLAY_BASE_URL |
— | Base URL for OpenAI-compatible replay (e.g. https://gateway.provara.xyz). Accepts with or without trailing /v1 |
OPENAI_API_KEY |
— | Fallback key when REPLAY_API_KEY isn't set and provider is OpenAI-compatible |
ANTHROPIC_API_KEY |
— | Fallback key when REPLAY_API_KEY isn't set and provider is Anthropic |
PATHLIGHT_SEAL_KEY |
— | 32-byte base64 master key for the BYOK encrypted key store. When set, the collector mounts /v1/projects/:id/keys. Fail-stops on malformed input. |
PATHLIGHT_FIX_APPLY_ROOTS |
— | Optional comma-separated allowlist of directories where /v1/fix-apply may run git apply. Unset preserves local-dev behavior; set it before exposing the collector. |
PATHLIGHT_LLM_API_KEY |
— | BYOK LLM key consumed by pathlight fix. Required for the CLI. |
PATHLIGHT_GIT_TOKEN |
— | Read-only git token consumed by pathlight fix --git-url. Never logged. |
# Docker (recommended)
docker compose up -d # Pull + start everything
docker compose down # Stop (data preserved in volume)
docker compose down -v # Stop + wipe data
docker compose pull && docker compose up -d # Upgrade to latest images
# Local dev (monorepo)
npx turbo dev # Collector + web together
npx turbo build # Build all packages
npx turbo test # Run the full test suite
npm run dev -w packages/collector # Collector only (4100)
npm run dev -w apps/web # Web only (3100)
# Database (local dev — Docker handles migrations automatically)
npm run db:generate -w packages/db # Generate Drizzle migration
DATABASE_URL="file:$(pwd)/packages/collector/pathlight.db" \
npm run db:migrate -w packages/db # Apply migrations
npm run db:studio -w packages/db # Open Drizzle Studio
npm run db:retire -w packages/db -- ../collector/pathlight.db # Archive
npm run db:retire -w packages/db -- --delete ../collector/pathlight.db # Delete
# CLIs (after install)
pathlight share <trace-id> --out report.html
pathlight-eval specs/my-checks.mjs --base-url http://localhost:4100
# Code-fixing agent (BYOK)
export PATHLIGHT_LLM_API_KEY=sk-ant-...
pathlight fix <trace-id> --source-dir . --apply
pathlight fix <trace-id> --git-url https://github.com/acme/repo.git --provider openai
pathlight fix <trace-id> --bisect --from <good> --to <bad> --git-url <url>- Collector: Hono — lightweight, fast
- Dashboard: Next.js 15 + Tailwind CSS
- Database: SQLite via Drizzle ORM (libSQL-compatible; Turso-ready)
- Monorepo: Turborepo + npm workspaces
- Streaming: Server-sent events (native
EventSource+hono/streaming)
Everything runs locally by default. SQLite file, single collector process,
no external services required. The collector binds to 127.0.0.1 for local
npm runs. If you bind it outside localhost, set PATHLIGHT_ACCESS_TOKEN and
pass the same value to SDKs or the dashboard build. API keys (for replay and
BYOK fix) read from env vars, browser session state, or the encrypted
libsodium-backed key store — never sent to any third-party beyond the LLM
provider itself.
/v1/fix-apply writes patches on the collector host. For personal local
development this is intentional; for shared or exposed collectors, set
PATHLIGHT_FIX_APPLY_ROOTS=/path/to/workspaces,/another/root so patches can
only apply inside approved working-tree roots.
BYOK is a hard line. The code-fixing agent, LLM replay, and OpenClaw plugin all use the user's own keys. Pathlight never acts as an inference proxy and never stores plaintext keys at rest.
MIT