Add TraceRoot Codex observability plugin (v0.1.0)#1
Merged
Conversation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…-flush - README + plugin.json: remove current "subagents" tracing claim (not implemented in v0.1.0); add honest note that subagent-subtree tracing is planned. SubagentStop hook registration unchanged. - src/emit.ts: collect emittedIds during the loop; call markSpanEmitted for each id AFTER tracing.shutdown() completes so a span is persisted to the sidecar only after delivery is confirmed. If shutdown throws, marking is skipped and spans will retry on the next hook invocation. - src/config.ts: guard maxChars against NaN (non-numeric env/JSON value) by falling back to 20000 via Number.isFinite check. All 37 tests pass. Typecheck clean. dist/index.mjs rebuilt. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Move src/, test/, hooks/, .codex-plugin/, tsdown.config.ts, and dist/ into plugins/tracing/ to match the marketplace-monorepo structure. Add .agents/plugins/marketplace.json, pnpm-workspace.yaml, LICENSE (Apache-2.0), and a minimal .github/workflows/ci.yml. Update root package.json build/lint:dist scripts, tsconfig.json include paths, and vitest.config.ts to reflect the new layout. Rebuild bundle at its new path (plugins/tracing/dist/index.mjs). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…d events Captures exec kind, completion status, exit code, and error detail from Codex exec_command_end (and other *_end) event_msg events, surfacing them as structured fields on ToolCall and in traceroot.span.metadata on TOOL spans. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- planTurnSpans: add optional PlanOpts (seedPrefix/traceId/rootParentSpanId); seed="" default keeps all existing span ids byte-identical (39 existing tests unchanged).
- types: add SubagentRef + Turn.subagents optional field; subagentThreadIds kept for backward compat.
- transcript: capture {threadId, spawnCallId} from collab_agent_spawn_end into turn.subagents.
- subagents: new findSubagentRollout scans sessions/YYYY/MM/DD/ for rollout-*-<threadId>.jsonl.
- emit: recursive emitTurnTree with visited-set (cycle/diamond safe) + STACK_GUARD=32 paranoia; child spans share parent traceId, nested under spawn TOOL span via seedPrefix.
- tests: 18 new tests (planTurnSpans opts, findSubagentRollout, 5 subagent emit assertions incl. idempotency + cycle guard). All 57 tests (39 + 18) pass, typecheck clean, dist rebuilt.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- emit: wrap per-subagent resolve/read/parse/recurse in try/catch so a throwing findSubagent/readRollout/parseSession never propagates out of dispatch (fail-open; parent spans still flush + get marked). visited.add() stays BEFORE the try so a failing/cyclic thread is not retried. Document Deps.findSubagent as should-be-fail-soft but not assumed. - types: make Turn.subagentThreadIds optional (read nowhere); transcript no longer writes it. - spans: comment that child subagent spans carry the CHILD session id and the backend groups the trace off the ROOT span; note future overrideSessionId PlanOpts as the extension. - test: add fail-soft test (throwing findSubagent → dispatch resolves, parent spans emit). 58 tests pass (57 + 1), typecheck clean, dist rebuilt. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Use @opentelemetry/exporter-trace-otlp-proto (was -http/JSON). The Traceroot ingest endpoint accepts OTLP protobuf only; JSON was silently 415'd → 0 traces. Aligns with the traceroot-ts SDK. - Parse custom_tool_call/custom_tool_call_output so apply_patch (file edits) and other custom tools are captured as TOOL spans (Codex encodes apply_patch as a custom_tool_call, not function_call). - Resolve CODEX_HOME at call time in getConfig (was module-load) so it reflects the env and is test-isolatable; config test now isolates via a temp CODEX_HOME. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
e2e against prod surfaced two issues with real multi-agent Codex sessions:
- Subagent detection: real Codex (multi_agent v1) spawns via a `spawn_agent`
function_call whose function_call_output carries {"agent_id": "<thread>"},
not the collab_agent_spawn_end event our fixtures assumed. Detect spawn_agent
and link the child thread (+ spawn call_id) so subagent spans nest under the
spawn_agent TOOL span, in the same trace.
- Export: BatchSpanProcessor instead of SimpleSpanProcessor. Simple sends one
POST per span; a burst (a turn's many tools, or a subagent subtree flushed at
once) lost the tail before the short-lived process exited — confirmed in prod
(only 1 of 3 subagents landed). Batch coalesces into few large POSTs that
forceFlush ships reliably; verified 54/54 spans + 3/3 subagents land.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codex fires hooks for subagent sessions too, so each subagent was emitted both as a standalone top-level trace AND nested under its parent's trace. Detect a subagent session (session_meta.thread_source === "subagent" / parent_thread_id) and skip standalone emission — the parent owns the nesting. Keeps one trace per top-level query with subagents nested inside. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Previously LLM (model-step) spans had no input and output only when the step
emitted assistant text — so every tool-calling step looked empty in the UI.
Now each LLM span carries:
- input: the user prompt (first step) or the prior step's tool results
- output: {content, reasoning, tool_calls:[{name,args}]} — never empty
- metadata: codex.step_index
and the root AGENT span carries codex.turn_id/thread_id/model/model_provider/
cli_version/tool_call_count. Matches the richness of the reference plugins.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The UI's SDK field reads traceroot.sdk.name/version span attributes (as the Claude Code plugin sets). We only set the OTLP scope + request headers, so the field was empty for our traces. Set them on every span via commonTrace. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The root AGENT span ("Codex Turn") was emitted only at task_complete, so during
live tracing the trace had no root yet and the platform named it after the first
span it saw (an LLM step, "gpt-5.5"), then flipped to "Codex Turn" at the end.
Now the trace root is emittable immediately and re-emitted on every hook (the
backend keeps the latest version of a span id), so the trace is named "Codex
Turn" throughout and its end/output refine as the turn progresses.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- emit.ts: pass only the findSubagent function through the recursion instead of the whole resolvedDeps struct (emitTurnTree never used buildTracing/getGit); removes the resolvedDeps assembly. Public Deps type unchanged (tests inject all). - types.ts: delete the dead `subagentThreadIds` field (unreleased, read nowhere) and the three test assignments. - sdk.ts: single-source SDK_NAME/SDK_VERSION (were duplicated in exporter.ts and spans.ts). - Verified finding 4: the SubagentStop hook is NOT a no-op — its transcript_path is the PARENT session (child under agent_transcript_path), so it re-walks the parent to nest a just-finished subagent live. Kept + documented. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…vergence - parseSession -> parseRollout (pairs with readRollout; it parses rollout lines, not a "session") - findSubagentRollout -> locateSubagentRollout These two were the only verbatim distinctive names shared with the reference plugin; the rest of the overlap is Codex's own wire-format vocabulary (RolloutLine, EventMsgPayload, ToolCall, …) or idiomatic util names, which are intentionally left as-is. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Subagents are now traced (nested under their spawn tool, same trace) — drop the "planned for a future release" note. Document live streaming + the "Codex Turn" naming, richer tool capture (exec/apply_patch/exit codes/status), the deterministic-id design, and the per-hook roles. Recommend the ~/.codex/ traceroot.json config file as the primary setup since a Codex hook subprocess does not reliably inherit the shell environment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move the full settings table into a collapsible <details>, drop the implementation-internals 'How it works' section (deterministic ids, sidecar, per-hook roles) — users need install + config, not the engineering detail.
…s-is Use the TraceRoot brand capitalization in all prose/display strings (README, manifest descriptions, author/displayName, LICENSE copyright, hook status messages, code comments). Package name, env vars (TRACEROOT_*), config files (traceroot.json), span attributes (traceroot.*), and the domain stay lowercase.
- ci.yml: drop the pinned pnpm 'version: 10' so action-setup reads the version from package.json's packageManager (was erroring on Multiple versions of pnpm). - .gitattributes: mark plugins/tracing/dist/index.mjs linguist-generated + -diff so the committed bundle (required for zero-build install) is collapsed in PR diffs and excluded from language stats. - .gitignore: add .env*, coverage/, editor dirs; note that dist/ stays committed.
Mark pnpm-lock.yaml and LICENSE as generated so they fold in the PR diff and drop out of language stats, leaving the reviewable diff as real code.
Contributor
Author
|
@cubic review it |
There was a problem hiding this comment.
4 issues found across 51 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="plugins/tracing/src/emit.ts">
<violation number="1" location="plugins/tracing/src/emit.ts:42">
P2: Nested subagent root spans are suppressed while the subagent is running, so live child tool/LLM spans can arrive without their parent AGENT span. This breaks the advertised live nested subagent view until the child turn completes.</violation>
</file>
You're on the cubic free plan with 11 free PR reviews remaining this month. Upgrade for unlimited reviews.
Re-trigger cubic
| let n = 0; | ||
|
|
||
| for (const span of planTurnSpans(sessionMeta, turn, ctx, opts)) { | ||
| if (!span.complete) continue; |
There was a problem hiding this comment.
P2: Nested subagent root spans are suppressed while the subagent is running, so live child tool/LLM spans can arrive without their parent AGENT span. This breaks the advertised live nested subagent view until the child turn completes.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At plugins/tracing/src/emit.ts, line 42:
<comment>Nested subagent root spans are suppressed while the subagent is running, so live child tool/LLM spans can arrive without their parent AGENT span. This breaks the advertised live nested subagent view until the child turn completes.</comment>
<file context>
@@ -0,0 +1,154 @@
+ let n = 0;
+
+ for (const span of planTurnSpans(sessionMeta, turn, ctx, opts)) {
+ if (!span.complete) continue;
+ // The trace root (AGENT, no parent) is re-emitted on every hook so the trace
+ // is named "Codex Turn" from the first span (not the first LLM step's model
</file context>
…ved-config exit code - config: host_url/api_key now come only from env or global ~/.codex/traceroot.json, never project-local .codex/traceroot.json — a checked-in project config could otherwise redirect authenticated uploads (env/global key) to an attacker endpoint. - sidecar: loadEmittedSpanIds fails open on any read error (not just ENOENT) — an unreadable dedup file degrades to 'nothing emitted yet' rather than aborting the trace; duplicate emits are idempotent (deterministic ids + backend upsert). - index: runHook sets process.exitCode from the resolved config so fail_on_error set via JSON (not only the env var) is honored; module handler is now a pre-config backstop. Rebuilt bundle. 66 tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the TraceRoot Codex observability plugin (v0.1.0) — a TypeScript plugin that traces OpenAI Codex sessions to TraceRoot.
What it does
Every Codex query becomes one live trace — the agent turn, model calls, tool executions, file edits, and any subagents it spawns — streamed as the agent works and linked to your git repo.
Codex Turnfrom the first span).spawn_agentworkers are nested under the spawn tool that created them, in the same trace.How it works
Stateless Codex hooks (
PostToolUse/Stop/SubagentStop) read the rollout transcript and emit OpenTelemetry (protobuf) spans. The key idea is deterministic span ids (hash of session/turn/call ids) so a child span can reference its parent before the parent exists and re-emission is idempotent — a sidecar ledger dedups what's been sent. No background process.What's captured
exec_command,apply_patch(file edits), MCP, search — with args, output, and exit code/status/kind where Codex reports ittraceroot.sdk.*)Testing & validation
tsc --noEmitclean, committed self-containeddist/index.mjswith a build drift-guard (lint:dist)Install
Enable in
~/.codex/config.tomland add~/.codex/traceroot.jsonwith your API key — see the README.🤖 Generated with Claude Code