Skip to content

Capture Claude Code native usage through the existing telemetry path - #4802

Merged
EHotwagner merged 9 commits into
mainfrom
routine/claude-native-usage-capture
Oct 22, 2026
Merged

EHotwagner merged 9 commits into
mainfrom
routine/claude-native-usage-capture

Conversation

@EHotwagner

@EHotwagner EHotwagner commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Refs #4801 (source-only Delivery; parent Outcome #4640 keeps installed adoption, Host/receiver ingestion and display)

Summary

Claude Code native usage now reaches the existing local telemetry path. It uses the same semantics as Codex and needs no new wire schema.

  • Strict transcript reader (RuntimeUsage.ClaudeTranscript, src/FS.GG.Coord.Core/Telemetry.fs{,i}). The CLI and the passive scanner share it. It refuses invalid UTF-8, comments, trailing commas and duplicate properties at any depth. Lines are capped at 4 MiB and files at 128 MiB. It keeps only allowlisted identities and counters. It tolerates other record types such as user, attachment, system and mode, and ignores <synthetic>-model local notices.
  • RuntimeUsage.collectClaudeTranscript emits one row per distinct API response, keyed by message.id, with requestId, model, effort, version and timestamp bound.
    • Per-content-block repeats are deduplicated. Only completed records (stop_reason set) count. Repeated completed usage that disagrees is refused.
    • Input is uncached + cache-read + cache-creation. reasoning is output_tokens_details.thinking_tokens and is set only when present.
    • turn_id is the promptId of the nearest preceding user record at completion. Turn totals accumulate within (thread, prompt), and thread totals stay empty.
    • Sidechain records get a derived, stable version-8 thread identity per agentId, so they never join main-thread totals.
  • Fail-closed, never zero. These all refuse with a named reason: partial tail, malformed line, oversized line or file, missing or null counters, thinking > output, missing version, no preceding prompt, mixed sessions, and an incomplete streaming response in the selection. An incomplete response also makes later turn totals in its turn unknown.
  • CLI. fsgg-coord telemetry usage collect claude takes --session-file with the Codex --turn-id/--all-responses/--since/--until contract. The existing --snapshot still works; the two options are mutually exclusive. Source provenance is claude-transcript-jsonl:sha256:<digest>.
  • Passive capture.
    • A fsgg.telemetry.passive-selection/1 file may name exactly one of codexHome or claudeHome. Claude sources must be regular files below <claudeHome>/projects, opened with the existing no-follow opener.
    • Rows are provider-usage-observation/1 with provenance claude-local-transcript/1 and stable identities in the claude-transcript/1 namespace. Coverage stays partial/unknown and attribution canonical-only.
    • The cursor keeps the prompt context, so resumed passes attribute the same way a full read does. Codex cursor bytes are unchanged.
    • The digest leaves out per-block timestamps, so identical repeats are not corrections, while disagreeing ones become numbered revisions.
    • TelemetryStore maps each admitted provenance to its own identity namespace.
  • Guidance. The pnext-item lifecycle-ledger, work-roadmap lifecycle-log and runtime-observation now name the working transcript route in place of the unwired status-line/Stop-hook recipe. Both roots are byte-identical and the driver manifest is regenerated. The Cli README is updated too.
  • Atlas. New lesson github/claude-transcript-native-usage in docs/atlas and both seeds.

Measured (one host, Claude Code 2.1.299; shape only, no transcript content committed)

  • Main-thread transcript: 151 assistant records across 78 message ids. Every record is completed, and usage is identical within each id.
  • Subagent transcript: observed at <session>/subagents/agent-<agentId>.jsonl with a .meta.json. The writer streams: per-block records carry stop_reason: null and provisional output_tokens before one completed record, and 11 of 17 ids disagreed. That is why dedupe uses completed records only.
  • Turn attribution: sequential promptId agreed with parentUuid-chain resolution for all 209 records. In subagent transcripts promptId follows the session's current prompt.
  • Local parity check (not committed): on copies of these real transcripts, CLI rows and passive records gave 86/86 (main) with 0 counter/turn/thread mismatches. The subagent gave 92 CLI rows inside the --until window against 103 passive records, with 0 mismatches.

Verification (local, at this head)

  • dotnet test tests/FS.GG.Coord.Core.Tests -c Release: 1227 passed
  • dotnet test tests/FS.GG.Coord.Cli.Tests -c Release: 750 passed (focused Claude passive tests rerun after the rotation control was added: 3 passed)
  • dotnet test tests/FS.GG.Telemetry.Tests -c Release: 118 passed
  • Causal mutation checks: removing the disagreement refusal, defaulting a missing counter to 0, or counting streaming records each fails the focused Claude tests (1, 1 and 6 failures).
  • python3 tests/telemetry-parity/run.py: pass (51 positive, 47 rejection)
  • python3 scripts/check-fsc02-telemetry-closure.py: no issues
  • bash scripts/check-skill-quality: OK (29 skills). scripts/skill-view-parity.sh: AGREE. scripts/generate-driver-manifest --check: current.
  • python3 tests/atlas/check.py --baseline 1ddc8c33…: 355 records, 674 local citations verified
  • python3 scripts/check-signature-doc-siting.py: OK
  • The skill-quality workflow step was run locally: skill-fsharp contracts/run.py, readers/run.py, adapter/run.sh, preflight/run.sh, cli/run.py and tests/skill-quality/run.sh all pass. ReadersProbe now references Core, because the passive reader shares Core's transcript decoder; its lock file is updated.

Unknown / deferred

  • Whether current status-line payloads still carry prompt_id/timestamp was not measured; --snapshot is unchanged.
  • The subagent transcript path is one observation; the readers take explicit files and assume no path convention.
  • Older transcripts without promptId, isSidechain or agentId fail closed.
  • A receiver must run a Host built from this source to admit claude-local-transcript/1. Installed adoption, Host ingestion and display stay with [V2-EFF-01.2–.6] Qualify efficiency telemetry, dashboard and selected receiver adoption #4640, and Project 3 enrollment is a separate Programme/root effect.
  • --since/--until compare timestamps ordinally, as Codex does, so …:30.000Z sorts before …:30Z.

Coherent113 source composition

The PR edits packaged kit guidance and the CLI, so it composes source 0.113.0 like #4793: FsggCoherentSetVersion, registry/dependencies.yml, regenerated version projections (architecture map, compatibility, publishing-and-deployment inventory), CLI PackageReleaseNotes, registry/skills changelogs, and the skills.yml/driver-manifest digests. Source only: published112 and installed pins are unchanged, and no candidate or publication is authorized. After merge, main's kit-published-coherence / published arm stays red until coherent113 is published, the same state as after the 0.112 source bump.

🤖 Generated with Claude Code

EHotwagner and others added 5 commits October 22, 2026 06:25
Refs #4801

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #4801

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #4801

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…obe to Core

Refs #4801

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…text

Refs #4801

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EHotwagner and others added 4 commits October 22, 2026 07:15
The PR edits packaged pnext-item/work-roadmap kit guidance and the CLI,
so kit-published-coherence pr-arm requires the coherent-set scalar to
lead published 0.112.0. Source only; publication and installed pins
remain separate, as for #4793.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Regenerate the version projections (architecture map, compatibility,
publishing-and-deployment inventory), advance CLI PackageReleaseNotes,
add the registry and skills changelog entries, and reconcile the
publishing-and-deployment digest in skills.yml and the driver manifest,
mirroring #4793. Fixes coherence/projection, architecture-map,
engine-release-notes and registry-changelog failures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keep the three-part PackageReleaseNotes structure (new 0.113.0 entry,
$(FsggStandingAdvisories), served-history pointer) required by
check-engine-release-notes, and fix a doubled registry changelog prefix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mirror #4793: SOURCE_COHERENT_VERSION, NewSddWorkspace release-notes
wording and the successor test expectations (0.112.0 joins the stale
versions). Fixes new-sdd-workspace-selftest / parse.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant