WEB-4507: Capture Claude Code skill & agent invocations from hooks#99
Open
anonpran wants to merge 3 commits into
Open
WEB-4507: Capture Claude Code skill & agent invocations from hooks#99anonpran wants to merge 3 commits into
anonpran wants to merge 3 commits into
Conversation
Detect skill and agent/subagent invocations during agentic operations and send them to the backend (analytics later consume this). - parse_transcript_file: capture Skill + Task/Agent tool_use blocks from the session transcript (tool_use_id, cwd, gitBranch). PostToolUse is unreliable for these, so the transcript is the source of truth. - collect_subagent_skill_tool_uses: sweep <session>/subagents/*.jsonl so skills/agents invoked inside subagents (separate transcripts the parent Stop never reads) are captured too, flagged is_subagent. - build_llm_exchange: forward transcript tool_uses, skip the duplicate PostToolUse copies, and tag trigger (namespace-aware user-typed /skill vs agent auto-invoke) + is_subagent. tool_use_id enables downstream dedup. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- build_llm_exchange trigger: replace pure-suffix match with full / bare-vs- namespaced (either direction) exact comparisons, so two skills sharing a base name across namespaces (ns1:deploy vs ns2:deploy) no longer mislabel an agent invocation as user-typed. - parse_transcript_file + collect_subagent_skill_tool_uses: only capture tool_use blocks that carry an id, so a row can always dedup on tool_use_id (avoids None ids that the partial unique index would skip). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Condensed the multi-line explanatory comments added for skill/agent capture to concise one-liners. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Phase 1 of WEB-4507 (Skills application): detect Claude Code skill and agent/subagent invocations during agentic operations and send them to the backend. Analytics surfacing and policy enforcement are separate later phases.
Changes (
claude-code/hooks/unbound.py)parse_transcript_file: captureSkill+Task/Agenttool_useblocks from the session transcript, withtool_use_id,cwd,gitBranch.PostToolUsedoes not reliably fire for these, so the transcript is the source of truth.collect_subagent_skill_tool_uses(new): sweep<session>/subagents/*.jsonl— subagents write to separate transcripts the parentStopnever reads — so skills/agents invoked inside a subagent are captured too (flaggedis_subagent).build_llm_exchange: forward transcript tool_uses, skip the duplicatePostToolUsecopies (they lack a reliable id), and tag each withtrigger(namespace-aware: user-typed/skillvs agent auto-invoke) andis_subagent.tool_use_idenables downstream dedup.Testing
Verified e2e against real session transcripts: Skill + Agent capture, subagent sweep (a genuinely-missed subagent skill now captured), and namespace-aware trigger (
/stripe:test-cards→user;/buildervsbuildno false-positive).Notes
ai-gateway-dataPR (backend converterskill/agentbranches + idempotent dedup).main(this repo has nostaging).🤖 Generated with Claude Code
Greptile Summary
This PR adds Phase 1 of skill/agent invocation capture:
parse_transcript_filenow recordsSkill,Task, andAgenttool_useblocks from the parent session transcript, and a newcollect_subagent_skill_tool_useshelper sweeps subagent sidecar transcripts that the parentStopevent never sees.build_llm_exchangeskips the id-lessPostToolUseduplicates for these tool types and instead enriches each transcript-sourced entry withtrigger(uservsagent) andis_subagentmetadata before forwarding to the backend.parse_transcript_file— adds anelifbranch inside theassistantcontent loop that collectsSkill/Task/Agenttool_useitems withtool_use_id,cwd, andgit_branch; entries without anidare skipped so every row landing in the backend carries a dedup key.collect_subagent_skill_tool_uses— new function that globs{session_dir}/subagents/*.jsonl, applies the same timestamp filter and id guard, and flags all hitsis_subagent: True; results are merged intotranscript_tool_usesinprocess_stop_event.build_llm_exchange— filters outPostToolUseevents for the three tool names, then appends the richer transcript-sourced entries, resolvingtriggervia a three-way namespace-aware comparison (full match, bare vs namespaced, or namespaced vs bare).Confidence Score: 5/5
Safe to merge — the change is additive (new collection path + enriched metadata), guarded throughout with try/except, and both previous blocking concerns (null tool_use_id and namespace false-positive) are addressed in the current code.
All three new code paths (parent transcript capture, subagent sweep, trigger tagging) have appropriate guards: the content_item.get('id') requirement ensures no entry reaches the backend without a dedup key, os.path.isdir silently skips sessions with no subagent directory, and individual subagent file failures are caught and continued. The trigger detection uses three exact comparisons that correctly distinguish namespaced from bare skill names without the original suffix-only false positive.
No files require special attention; the single changed file is well-contained and the logic is straightforward.
Important Files Changed
Sequence Diagram
sequenceDiagram participant CC as Claude Code (Stop event) participant PSE as process_stop_event participant PTF as parse_transcript_file participant CSA as collect_subagent_skill_tool_uses participant BLE as build_llm_exchange participant API as Unbound API CC->>PSE: Stop event (transcript_path, session_id) PSE->>PTF: parse parent transcript (user_prompt_timestamp) PTF-->>PSE: "transcript_data {tool_uses, messages, usage, model}" PSE->>CSA: sweep subagents dir (transcript_path, user_prompt_timestamp) CSA-->>PSE: "subagent tool_uses (is_subagent=True)" PSE->>PSE: merge → transcript_tool_uses PSE->>BLE: build_llm_exchange(events, transcript_tool_uses, ...) BLE->>BLE: skip PostToolUse Skill/Task/Agent (no stable id) BLE->>BLE: tag each transcript tool_use with trigger + is_subagent BLE-->>PSE: exchange dict PSE->>API: POST /v1/hooks/claude (exchange)Reviews (3): Last reviewed commit: "WEB-4507: trim verbose comments (no func..." | Re-trigger Greptile