Distinguish human-typed prompts from tool results in Claude history by tony · Pull Request #68 · tony/agentgrep

tony · 2026-06-14T13:06:48Z

Summary

Fix the Claude Code adapter so prompt search no longer treats tool results and subagent output as human-typed prompts. Claude records tool_result blocks, <local-command-stdout>, the [Request interrupted] marker, and subagent transcripts as type=user messages, so a user-role turn is not reliably a typed prompt.
Add claude_event_is_human_authored(), which classifies a Claude project JSONL event by structure (tool_result/tool_use blocks, the toolUseResult marker, isSidechain, non-user event types, and machine-authored string content).
Thread the verdict through parse_claude_project_file into build_search_record(human_typed=...), which sets metadata["human_typed"]=False only for non-human turns.

This is additive: a normal record's metadata stays empty, so existing consumers and tests are unchanged — the new signal is opt-in for anything (search ranking, analytics, the in-progress insights work) that wants to filter to the user's actual asks.

On a real history sample, 397 of 436 user-role records were tool/sidechain noise rather than typed prompts; the tag makes that separable.

Test plan

tests/test_claude_human_typed.py — parametrized cases for string prompts, text-block prompts, tool_result/tool_use blocks, isSidechain, assistant events, <local-command-stdout>, and the list-form [Request interrupted] marker, plus the additive-metadata behavior of build_search_record.
ruff check, ruff format, ty check clean.
Existing Claude parser tests pass (the change is additive).

Goal

Idempotent completion condition for /goal. It asserts an end state, not an action: if the state already holds, the first evaluation passes and nothing is edited.

/goal Work only in the existing worktree for local branch agentgrep-human-typed-prompts, never in the shared master tree; never push, and no commit, PR or merge is part of this condition. FIRST report there: git rev-parse --abbrev-ref HEAD; git merge-base --is-ancestor master HEAD && echo ANCESTOR-OK || echo NEEDS-REBASE, which prints NEEDS-REBASE today against the local master ref; rg -n human docs/library/query-language.md, which matches nothing today while that page says the registry ships sixteen fields; and NO_COLOR=1 env -u VIRTUAL_ENV -u UV_NO_SYNC .venv/bin/python -m pytest -m "" --reruns 0 -q, moving any untracked scripts/benchmark.local.toml aside first. End state: HEAD is that branch, ANCESTOR-OK prints, that page carries a human:true|false row in the record-level field table with the field count corrected, agentgrep.query.help.query_language_fields reflects the field, pytest exits 0, and uv run ty check, uv run ruff check . and just build-docs succeed there. With the field omitted, search, grep, find, CLI JSON/NDJSON and MCP results match master because metadata stays empty for human-authored turns, and an agcur1: cursor minted before this change still parses with an unchanged payload version. Constraints: keep the signal additive - an absent metadata['human_typed'] means human-authored, and nothing may treat it as an identity, authorship-provenance or dedupe contract that #80 or #142 would have to break. Do not retire, re-version or re-encode agcur1:, and do not change offset or limit semantics; cursor evolution is separate work. Do not extend the classifier to agents the branch does not already cover, and do not filter human-authored turns by default anywhere. Or stop after 10 turns and report what remains.

Proof — git rev-parse --abbrev-ref HEAD && git merge-base --is-ancestor master HEAD && rg -n human docs/library/query-language.md && NO_COLOR=1 env -u VIRTUAL_ENV -u UV_NO_SYNC .venv/bin/python -m pytest -m "" --reruns 0 -q (run in that worktree)
Idempotence — Re-running when the branch already contains local master and the human row and corrected count are already on the page reproduces the same four reports and edits nothing.
Blocked by — nothing
Blocks — nothing hard; Insights graph similarity engine + non-gated transformers LLM defaults #69 may bundle these commits or drop them behind a capability check
Preserves — metadata['human_typed'] stays an additive opt-in signal, not an identity or provenance contract, so Deterministic IDs for conversations / prompts #80's content id and Durable prompt corpus and derived search indexes (ADR 0019) #142's corpus keys stay free; agcur1: keeps its payload version and offset/limit semantics, leaving cursor retirement open; default field-omitted behaviour is byte-identical to master, so no shipped result set moves.

why: PR #68 separated Claude's typed prompts from tool results, but the same flattening happens in Codex (function_call_output) and Grok (tool_use/tool_result), and the human/tool tag was only reachable from the graph engine. Tagging every adapter that carries inline tool-output and exposing it as a `human:` query field lets `agentgrep search`/`grep` and the MCP server filter the user's real asks from tool noise. what: - Add codex_event_is_human_authored() and tag Codex response_item records; function_call/function_call_output/reasoning are non-human. - Tag Grok tool_use/tool_result turns with metadata["human_typed"]=False. - Register a record-layer `human` enum field (true/false) and dispatch it in the query compiler: human:true keeps untagged turns, human:false selects tool/agent output. - Cover the Codex detector, Grok tagging, and the human: predicate.

…ted transformers LLM defaults why: The insights ladder ended at the L5 narrative summary over a single gated Gemma model. Users had no way to see which prompts they repeat, which past conversations resemble the one in front of them, or which workflows are worth saving as Skills — and the only GPU summary path required an HF token plus an accepted license. This adds the `graph` enrichment level and makes the local-LLM backend work out of the box. what: - Add the `graph` level: a prompt/reply/conversation similarity network (sentence-transformers or model2vec embeddings, optional HDBSCAN archetype clustering, sqlite-vec or LanceDB IVF-PQ vector store) that surfaces recurring asks, forgotten-but-similar conversations, and mined workflows, persisted incrementally in a content-hash-keyed graph store. - Draft reusable Skills (SKILL.md) from mined workflows, print-by-default, also exposed over an `insights_skills` MCP tool. - Add a transformers/CUDA LLM backend with a non-gated default chain that needs no HF token: Phi-4-mini (4-bit, native phi3), SmolLM2-1.7B (fp16), Granite-3.3-2b (4-bit), tried in order until one loads. gemma-3-1b-it stays curated but gated and is no longer the default. 4-bit weights load through bitsandbytes NF4 behind the insights-llm-transformers-quant extra. - Add optional conversation-summary vectors: each conversation is embedded by a cached LLM one-line summary instead of a prompt mean, sharpening forgotten-but-similar. - Bundle the #68 human-typed prompt detection (human: query field, Claude/Codex authored-turn tagging) so the branch hand-tests self-contained; it overlaps with that PR and should be reconciled at merge time.