Skip to content

Commit a2d7f46

Browse files
skill-mandate: count Agent alongside Task; stop treating heredoc/$VAR noise as writes
Dogfooded against a real 15MB transcript with 70 Agent tool_use dispatches: task_count only ever matched "Task" (this build's SDK names the dispatch tool "Agent"), so the delegation mandate reported "zero subagents" over 70 real ones, and the Task-gated agent-naming mandate was structurally unable to ever fire. Now counts Task or Agent (not TaskCreate, confirmed to be a todo-tracker item shape, not a dispatch call). The Bash-write extraction added in v1.36.0 was over-matching by an order of magnitude: unexpanded $VAR redirect targets (fixture literals, never real files) and write-shaped lines inside inert heredoc bodies (`cat > script.sh <<EOF ... EOF`, where body lines describing a write are the generated file's content, not this command's own action) were both counted as real writes, measured at 53 directories / 83 extensions for a session that wrote one real file. emit() now drops any $-containing candidate, and heredoc bodies are skipped only when the same line already matched a write rule (line_had_write) -- which is what keeps an executed `python3 - <<PY` heredoc's real open(...,'w') write from being suppressed too, the regression the first pass of this fix introduced and PROOF 5 caught. tests/test-breadth-mandate.sh: PROOF 10 (Agent dispatch suppresses the breadth mandate), PROOF 11 (zero Task AND zero Agent still trips it), PROOF 12 ($VAR-containing Bash write target is not counted). All 12 proofs pass; falsified by hand against the pre-fix hook (146a395), restored byte-identical (sha256 68fa3e69...). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent a1ba1f3 commit a2d7f46

2 files changed

Lines changed: 185 additions & 21 deletions

File tree

claude/hooks/skill-mandate.sh

Lines changed: 118 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -114,11 +114,30 @@ skills=$(
114114
# mandate name a Bash line it otherwise would not have, never suppress one that is real
115115
# - the same trap in the mirror: `[[ "$x" > "$y" ]]` and `[ "$a" > "$b" ]` are lexicographic
116116
# string comparisons, not redirects, and the redirect scan cannot tell the difference -- it
117-
# reads `$y`/`$b` off as a write target. Checked before shipping this: the fallout is bounded
118-
# to inert noise, because a bare `$var` has no `/` (parent dir collapses to the same `.`
119-
# every extension-less token already collapses to) and no `.` (no extension token is emitted
120-
# at all), so it cannot supply the second directory or the second extension the conjunctive
121-
# threshold needs on its own -- confirmed by hand, not assumed
117+
# reads `$y`/`$b` off as a write target. This used to be dismissed here as inert noise on the
118+
# theory that a bare `$var` has neither `/` nor `.` to supply a second directory or extension
119+
# on its own. That theory was wrong, measured against a real 15MB transcript: combined with
120+
# heredoc-body leakage (below), unexpanded-`$` candidates were not inert, they were the
121+
# dominant term behind a reported 53 directories / 83 extensions for a session that had
122+
# written one real file. `emit()` now drops any candidate containing `$` outright rather
123+
# than trusting it to stay harmless -- cheap, and it removes this whole family in one place
124+
# regardless of which rule produced the candidate.
125+
# - a line inside an open heredoc body is skipped entirely (tracked by the in_hd/hd_delim state
126+
# machine in emit()'s caller), so `cat > new-check.sh <<'EOF'` counts only new-check.sh, not
127+
# every `>`/`cp`/`mv`/`open()`-shaped line the generated script's own body happens to contain.
128+
# Before this, a test-fixture generator that wrote fixture code containing its own example
129+
# `printf ... > path` was read as this session performing that write too -- the other half of
130+
# the 53/83 measurement above. Residual within this fix: two heredocs opened on the same
131+
# physical line, or a delimiter line that also carries trailing content after it, are not
132+
# modeled -- both are rare enough in practice that chasing them was not worth it here.
133+
# - a same-line `>` used as a comparison/arithmetic/format-string character *outside* a heredoc
134+
# body -- `awk '{if (a>b) ...}'`, a `printf` format token like `%.3f` that happens to follow a
135+
# stray `>` earlier on the line -- is still misread as a redirect target exactly as before.
136+
# Suppressing heredoc bodies removed most of this family's real-world volume (it is what most
137+
# of these lines were embedded in), and the survivors are short, extension-poor, non-slash
138+
# tokens (`9.9`, `1{print`, `>>>>>`) that read the same way the existing extension-less-token
139+
# analysis above already argues is bounded -- left as a known, disclosed miss rather than
140+
# chased with a real shell tokenizer, which this file has never tried to be.
122141
# None of this is a shell parser. It is a best-effort net over write shapes that are actually
123142
# common in this repo's own history, traded deliberately against reads: `grep`, `cat`, `ls`,
124143
# `git add`, `find` without `-delete`, and command substitution never match any rule below, so
@@ -133,52 +152,119 @@ if [ -n "$AWK_BIN" ]; then
133152
| select(.type=="tool_use" and .name=="Bash") | .input.command // empty' "$tr_" 2>/dev/null
134153
)
135154
if [ -n "$bash_cmds" ]; then
155+
# emit() is the one gate every extracted candidate passes through before it can become a
156+
# path: empty, and -- new -- anything still carrying an unexpanded shell variable. A path
157+
# like $g_empty/app/src/C.tsx never resolved to a real file; it is a fixture literal (this
158+
# repo writes exactly that shape in its own test-generator scripts) or a template a later
159+
# command substitutes into, and either way this hook has no value to substitute it with, so
160+
# counting it as a directory/extension is pure noise, not a conservative guess.
161+
#
162+
# Heredoc-body suppression is the second half. `cat > new-check.sh <<'CHK' ... CHK` writes
163+
# exactly one real file -- new-check.sh, captured by the `>` rule on the opening line, same
164+
# as always -- but every line *inside* that heredoc's body is script source being written
165+
# to disk, not a command running in this turn. Before this fix the same `>>?` / `cp`/`mv`
166+
# regexes ran over body lines too, so a generated test fixture that itself contained
167+
# `printf ... > "$g_empty/app/src/C.tsx"` or `mv .github/workflows/verify.yml /tmp/...`
168+
# inside its own body text was read as a second, third, fourth real write by *this*
169+
# session -- confirmed against a real 15MB transcript, where it was the dominant term
170+
# behind a reported 53 directories / 83 extensions that should have been ~1. The state
171+
# machine below tracks whether the current physical line is inside an open heredoc body
172+
# (in_hd) and skips all extraction there, resuming only once a line equal to the opening
173+
# delimiter (optionally indented, for `<<-`) is seen. It does not attempt to track nested
174+
# heredocs inside a body -- while in_hd, every line is skipped outright, including one that
175+
# looks like it opens another heredoc, which is correct: it is body text either way, and the
176+
# only delimiter that matters is the outer one already being waited for.
177+
#
178+
# A here-string (`cmd <<<"$x"`) does not falsely open heredoc-suppression: after consuming
179+
# `<<` plus an optional `-` plus optional spaces, the next character is still the third `<`,
180+
# which matches neither the stripped-quote branch nor a bareword delimiter, so the match
181+
# fails and in_hd is never set. A bitshift (`$((1 << 3))`) fails the same way -- the char
182+
# after `<<` is a digit, not `[A-Za-z_]`. Checked by hand against both shapes before shipping.
183+
#
184+
# Suppression only engages when the SAME physical line also matched one of the write rules
185+
# above (line_had_write) -- this is not incidental, it is the line separating an inert
186+
# heredoc from an executed one. `cat > file <<EOF` and `tee file <<EOF` both write the body
187+
# to a file verbatim; the body never runs, so any `>`/`cp`/`open()`-shaped text inside it
188+
# describes what the generated file contains, not what this Bash call did. `python3 - <<PY`
189+
# and `bash <<EOF`, by contrast, pipe the body to an interpreter that executes it immediately
190+
# in this same tool call -- neither line matches any write rule on its own (no redirect, no
191+
# cp/mv, nothing), so line_had_write is 0 and the body is scanned normally, meaning
192+
# `open("config/c.json", "w")` inside that body is still counted, correctly, as a real write.
193+
# Gating on line_had_write rather than on the presence of `<<` alone is what tells these two
194+
# shapes apart without knowing what command is on the line -- the first version of this fix
195+
# suppressed both alike and silently broke the one fixture (test-breadth-mandate.sh PROOF 5)
196+
# that depends on a `python3 - <<PY` heredoc's write being seen.
136197
bash_write_extract='
198+
function emit(f) {
199+
if (f == "" || f ~ /\$/) return
200+
print f
201+
}
202+
BEGIN { in_hd = 0; hd_delim = "" }
137203
{
138204
line = $0
205+
line_had_write = 0
206+
if (in_hd) {
207+
d = line
208+
gsub(/^[ \t]+/, "", d)
209+
if (d == hd_delim) { in_hd = 0; hd_delim = "" }
210+
next
211+
}
139212
if (match(line, /(^|[;&|]| )sed[ \t]+-i[^ \t]*[ \t]+/)) {
213+
line_had_write = 1
140214
rest = substr(line, RSTART + RLENGTH)
141215
n = split(rest, toks, /[ \t]+/)
142216
if (n >= 1) {
143217
f = toks[n]
144218
gsub(/^["'"'"']|["'"'"';]+$/, "", f)
145-
if (f !~ /^-/ && f != "") print f
219+
if (f !~ /^-/) emit(f)
146220
}
147221
}
148222
work = line
149223
while (match(work, />>?[ \t]*[^ \t;&|)]+/)) {
224+
line_had_write = 1
150225
tgt = substr(work, RSTART, RLENGTH)
151226
work = substr(work, RSTART + RLENGTH)
152227
sub(/^>>?[ \t]*/, "", tgt)
153228
gsub(/^["'"'"']|["'"'"']$/, "", tgt)
154-
if (tgt !~ /^&/ && tgt !~ /^\/dev\//) print tgt
229+
if (tgt !~ /^&/ && tgt !~ /^\/dev\//) emit(tgt)
155230
}
156231
if (match(line, /(^|[;&|]| )tee[ \t]+(-a[ \t]+)?/)) {
232+
line_had_write = 1
157233
rest = substr(line, RSTART + RLENGTH)
158234
n = split(rest, toks, /[ \t]+/)
159235
for (i = 1; i <= n; i++) {
160236
t = toks[i]
161237
if (t ~ /^-/) continue
162238
gsub(/^["'"'"']|["'"'"']$/, "", t)
163-
if (t != "") print t
239+
emit(t)
164240
break
165241
}
166242
}
167243
if (match(line, /(^|[;&|]| )(cp|mv)[ \t]+/)) {
244+
line_had_write = 1
168245
rest = substr(line, RSTART + RLENGTH)
169246
n = split(rest, toks, /[ \t]+/)
170247
if (n >= 2) {
171248
f = toks[n]
172249
gsub(/^["'"'"']|["'"'"']$/, "", f)
173-
if (f !~ /^-/ && f != "") print f
250+
if (f !~ /^-/) emit(f)
174251
}
175252
}
176253
s = line
177254
while (match(s, /open\([ \t]*["'"'"'][^"'"'"']+["'"'"'][ \t]*,[ \t]*["'"'"'][waxWAX][^"'"'"']*["'"'"']/)) {
255+
line_had_write = 1
178256
m = substr(s, RSTART, RLENGTH)
179257
s = substr(s, RSTART + RLENGTH)
180258
if (match(m, /["'"'"'][^"'"'"']+["'"'"']/)) {
181-
print substr(m, RSTART + 1, RLENGTH - 2)
259+
emit(substr(m, RSTART + 1, RLENGTH - 2))
260+
}
261+
}
262+
if (line_had_write && match(line, /<<-?[ \t]*/)) {
263+
rest2 = substr(line, RSTART + RLENGTH)
264+
gsub(/^["'"'"']/, "", rest2)
265+
if (match(rest2, /^[A-Za-z_][A-Za-z0-9_]*/)) {
266+
hd_delim = substr(rest2, RSTART, RLENGTH)
267+
in_hd = 1
182268
}
183269
}
184270
}
@@ -229,10 +315,26 @@ fi
229315
#
230316
# Dotfile handling: a file starting with . and containing no further . has no extension
231317
# (it is pure name, not name + type). .eslintrc.json yields json; .eslintrc yields nothing.
318+
#
319+
# Dispatch-tool name: the subagent-launch tool is called "Task" in the classic Claude Code CLI
320+
# and "Agent" in the Claude Agent SDK build this hook was actually dogfooded against -- a real
321+
# 15MB transcript logged 70 "Agent" tool_use blocks and zero "Task" ones, which meant this
322+
# counter read 0 and both the delegation mandate and the agent-naming mandate below misfired on
323+
# every session running that build: the former claimed "zero subagents" over 70 of them, the
324+
# latter (gated on task_count>=1) was structurally unable to ever fire. Counting both names is
325+
# not future-proofed against a third rename; a build using neither counts 0 and this hook falls
326+
# back to its existing bias (say nothing rather than guess) rather than accusing a session that
327+
# genuinely delegated under a name this file does not yet know.
328+
#
329+
# "TaskCreate" is deliberately NOT included here. It exists in this same build and looked like a
330+
# third dispatch-tool alias at a glance, but its recorded .input is {subject, description,
331+
# activeForm} -- a todo/checklist item, the same shape as TodoWrite -- not {prompt,
332+
# subagent_type} like Task/Agent. Counting it would credit a session for delegating work it only
333+
# planned. Confirmed against 3 real transcripts on this machine before excluding it, not assumed.
232334
task_count=$( "$JQ" -s '[.[] | select(.type=="assistant") | .message.content[]?
233-
| select(.type=="tool_use" and .name=="Task")] | length' "$tr_" 2>/dev/null )
335+
| select(.type=="tool_use" and (.name=="Task" or .name=="Agent"))] | length' "$tr_" 2>/dev/null )
234336

235-
# Agent naming: if Task count >= 1, one of the roster must appear in assistant text.
337+
# Agent naming: if Task/Agent count >= 1, one of the roster must appear in assistant text.
236338
# Extract all assistant message text.
237339
if [ "$task_count" -ge 1 ]; then
238340
assistant_text=$( "$JQ" -r 'select(.type=="assistant") | .message.content[]?
@@ -265,7 +367,7 @@ extensions=$( printf '%s\n' "$paths" | sed -E '
265367
ext_count=$( [ -z "$extensions" ] && echo 0 || printf '%s\n' "$extensions" | grep -c . )
266368
if [ "$dir_count" -ge 3 ] && [ "$ext_count" -ge 2 ] && [ "$task_count" -eq 0 ]; then
267369
unmet="$unmet
268-
multi-directory work -- touched $dir_count directories with $ext_count file types, zero subagents (try /team, or TaskCreate: code-reviewer, qa, worker, planner, test-writer)"
370+
multi-directory work -- touched $dir_count directories with $ext_count file types, zero subagents (try /team, or dispatch one directly: code-reviewer, qa, worker, planner, test-writer)"
269371
fi
270372

271373
# Prove it works: a completion claim closing this turn with zero evidence produced in it.
@@ -304,7 +406,7 @@ fi
304406
# a completion claim is conversational, not a claim about code, and the false-block
305407
# cost of guessing otherwise is worse than the miss (see the false-positive note by
306408
# the pattern list below).
307-
# silent -- ANY Bash, Read, or Task tool_use anywhere in the turn, in any order relative to
409+
# silent -- ANY Bash, Read, or Task/Agent tool_use anywhere in the turn, in any order relative to
308410
# the edit. This is deliberately generous in both directions: a Bash call is treated
309411
# as evidence whether or not it happens to be a test invocation, and a Read is
310412
# treated as evidence even if it came before the edit (i.e. was investigation of the
@@ -320,7 +422,7 @@ fi
320422
# Known false-positive shape, disclosed rather than chased: a turn that Writes a genuinely
321423
# unverifiable artifact -- a poem, a note, a scratch file with no "works" to check -- and closes
322424
# with a plain "Done." trips this exactly like an unverified code fix would, because file-write
323-
# plus closing "done" plus no Bash/Read/Task in the same turn is indistinguishable from here.
425+
# plus closing "done" plus no Bash/Read/Task/Agent in the same turn is indistinguishable from here.
324426
# Scoping the edit check to code-like extensions was considered and rejected: it would have
325427
# meant maintaining an extension allowlist this hook has no way to keep current, trading one
326428
# false-positive shape for a false-negative one (a `.sh` fix that never runs is exactly the case
@@ -379,7 +481,7 @@ if [ -n "$piw_final_text" ] && printf '%s' "$piw_final_text" | grep -qiE "$piw_p
379481
if [ "$piw_edit_n" -ge 1 ] && [ "$piw_claims" -eq 1 ] \
380482
&& [ "$piw_bash_n" -eq 0 ] && [ "$piw_read_n" -eq 0 ] && [ "$piw_task_n" -eq 0 ]; then
381483
unmet="$unmet
382-
prove-it-works -- this turn edited a file and closed claiming it is done, with no Bash/Read/Task call in the turn to back it up"
484+
prove-it-works -- this turn edited a file and closed claiming it is done, with no Bash/Read/Task/Agent call in the turn to back it up"
383485
fi
384486

385487
[ -n "$unmet" ] || { rm -f "$cnt_file"; exit 0; }

0 commit comments

Comments
 (0)