Skip to content

Commit 7c7879d

Browse files
Release v1.40.0: unlatch the delegation-drift logger, scope both instrument corpora, add the authenticated plugin harness
skill-mandate.sh's 2-strike latch sat above the delegation logger, so a session that tripped the mandate twice logged nothing for the rest of its life -- blind in exactly the long multi-directory sessions the instrument exists to measure. The checkpoint counter moves above the latch; a latched Stop now emits a latched:true row with null counts before exiting. Logging full counts there was proposed and rejected on measurement: the mandate pipeline costs 1438ms mean / 1536ms p95 on a 17.5MB transcript, so even 1-in-10 sampling would stall the end of a long session by seconds. (MEESEEKS M-4) verify.sh check 40 and gate-falsifiability row 40 drive the real hook through three synthetic Stops and assert both directions, with the row watched red under its own mutation before being made green. (BIRDPERSON B-3) Both drift instruments counted subagent sub-transcripts as independent sessions: 965 of 3292 files, and 15 of 51 replayed sessions were leaves of two parents. delegation-drift excludes them outright; compaction-effect excludes them from the primary for pooling independence and prints the excluded count. Neither instrument's numbers moved -- the leaves were already failing the single-checkpoint filter, and all 8 compact boundaries are top-level. (GLOOTIE G-5) compaction-effect crossed to a result: no signal, is_error, both trigger types. delegation-drift stays honestly NOT EVALUATED. tests/plugin-manifests.sh covers the lane throwaway containers structurally cannot, with both positive controls biting. (BETH C-3) tests/evals/build-the-lever/PREREGISTRATION.md registers thresholds before the run; Stage 0 proved the skill description reaches the model byte-identical at MODEL=sonnet. Two harness findings fell out of it: ToolSearch is an unrecorded turn sink absent from auto-trigger's fence, and denying Skill removes the listing from context entirely. (ZEEP Z-3) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 1d9c3d1 commit 7c7879d

15 files changed

Lines changed: 1021 additions & 58 deletions

.claude-plugin/marketplace.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
"plugins": [
1111
{
1212
"name": "vstack",
13-
"version": "1.39.0",
13+
"version": "1.40.0",
1414
"source": "./claude",
1515
"description": "28 skills that fire without a slash command, 14 agents, 15 commands, and the session hook that routes situations to skills. Most skills are ported from pstack and Superpowers — see claude/skills/ATTRIBUTION.md for per-skill source and license.",
1616
"category": "workflow"

.claude/verify.sh

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2231,6 +2231,141 @@ else
22312231
bad "CHANGELOG.md structure" "CHANGELOG.md is missing"
22322232
fi
22332233
2234+
# --- 40. a latched session still writes a delegation-drift row ---------------------------------
2235+
#
2236+
# The 2-strike latch in skill-mandate.sh (cnt>=2) used to sit above the delegation-drift logger,
2237+
# so once a session accumulated 2 mandate strikes every later Stop returned before logging --
2238+
# permanently blind for exactly the long, multi-directory sessions the log exists to measure.
2239+
# Check 27 exercises the mandate's block/silent decisions in both directions but never drives a
2240+
# session past the latch, so it stayed green through the entire regression.
2241+
#
2242+
# The fix keeps the latch's blocking behaviour (a mandate the model cannot satisfy must not trap
2243+
# the session) and adds one cheap row on the way out: {..., dir_count:null, ext_count:null,
2244+
# task_count:null, named:null, latched:true, ts}. This drives the real hook through three Stops
2245+
# on one synthetic session -- two that trip an unmet mandate (cnt 0->1, 1->2) and a third that
2246+
# lands on the latch (cnt>=2) -- and checks both that logging survives the latch AND that
2247+
# blocking still stops there, because a check that only asserted the former would stay green if
2248+
# the latch also started emitting the block JSON it exists to suppress.
2249+
if command -v jq >/dev/null; then
2250+
sm="claude/hooks/skill-mandate.sh"
2251+
if [ ! -x "$sm" ]; then
2252+
bad "latched session still logs a delegation-drift row" "$sm is missing or not executable"
2253+
else
2254+
c40_dir=$(mktemp -d)
2255+
c40_log="$c40_dir/delegation-log.jsonl"
2256+
c40_transcript="$c40_dir/t.jsonl"
2257+
# $$-scoped and prefixed distinctly from check 27's "vfy-" ids: a real session_id is a UUID,
2258+
# neither of these can collide with one, and the two fixture families stay distinguishable
2259+
# from each other in a stray log line.
2260+
c40_sid="chk40-$$-$(date +%s 2>/dev/null || echo 0)"
2261+
c40_errs=""
2262+
2263+
# The real log this hook writes to by default, untouched by this check's own destination
2264+
# override below. Sampled before and after: this is the exact leak this file's own check 27
2265+
# comment documents -- a `VAR=x cmd1 | cmd2` scoping mistake let ~12 vfy-* rows land here
2266+
# once already. If this check ever regresses to that mistake, this is what catches it.
2267+
c40_real_log="${CLAUDE_CONFIG_DIR:-$HOME/.claude}/vstack-delegation-log.jsonl"
2268+
c40_real_before=$(wc -l < "$c40_real_log" 2>/dev/null | tr -d ' ')
2269+
[ -n "$c40_real_before" ] || c40_real_before=0
2270+
2271+
c40_cleanup(){
2272+
rm -rf "$c40_dir" 2>/dev/null
2273+
rm -f "${TMPDIR:-/tmp}/vstack-mandate-$c40_sid" "${TMPDIR:-/tmp}/vstack-mandate-ckpt-$c40_sid" 2>/dev/null
2274+
unset VSTACK_DELEGATION_LOG
2275+
}
2276+
trap c40_cleanup EXIT
2277+
2278+
# Exported for the rest of this block, not prefixed onto one half of a pipe -- `VAR=x printf
2279+
# ... | bash "$hook"` scopes the assignment to printf alone and the hook never sees it, which
2280+
# is exactly how fixture rows reached the real log before. `export` here reaches every
2281+
# invocation of $sm below, piped or not.
2282+
export VSTACK_DELEGATION_LOG="$c40_log"
2283+
2284+
# An unmet mandate every time it is asked: prose written, unslop never run. What matters
2285+
# here is the latch, not which mandate trips it, so the simplest one that trips reliably.
2286+
printf '%s\n' '{"type":"assistant","message":{"content":[{"type":"tool_use","name":"Write","input":{"file_path":"/x/README.md"}}]}}' \
2287+
> "$c40_transcript"
2288+
2289+
c40_hit(){
2290+
printf '{"session_id":"%s","transcript_path":"%s","stop_hook_active":false}' \
2291+
"$c40_sid" "$c40_transcript" | "./$sm" 2>/dev/null
2292+
}
2293+
2294+
c40_out1=$(c40_hit); c40_rc1=$?
2295+
c40_out2=$(c40_hit); c40_rc2=$?
2296+
c40_out3=$(c40_hit); c40_rc3=$?
2297+
2298+
# Direction 1: the two unlatched Stops both blocked and both logged. A check that only ever
2299+
# looked at the latched row would stay green if the logger started firing unconditionally --
2300+
# this is what tells "logs on every Stop" apart from "logs only once latched".
2301+
printf '%s' "$c40_out1" | grep -q '"decision":"block"' \
2302+
|| c40_errs="$c40_errs\nStop 1 (cnt=0, unmet mandate) did not block"
2303+
printf '%s' "$c40_out2" | grep -q '"decision":"block"' \
2304+
|| c40_errs="$c40_errs\nStop 2 (cnt=1, unmet mandate) did not block"
2305+
[ "$c40_rc1" -eq 0 ] || c40_errs="$c40_errs\nStop 1 exited $c40_rc1, expected 0"
2306+
[ "$c40_rc2" -eq 0 ] || c40_errs="$c40_errs\nStop 2 exited $c40_rc2, expected 0"
2307+
2308+
# Direction 2 (the regression itself): the third Stop lands on the latch (cnt=2) and must
2309+
# still produce a row. Blocking behaviour must be unchanged at the latch: exit 0, no stdout --
2310+
# the latch's entire purpose is that a mandate the model cannot satisfy does not trap the
2311+
# session, and a passing check that let the block JSON reappear here has gated the wrong
2312+
# thing.
2313+
[ -z "$c40_out3" ] \
2314+
|| c40_errs="$c40_errs\nStop 3 (cnt=2, latched) produced stdout, must stay silent: $c40_out3"
2315+
[ "$c40_rc3" -eq 0 ] || c40_errs="$c40_errs\nStop 3 (latched) exited $c40_rc3, expected 0"
2316+
2317+
c40_nrows=$(grep -c . "$c40_log" 2>/dev/null)
2318+
[ "${c40_nrows:-0}" -eq 3 ] \
2319+
|| c40_errs="$c40_errs\nexpected 3 delegation-drift rows, found ${c40_nrows:-0}: $(cat "$c40_log" 2>/dev/null)"
2320+
2321+
if [ "${c40_nrows:-0}" -eq 3 ]; then
2322+
c40_row1=$(sed -n '1p' "$c40_log")
2323+
c40_row2=$(sed -n '2p' "$c40_log")
2324+
c40_row3=$(sed -n '3p' "$c40_log")
2325+
2326+
# Rows 1 and 2: evaluated Stops. Dense -- every count field is a real number, checkpoint
2327+
# index advances, latched is false.
2328+
jq -e --arg sid "$c40_sid" \
2329+
'.session_id==$sid and .checkpoint_index==1 and .latched==false
2330+
and (.dir_count|type)=="number" and (.ext_count|type)=="number"
2331+
and (.task_count|type)=="number" and (.named|type)=="boolean"' \
2332+
<<<"$c40_row1" >/dev/null 2>&1 \
2333+
|| c40_errs="$c40_errs\nrow 1 is not a well-formed evaluated row: $c40_row1"
2334+
jq -e --arg sid "$c40_sid" \
2335+
'.session_id==$sid and .checkpoint_index==2 and .latched==false
2336+
and (.dir_count|type)=="number" and (.ext_count|type)=="number"
2337+
and (.task_count|type)=="number" and (.named|type)=="boolean"' \
2338+
<<<"$c40_row2" >/dev/null 2>&1 \
2339+
|| c40_errs="$c40_errs\nrow 2 is not a well-formed evaluated row: $c40_row2"
2340+
2341+
# Row 3: latched. checkpoint_index still advanced (the counter moved above the latch),
2342+
# latched is true, and every count field this row cannot afford to compute is JSON null,
2343+
# not zero -- null is what makes this row tellable apart from a dense row where breadth
2344+
# genuinely measured zero, which is the whole reason the fix chose null over 0.
2345+
jq -e --arg sid "$c40_sid" \
2346+
'.session_id==$sid and .checkpoint_index==3 and .latched==true
2347+
and (.dir_count|type)=="null" and (.ext_count|type)=="null"
2348+
and (.task_count|type)=="null" and (.named|type)=="null"' \
2349+
<<<"$c40_row3" >/dev/null 2>&1 \
2350+
|| c40_errs="$c40_errs\nrow 3 is not a well-formed latched row: $c40_row3"
2351+
fi
2352+
2353+
c40_real_after=$(wc -l < "$c40_real_log" 2>/dev/null | tr -d ' ')
2354+
[ -n "$c40_real_after" ] || c40_real_after=0
2355+
[ "$c40_real_before" = "$c40_real_after" ] \
2356+
|| c40_errs="$c40_errs\nthe operator's real delegation log changed ($c40_real_before -> $c40_real_after lines) -- fixture rows leaked into it"
2357+
2358+
c40_cleanup
2359+
trap - EXIT
2360+
2361+
[ -z "$c40_errs" ] \
2362+
&& ok "latched session still logs a delegation-drift row (3 Stops, both directions)" \
2363+
|| bad "latched session still logs a delegation-drift row" "$(printf '%b' "$c40_errs")"
2364+
fi
2365+
else
2366+
skip "latched session still logs a delegation-drift row" "jq not installed"
2367+
fi
2368+
22342369
echo
22352370
# Accounting. Every declared check must have reported either a result or a skip. A check
22362371
# that throws a shell error mid-body, or is wrapped in a conditional with no else, silently

CHANGELOG.md

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,83 @@ Versions follow [semver](https://semver.org). The version lives in two manifests
44
`.claude-plugin/marketplace.json` and `claude/.claude-plugin/plugin.json`, and check 13 of
55
`.claude/verify.sh` fails when they disagree.
66

7+
## 1.40.0 — 2026-08-23
8+
9+
**The instrument built to measure long-session delegation drift went blind in exactly the sessions
10+
it exists to measure.** `skill-mandate.sh`'s 2-strike latch — `[ "$cnt" -ge 2 ] && exit 0`, the
11+
guard that stops a mandate the model cannot satisfy from trapping a session — sat above both the
12+
checkpoint counter and the delegation logger. Once a session accumulated two mandate strikes every
13+
later Stop exited before logging, permanently. Long, multi-directory sessions are the ones that
14+
latch, and long, multi-directory sessions are the entire population `tests/delegation-drift.sh`
15+
was built to describe, so the log was not merely sparse: it was filtered against its own subject.
16+
Proven by a synthetic 3-Stop drive rather than inferred from an empty file — Stop 1 gives cnt=1,
17+
ckpt=1, one row; Stop 2 gives cnt=2, ckpt=2, two rows; Stop 3 latches and leaves both frozen.
18+
The checkpoint counter moved above the latch, and a latched Stop now emits
19+
`{latched:true, dir_count:null, ...}` before exiting; the full-evaluation row carries
20+
`latched:false`. Both paths share one `_delegation_log_row()`, so there is one rotation policy
21+
instead of two copies to drift apart. Blocking behaviour is untouched. Check 40 now drives the
22+
real hook through a synthetic multi-Stop session and asserts both directions, so this cannot
23+
regress silently again. (MEESEEKS M-4; check 40 and row 40, BIRDPERSON B-3.)
24+
25+
Logging the full counts on a latched Stop was proposed and rejected on measurement, not taste.
26+
The full evaluation path costs ~116ms on a one-line synthetic, which reads as affordable; on this
27+
machine's real transcripts it is 1438ms mean / 1536ms p95 at 17.5MB, and 2009ms / 2114ms at
28+
39.1MB, because the mandate pipeline scans the transcript five-plus separate times. Sampling one
29+
latched Stop in ten would still stall the end of a long session by 1.4-2.1 seconds, on precisely
30+
the population this latch exists to protect. The reasoning and the bar for revisiting it — a
31+
measured p99 under 200ms against this file's own real transcripts, not another synthetic — are
32+
recorded at the latch.
33+
34+
**Both drift instruments counted subagent sub-transcripts as independent sessions.** 965 of 3292
35+
files under `~/.claude/projects` are per-subagent leaves nested inside a parent session, and an
36+
unbounded `find` admitted each as a top-level session at equal weight. Live, not theoretical: 15
37+
of 51 replayed sessions were leaves of two parents, 13 of them from one. A session that fans out
38+
that fanned out to thirteen leaves counted as fourteen. For `delegation-drift.sh` the exclusion is outright — a leaf's turn
39+
1 is not the parent's turn 1, so it has no position in the lifetime being measured, and the parent
40+
transcript already records the `Task`/`Agent` call that spawned it, so counting the leaf counts
41+
one delegation twice. For `compaction-effect.py` the reasoning is different and is written down as
42+
different: a compaction inside a subagent is a real event, excluded from the primary for pooling
43+
independence rather than validity, and the excluded count now prints on its own line so a zero is
44+
stated instead of assumed. Neither instrument's current numbers moved — `contributing_sessions`
45+
held at 3 because the leaves were already failing the single-checkpoint filter, and all 8 compact
46+
boundaries were independently confirmed top-level. The defect had not yet reached a printed
47+
number. It would have. (GLOOTIE G-5.)
48+
49+
`tests/compaction-effect.sh` crossed from NOT EVALUATED to a result: **no signal.** `is_error`
50+
across 3 qualifying auto boundaries reads 1/45 pre against 0/45 post (ratio 0.00x), and across 6
51+
manual boundaries 5/90 against 7/90 (ratio 1.40x), both under a 1.5x threshold. `autoCompactWindow`
52+
at 300k neither helps nor hurts the error rate at the sample available on this machine. The corpus
53+
that unblocked it was boundary count and pooled calls, not session count. `delegation-drift.sh`
54+
remains honestly NOT EVALUATED at 2 and 3 eligible windows against a floor of 8, and its secondary
55+
block now carries the contributing-session count and no-verdict qualifier on the rate lines
56+
themselves rather than in a header a reader can skim past. (GLOOTIE G-5.)
57+
58+
**Added `tests/plugin-manifests.sh`**, the by-hand authenticated-machine harness for the one lane
59+
`tests/container-matrix.sh` structurally cannot measure, because a throwaway container never
60+
installs `claude`. Eight checks, both positive controls biting: a validator that stops
61+
discriminating aborts the run at rc=2 rather than reporting its silence as health, and neutered
62+
`ok`/`bad` helpers trip the `ran == 0` floor. It covers what check 19 never did — cross-referencing
63+
`claude plugin details`'s live component inventory against disk, every skill, command and agent
64+
entry matched, plus SKILL.md presence and `hooks.json` script resolution. A skill directory with broken
65+
frontmatter that the loader silently drops passes `claude plugin validate` and fails here. Also
66+
disproved a standing assumption while building it: `plugin validate` and `plugin details` are
67+
static and local, answering correctly under an empty unauthenticated config dir. (BETH C-3.)
68+
69+
**Pre-registered the `principle-build-the-lever` investigation** at
70+
`tests/evals/build-the-lever/PREREGISTRATION.md`, with thresholds written before any run: confirm
71+
at k>=8/10, falsify at k<=2/10, 3-7 reported as nothing else, whole run void if the control drops
72+
below 7/10. Stage 0 spent 3 calls to establish that the skill description reaches the model
73+
verbatim at `MODEL=sonnet` — byte-identical, 171/171 — killing the worry that
74+
`skillListingBudgetFraction` was truncating the listing on the model the suite actually pins, and
75+
establishing that the six dead hypotheses were tested against text the model really saw. Two
76+
findings about the harness came out of the discarded probes. `ToolSearch` is absent from
77+
`auto-trigger.sh`'s `--disallowedTools` and this build has a deferred-tool registry: two turns
78+
went to tool discovery returning `No matching deferred tools found`, which is the entire budget of
79+
a case at the suite default of 3. That is not evidence any case has lost turns; it is evidence
80+
nobody has looked. And `Skill` must never be denied in any harness — deny it and the skill listing
81+
is not in context at all, so the harness measures a fleet that is not mounted. (ZEEP Z-3.)
82+
83+
784
## 1.39.0 — 2026-08-23
885

986
**The headline curl-pipe installed three Claude Code plugins and edited another vendor's config

README.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ Two directory pairs differ only by a leading dot, and the difference is the whol
2323
| path | what it is |
2424
|---|---|
2525
| `claude/` | the **shipped payload** — skills, subagents, commands, hooks, installed to `~/.claude/` |
26-
| `.claude/verify.sh` | **this repository's own gate**, 42 checks; not shipped to anyone |
26+
| `.claude/verify.sh` | **this repository's own gate**, 43 checks; not shipped to anyone |
2727
| `conductor/` | payload copied to `~/.conductor/` |
2828
| `.conductor/` | this repository's own workspace config |
2929
| `tests/` | the suites: the falsifiability harness, the install matrix, trigger and baseline tests |
@@ -61,8 +61,8 @@ cd ~/Projects/vstack && ./install.sh
6161
Pin a release rather than tracking `main`:
6262

6363
```bash
64-
curl -fsSL https://raw.githubusercontent.com/itsvedantkumar/vstack/v1.39.0/bootstrap.sh -o bootstrap.sh
65-
VSTACK_REF=v1.39.0 bash bootstrap.sh # installs that tag, not main
64+
curl -fsSL https://raw.githubusercontent.com/itsvedantkumar/vstack/v1.40.0/bootstrap.sh -o bootstrap.sh
65+
VSTACK_REF=v1.40.0 bash bootstrap.sh # installs that tag, not main
6666
```
6767

6868
The curl one-liner above always runs `./setup-machine.sh` first, which installs the tools this
@@ -151,14 +151,14 @@ reaches for `unslop`, reviewing TypeScript reaches for `typescript-best-practice
151151

152152
## Checks that can fail
153153

154-
The gate is 42 checks. `tests/gate-falsifiability.sh` breaks the repository once per check, at
154+
The gate is 43 checks. `tests/gate-falsifiability.sh` breaks the repository once per check, at
155155
least once and more where a check can fail in more than one way, requires the gate to go red
156156
naming that check, restores the tree byte for byte, and fails if anything was left behind.
157157
**Check 16 fails if any check has no mutation row**, so a check cannot be added without proof it
158158
can fail.
159159

160160
```bash
161-
./.claude/verify.sh # 42 checks
161+
./.claude/verify.sh # 43 checks
162162
git clone . /tmp/vstack-check && cd /tmp/vstack-check && ./tests/gate-falsifiability.sh
163163
```
164164

claude/.claude-plugin/plugin.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "vstack",
3-
"version": "1.39.0",
3+
"version": "1.40.0",
44
"description": "Skills that fire on the situation instead of a slash command, plus the subagents, commands, and session hook that make them fire. Verification gates, parallel fan-out, code review, and writing discipline.",
55
"author": {
66
"name": "Vedant Kumar"

0 commit comments

Comments
 (0)