Make two skill routings mandatory instead of merely instructed

Everything vstack does to route skills is instruction: the SessionStart
digest spells out "any prose you write -> unslop", and the skill
descriptions carry their own triggers. auto-trigger.sh measures that
landing on 12 cases, which is a weaker claim than "always".

claude/hooks/skill-mandate.sh runs on Stop and makes two of them
certain. It reads the transcript for what actually happened — which
files were written, which skills were invoked — and blocks the turn
when a rule went unmet: .md/.mdx requires unslop, .ts/.tsx requires
typescript-best-practices.

Two rules, not twenty. The bar for a third is that the situation is
decidable from a tool call rather than from judgement, and that the
skill is the right answer every single time. A rule right nine times
in ten belongs in the routing block; as a gate it would only teach you
to switch the gate off. It blocks at most twice per session, never
fires inside an active Stop hook, and VSTACK_NO_MANDATE=1 disables it.
Check 27 exercises all seven behaviours in both directions.

Measured first, rather than assumed: --debug-file on the real CLI
reports 73 skills sent and no dropped descriptions at the current
0.016 listing fraction, so there is headroom for mandated routing.
That measurement also confirmed the merged research PR's correction
that Claude Code folds slash commands into skills — 32 user skills
plus 14 legacy commands.

Also in this commit:

The redactor added last commit ate ordinary prose. The session digest
line "TOKENS: never read whole files" came back as "TOKENS: [REDACTED]
read whole files", because matching the name alone is not enough. The
name now only nominates a candidate and the value decides: opaque
enough to be a credential, or left alone. Check 25 gained four prose
controls, which is what the single control line missed.

New check 28: every doc under docs/ must be reachable from another
doc. The merged research handoff was 783 lines nothing linked to. A
link to the containing directory counts, since provenance/README.md
indexes plans/ that way.

The curl and marketplace lanes measured a published install against
local counts, so adding a hook made the matrix red for the correct
reason that it was not pushed yet — unclearable, since preflight gates
the commit. They now measure against the tree they actually installed
and print a note when this checkout is ahead.

verify 30/30, falsifiability 32/32, matrix 22/22, preflight green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: itsvedantkumar

.claude/verify.sh

@@ -1002,15 +1002,27 @@ DATABASE_URL=postgres://user:s3cr3tpw@host/db|s3cr3tpw
 GITHUB_TOKEN=ghp_FAKEVAL|ghp_FAKEVAL
 REDEOF
 
-    # The other half of the claim: redaction that eats ordinary diagnostics is its own defect,
-    # because the tail exists to be read. A control line has to come back intact.
-    ctl="error: cannot find module at /usr/local/lib/thing.js (exit 2)"
-    cout=$(printf '%s' "$(jq -cn --arg e "$ctl" '{tool_name:"Bash",tool_response:{stderr:$e}}')" \
-           | "$fd" 2>/dev/null)
-    case "$cout" in
-      *"cannot find module at /usr/local/lib/thing.js"*) ;;
-      *) leaks="$leaks\n  over-redacted an ordinary error line" ;;
-    esac
+    # The other half of the claim, and the half that actually broke. Redaction that eats ordinary
+    # diagnostics is its own defect, because the tail exists to be read. One control line was not
+    # enough: with only "cannot find module" here, a redactor that matched on the NAME alone
+    # passed this check while turning the session digest "TOKENS: never read whole files" into
+    # "TOKENS: [REDACTED] read whole files". Every line below contains a credential-shaped word
+    # followed by a colon and ordinary prose, which is the case that was missed.
+    while IFS= read -r ctl; do
+      [ -n "$ctl" ] || continue
+      cout=$(printf '%s' "$(jq -cn --arg e "$ctl" '{tool_name:"Bash",tool_response:{stderr:$e}}')" \
+             | "$fd" 2>/dev/null)
+      case "$cout" in
+        *"$ctl"*) ;;
+        *) leaks="$leaks\n  over-redacted an ordinary line: $ctl" ;;
+      esac
+    done <<'CTLEOF'
+error: cannot find module at /usr/local/lib/thing.js (exit 2)
+TOKENS: never read whole files, use grep
+Keystrokes: 1420 recorded
+passwords: are stored hashed in this table
+npm ERR! code ELIFECYCLE, exit status 1
+CTLEOF
 
     [ -z "$leaks" ] \
       && ok "failure tail redacts credentials (9 shapes masked, plain errors intact)" \
@@ -1048,6 +1060,86 @@ else
   skip "documented platforms match CI" "no .github/workflows/verify.yml"
 fi
 
+# --- 27. the skill mandate blocks on an unmet rule and stays quiet otherwise -------------------
+#
+# The digest tells the model to route to a skill and the descriptions carry their own triggers.
+# Both are instructions, and an instruction is a probability -- auto-trigger.sh measures the
+# routing landing, not the routing being certain. This hook is the part that is certain, for the
+# few rules decidable from tool calls rather than judgement, so it has to be exercised in both
+# directions. A gate that never blocks and a gate that always blocks look identical from a
+# passing test that only checks one of them.
+if command -v jq >/dev/null; then
+  sm="claude/hooks/skill-mandate.sh"
+  if [ ! -x "$sm" ]; then
+    bad "skill mandate decides correctly" "$sm is missing or not executable"
+  else
+    md=$(mktemp -d); errs=""
+    say_(){ printf '%s\n' "$@" > "$md/t.jsonl"; }
+    hit_(){ printf '{"session_id":"vfy-%s","transcript_path":"%s/t.jsonl","stop_hook_active":%s}' \
+              "$1" "$md" "${2:-false}" | "./$sm" 2>/dev/null; }
+    W='{"type":"assistant","message":{"content":[{"type":"tool_use","name":"Write","input":{"file_path":"/x/README.md"}}]}}'
+    T='{"type":"assistant","message":{"content":[{"type":"tool_use","name":"Edit","input":{"file_path":"/x/App.tsx"}}]}}'
+    U='{"type":"assistant","message":{"content":[{"type":"tool_use","name":"Skill","input":{"skill":"unslop"}}]}}'
+    P='{"type":"assistant","message":{"content":[{"type":"tool_use","name":"Write","input":{"file_path":"/x/main.py"}}]}}'
+
+    # blocks when the rule is unmet
+    say_ "$W"; hit_ a | grep -q '"decision":"block"' || errs="$errs\nwrote prose without unslop and it did not block"
+    say_ "$T"; hit_ b | grep -q '"decision":"block"' || errs="$errs\nwrote TypeScript without the ts skill and it did not block"
+    # silent when the rule is met, or does not apply
+    say_ "$W" "$U"; [ -z "$(hit_ c)" ] || errs="$errs\nblocked even though unslop had run"
+    say_ "$P";      [ -z "$(hit_ d)" ] || errs="$errs\nblocked on a file no mandate covers"
+    # cannot trap the session
+    say_ "$W"; [ -z "$(hit_ e true)" ] || errs="$errs\nblocked while stop_hook_active was already true"
+    say_ "$W"; [ -z "$(VSTACK_NO_MANDATE=1 hit_ f)" ] || errs="$errs\nignored VSTACK_NO_MANDATE=1"
+    say_ "$W"; hit_ g >/dev/null; hit_ g >/dev/null
+    [ -z "$(hit_ g)" ] || errs="$errs\ndid not latch open after 2 blocks in one session"
+
+    rm -rf "$md"; rm -f "${TMPDIR:-/tmp}"/vstack-mandate-vfy-* 2>/dev/null
+    [ -z "$errs" ] && ok "skill mandate decides correctly (7 cases, both directions)" \
+      || bad "skill mandate decides correctly" "$(printf '%b' "$errs")"
+  fi
+else
+  skip "skill mandate decides correctly" "jq not installed"
+fi
+
+# --- 28. every doc is reachable from another doc ----------------------------------------------
+#
+# A file nobody links to is a file nobody reads and nobody updates, and it rots in public. A
+# 783-line research handoff landed in docs/ and was reachable from nothing: not the README index,
+# not another document. It read as deleted while still being served to anyone browsing the repo.
+#
+# The rule is reachability, not a fixed index: a doc linked from another doc is fine. What is not
+# fine is a doc no path in the repository leads to.
+docs_all=$(find docs -name '*.md' 2>/dev/null | sort)
+if [ -z "$docs_all" ]; then
+  skip "every doc is reachable" "no docs/ directory"
+else
+  # Written to a file and grepped directly. `printf ... | grep -q` returns 141 under pipefail
+  # when grep exits early on a match, so every doc that WAS linked reported as an orphan -- the
+  # same pipe-and-exit-status trap that has bitten this repository twice before.
+  linkable=$(mktemp)
+  cat README.md CHANGELOG.md $(find docs -name '*.md') claude/skills/*/SKILL.md > "$linkable" 2>/dev/null
+  orphans=""
+  for d in $docs_all; do
+    base=${d##*/}
+    # Either the repo-relative path or the bare filename, since docs link to each other by
+    # sibling name.
+    dir=${d%/*}
+    # A link to the containing directory counts. docs/provenance/README.md points at `plans/`
+    # rather than naming each plan, and a reader following it lands on all three -- that is
+    # reachable, and demanding a per-file link would only produce an index nobody maintains.
+    if ! grep -qF "$d" "$linkable" \
+       && ! grep -qF "]($base" "$linkable" \
+       && ! grep -qE "\]\((\./)?${dir##*/}/\)" "$linkable"; then
+      orphans="$orphans\n  $d"
+    fi
+  done
+  rm -f "$linkable"
+  [ -z "$orphans" ] \
+    && ok "every doc is reachable ($(printf '%s' "$docs_all" | wc -l | tr -d ' ') under docs/)" \
+    || bad "every doc is reachable" "$(printf 'nothing in this repository links to:%b' "$orphans")"
+fi
+
 echo
 # Accounting. Every declared check must have reported either a result or a skip. A check
 # that throws a shell error mid-body, or is wrapped in a conditional with no else, silently

README.md

@@ -25,7 +25,7 @@ That claim is easy to make and most setups make it. Here it is measured: `tests/
 runs 12 prompts against the real model and reports which attempt each one landed on, so the day
 routing starts eroding shows up as a number rather than a feeling.
 
-The same idea runs through the rest. Every one of the 28 checks in the verification gate has a
+The same idea runs through the rest. Every one of the 30 checks in the verification gate has a
 row in a suite that breaks what that check watches and requires the gate to go red naming it — a
 check nobody has watched fail is indistinguishable from a check that always passes. The
 installer is run into 22 throwaway home directories on Linux, macOS and Alpine on every
@@ -150,7 +150,7 @@ exits 127 on every hook event inside a sandbox.
 | Skills | 26 | `~/.claude/skills/` |
 | Subagents | 8 | `~/.claude/agents/` |
 | Commands | 14 | `~/.claude/commands/` |
-| Hooks | 5 | `~/.claude/hooks/` |
+| Hooks | 6 | `~/.claude/hooks/` |
 | CLI wrappers | 7 | `~/.config/agents/bin/` |
 | MCP servers | 2 | merged into `~/.claude.json` |
 | Global directives | `CLAUDE.md` | `~/.claude/CLAUDE.md` |
@@ -214,6 +214,30 @@ start.
 which skills fired. Strip the routing block and cases fail, which is how you know the test is
 worth having. [docs/how-skills-fire.md](docs/how-skills-fire.md) has the measurements.
 
+### The ones that are not optional
+
+Everything above is instruction, and an instruction is a probability. The routing lands on the
+cases that measure it, and "lands on the cases that measure it" is a weaker claim than "always".
+
+`claude/hooks/skill-mandate.sh` runs on `Stop` and makes two of them certain. It reads the
+session transcript for what actually happened, which files were written and which skills were
+invoked, and refuses to let the turn finish when a rule went unmet:
+
+| You wrote | This must have run |
+|---|---|
+| `.md` or `.mdx` | `unslop` |
+| `.ts` or `.tsx` | `typescript-best-practices` |
+
+Two rules, not twenty, and the bar for adding a third is high: the situation has to be decidable
+from a tool call rather than from judgement, and the skill has to be the right answer every
+single time. A rule that is correct nine times in ten belongs in the routing block as guidance.
+As a gate it would just teach you to switch the gate off.
+
+It blocks at most twice per session and then latches open, it never fires while a `Stop` hook is
+already running, and `VSTACK_NO_MANDATE=1` turns it off. Check 27 exercises all seven of those
+behaviours in both directions, because a gate that always blocks passes any test that only
+checks that it blocks.
+
 ## What this does that an unconfigured setup does not
 
 Most setups in this space claim to make you better and none of them show it, because the claim
@@ -428,6 +452,9 @@ true.
   each of its own checks can fail
 - [Why these skills](docs/provenance/pstack-audit.md), the fit-and-benefit audit of all 44 pstack
   skills that decided which 18 were worth porting
+- [Provenance](docs/provenance/README.md), the design history this project was extracted from
+- [Five open investigations](docs/provenance/research-v1.7.0.md), an outside research pass on
+  skill reachability, false-success labelling, ablation design, evidence bundles and a UI floor
 - [Changelog](CHANGELOG.md), with the measurements behind each release
 
 ## Credits

claude/hooks/failure-diagnose.sh

@@ -27,12 +27,49 @@ else tool=$(printf '%s' "$in" | sed -n 's/.*"tool_name" *: *"\([^"]*\)".*/\1/p'
 #
 # Over-redaction is the cheap direction here: this text is diagnostic context for a model, and a
 # masked value costs a retry while a leaked one is permanent.
-redact(){ sed -E \
-  -e 's/(sk-ant-|sk-proj-|sk-|github_pat_|ghp_|gho_|ghu_|ghs_|ghr_|glpat-|xoxb-|xoxp-|xoxa-|xapp-|AKIA|ASIA|AIza|ya29\.|hf_|npm_|dop_v1_)[A-Za-z0-9_\/+.-]{6,}/\1[REDACTED]/g' \
-  -e 's/(eyJ[A-Za-z0-9_-]{4,})\.[A-Za-z0-9_-]{4,}\.[A-Za-z0-9_-]+/\1.[REDACTED]/g' \
-  -e 's/([Aa]uthorization|[Pp]roxy-[Aa]uthorization)([[:space:]]*[:=][[:space:]]*)("?)([A-Za-z]+[[:space:]]+)?[^[:space:]"'"'"']{6,}/\1\2\3\4[REDACTED]/g' \
-  -e 's/(([Aa]pi[_-]?|[Aa]ccess[_-]?|[Ss]ecret[_-]?|[Aa]uth[_-]?|[Pp]rivate[_-]?|[Bb]earer[_-]?|[Rr]efresh[_-]?|[Ss]ession[_-]?)?([Kk][Ee][Yy]|[Tt][Oo][Kk][Ee][Nn]|[Ss][Ee][Cc][Rr][Ee][Tt]|[Pp][Aa][Ss][Ss][Ww][Oo][Rr][Dd]|[Pp][Aa][Ss][Ss][Ww][Dd]|[Cc][Rr][Ee][Dd][Ee][Nn][Tt][Ii][Aa][Ll][Ss]?)[A-Za-z0-9_-]*)("?[[:space:]]*[:=][[:space:]]*)("?)[^[:space:],;"'"'"']{4,}/\1\4\5[REDACTED]/g' \
-  -e 's|([A-Za-z][A-Za-z0-9+.-]*://[^/:[:space:]]+):[^@[:space:]]+@|\1:[REDACTED]@|g'
+redact(){
+  # Prefixed tokens, JWTs, auth headers and URL userinfo are unambiguous, so sed handles them.
+  sed -E \
+    -e 's/(sk-ant-|sk-proj-|sk-|github_pat_|ghp_|gho_|ghu_|ghs_|ghr_|glpat-|xoxb-|xoxp-|xoxa-|xapp-|AKIA|ASIA|AIza|ya29\.|hf_|npm_|dop_v1_)[A-Za-z0-9_\/+.-]{6,}/\1[REDACTED]/g' \
+    -e 's/(eyJ[A-Za-z0-9_-]{4,})\.[A-Za-z0-9_-]{4,}\.[A-Za-z0-9_-]+/\1.[REDACTED]/g' \
+    -e 's/([Aa]uthorization|[Pp]roxy-[Aa]uthorization)([[:space:]]*[:=][[:space:]]*)("?)([A-Za-z]+[[:space:]]+)?[^[:space:]"'"'"']{6,}/\1\2\3\4[REDACTED]/g' \
+    -e 's|([A-Za-z][A-Za-z0-9+.-]*://[^/:[:space:]]+):[^@[:space:]]+@|\1:[REDACTED]@|g' \
+  | awk '
+  # NAME=VALUE needs both halves judged, which sed cannot do. Matching the name alone ate
+  # ordinary output: the hook digest line "TOKENS: never read whole files" came back as
+  # "TOKENS: [REDACTED] read whole files", and "Keystrokes: 1420 recorded" went the same way.
+  # A diagnostic tail that has been chewed up is worth nothing, and the point of this hook is
+  # that the tail gets read. So the name only nominates a candidate; the value decides.
+  #
+  # A credential is opaque. English is not. Sixteen or more characters, or six-plus with a
+  # digit in them, or base64 padding, and it is not a word anyone typed as prose. "never" and
+  # "1420" survive that test and every real key shape below fails it.
+  function looks_secret(v,   n) {
+    gsub(/^["\047]+|["\047]+$/, "", v)
+    n = length(v)
+    if (n < 4) return 0
+    if (n >= 16) return 1
+    if (v ~ /[0-9]/ && n >= 6) return 1
+    if (v ~ /[\/+=]/) return 1
+    return 0
+  }
+  {
+    line = $0; out = ""
+    while (1) {
+      lc = tolower(line)
+      if (!match(lc, /(^|[^a-z0-9_-])[a-z0-9_-]*(key|token|secret|passwd|password|credential)[a-z0-9_-]*"?[ \t]*[:=][ \t]*"?[^ \t,;"\047]+/)) break
+      st = RSTART; ln = RLENGTH
+      seg = substr(line, st, ln)
+      if (match(seg, /[:=][ \t]*"?/)) {
+        head = substr(seg, 1, RSTART + RLENGTH - 1)
+        val  = substr(seg, RSTART + RLENGTH)
+        if (looks_secret(val)) seg = head "[REDACTED]"
+      }
+      out = out substr(line, 1, st - 1) seg
+      line = substr(line, st + ln)
+    }
+    print out line
+  }'
 }
 if [ -n "$JQ" ]; then
   err=$(printf '%s' "$in" | "$JQ" -r '(.tool_response.error // .tool_response.stderr // .tool_response // "") | tostring' 2>/dev/null \

claude/hooks/skill-mandate.sh

@@ -0,0 +1,87 @@
+#!/usr/bin/env bash
+# Stop: refuse to finish when the session did work that a skill is mandated for and never ran it.
+#
+# vstack already tells the model how to route: the SessionStart digest spells out "any prose you
+# write -> unslop", and the skill descriptions carry their own triggers. Both are instructions,
+# and an instruction is a probability. Measured over 12 prompts the routing lands, but "lands most
+# of the time" is not the same claim as "always", and the second one is the one worth having.
+#
+# This closes the gap for the few rules where the situation is decidable from the transcript
+# rather than from judgement. It reads what the session actually did -- which files were written,
+# which skills were invoked -- and blocks Stop when a mandate went unmet. Nothing here guesses at
+# intent: if the rule cannot be decided by looking at tool calls, it does not belong in this file.
+#
+# Escape hatch: VSTACK_NO_MANDATE=1 disables it entirely. A gate you cannot turn off gets deleted
+# by the first person it inconveniences, which is worse than one that is on by default.
+set -uo pipefail
+
+[ "${VSTACK_NO_MANDATE:-0}" = "1" ] && exit 0
+
+JQ=""
+if [ -x /usr/bin/jq ]; then JQ=/usr/bin/jq
+elif command -v jq >/dev/null 2>&1; then JQ=$(command -v jq); fi
+# Without jq there is no way to read the transcript path or emit a block reason safely. Say
+# nothing rather than guess: a mandate that misfires is worse than one that abstains.
+[ -n "$JQ" ] || exit 0
+
+input=$(cat 2>/dev/null || true)
+
+# Claude Code sets this when Stop already fired once for this turn. Blocking again from inside a
+# block is how a hook turns into an infinite loop.
+[ "$(printf '%s' "$input" | "$JQ" -r '.stop_hook_active // false')" = "true" ] && exit 0
+
+tr_=$(printf '%s' "$input" | "$JQ" -r '.transcript_path // empty')
+[ -n "$tr_" ] && [ -f "$tr_" ] || exit 0
+
+sid=$(printf '%s' "$input" | "$JQ" -r '.session_id // empty'); [ -n "$sid" ] || sid="pid$PPID"
+cnt_file="${TMPDIR:-/tmp}/vstack-mandate-$sid"
+cnt=$(cat "$cnt_file" 2>/dev/null || echo 0)
+# Same latch as the verify gate. A mandate the model cannot satisfy must not trap the session.
+[ "$cnt" -ge 2 ] && exit 0
+
+# Every file this session wrote or edited, and every skill it invoked. Both come from the
+# transcript, so this measures what happened rather than what was asked for.
+paths=$(
+  "$JQ" -r 'select(.type=="assistant") | .message.content[]?
+            | select(.type=="tool_use" and (.name=="Write" or .name=="Edit" or .name=="NotebookEdit"))
+            | .input.file_path // empty' "$tr_" 2>/dev/null | sort -u
+)
+skills=$(
+  "$JQ" -r 'select(.type=="assistant") | .message.content[]?
+            | select(.type=="tool_use" and .name=="Skill") | .input.skill // empty' "$tr_" 2>/dev/null \
+  | sed 's/.*://' | sort -u
+)
+
+fired(){ printf '%s\n' "$skills" | grep -qxF "$1"; }
+
+unmet=""
+# --- the mandates -----------------------------------------------------------------------------
+# Each one needs a situation decidable from a tool call, and a skill that is the answer to it
+# every single time. That second half is the strict part: a rule that is right nine times out of
+# ten belongs in the digest as guidance, not here as a gate.
+
+# Prose. Any Markdown that is not a machine-written log or a vendored file.
+prose=$(printf '%s\n' "$paths" | grep -iE '\.(md|mdx)$' \
+        | grep -viE '(CHANGELOG\.md|node_modules|\.audit/|/(dist|build|vendor)/)' | head -5)
+if [ -n "$prose" ] && ! fired unslop; then
+  unmet="$unmet
+  unslop -- you wrote prose and it never ran: $(printf '%s' "$prose" | tr '\n' ' ')"
+fi
+
+# TypeScript. Reading one is judgement; writing one is not.
+ts=$(printf '%s\n' "$paths" | grep -E '\.(ts|tsx)$' | grep -v node_modules | head -5)
+if [ -n "$ts" ] && ! fired typescript-best-practices; then
+  unmet="$unmet
+  typescript-best-practices -- you wrote TypeScript and it never ran: $(printf '%s' "$ts" | tr '\n' ' ')"
+fi
+
+[ -n "$unmet" ] || { rm -f "$cnt_file"; exit 0; }
+
+echo $((cnt+1)) > "$cnt_file"
+reason="A vstack skill mandate went unmet (attempt $((cnt+1))/2). These fire every time, not when they seem relevant:
+$unmet
+
+Run each named skill with the Skill tool against the files listed, apply what it says, then finish.
+Set VSTACK_NO_MANDATE=1 to disable this gate."
+"$JQ" -cn --arg r "$reason" '{decision:"block",reason:$r}'
+exit 0

claude/settings.json

@@ -70,6 +70,10 @@
           {
             "type": "command",
             "command": "\"$CLAUDE_PROJECT_DIR/.claude/hooks/verify-gate.sh\""
+          },
+          {
+            "type": "command",
+            "command": "\"$CLAUDE_PROJECT_DIR/.claude/hooks/skill-mandate.sh\""
           }
         ]
       }