Audit: nine greens that measured nothing, and the 141 that hid them

.claude-plugin/marketplace.json

@@ -10,7 +10,7 @@
   "plugins": [
     {
       "name": "vstack",
-      "version": "1.8.0",
+      "version": "1.9.1",
       "source": "./claude",
       "description": "26 skills that fire without a slash command, 8 agents, 14 commands, and the session hook that routes situations to skills. Most skills are ported from pstack and Superpowers — see claude/skills/ATTRIBUTION.md for per-skill source and license.",
       "category": "workflow"

.claude/verify.sh

@@ -35,7 +35,7 @@ for t in jq git; do command -v "$t" >/dev/null 2>&1 || missing="$missing $t"; do
 # --- 1. every shell script parses ----------------------------------------------------------
 errs=""
 while IFS= read -r f; do
-  head -1 "$f" | grep -q '^#!.*sh' || continue
+  grep -q '^#!.*sh' <<<"$(head -1 "$f" 2>/dev/null)" || continue   # not a pipe: SIGPIPE + pipefail = 141
   out=$(bash -n "$f" 2>&1) || errs="$errs\n$f: $out"
 done < <(find . -path ./.git -prune -o -type f \( -name "*.sh" -o -path "./bin/*" \) -print)
 [ -z "$errs" ] && ok "shell syntax" || bad "shell syntax" "$(printf '%b' "$errs")"
@@ -345,6 +345,13 @@ command -v jq >/dev/null && nmc=$(jq 'keys|length' mcp/servers.json 2>/dev/null
 # at 25 against a gate of 26 and nothing could see it, because the noun was not in the map.
 nck=$TOTAL
 
+# Same shape, one noun later. The CHANGELOG said "29 shell scripts" against a tree of 27 .sh
+# files and 31 shebang scripts, and nothing could see it because "shell script" was not in the
+# map. Derived the same way check 29 selects, so the two can never disagree.
+nsh=$(git ls-files 2>/dev/null | while IFS= read -r f; do
+  grep -q '^#!.*sh' <<<"$(head -1 "$f" 2>/dev/null)" && printf 'x\n'
+done | grep -c .)
+
 want_for(){ # noun (lowercased, plural or singular) -> expected count, or empty if not covered
   case "$1" in
     skill|skills)                                   printf '%s' "$nsk" ;;
@@ -355,6 +362,7 @@ want_for(){ # noun (lowercased, plural or singular) -> expected count, or empty
     "cli wrapper"|"cli wrappers")                   printf '%s' "$nwr" ;;
     case|cases)                                     printf '%s' "$ncs" ;;
     "mcp server"|"mcp servers")                     printf '%s' "$nmc" ;;
+    "shell script"|"shell scripts")                 printf '%s' "$nsh" ;;
   esac
 }
 
@@ -369,11 +377,40 @@ exempt_phrases(){
   esac
 }
 
+# Every noun this check can resolve, spelled once. The extraction regex is built from this
+# list, so a noun can no longer be resolvable-but-never-extracted -- which is how "29 shell
+# scripts" sat in the CHANGELOG unchallenged. want_for had no case for it, and even after one
+# was added the claim stayed invisible, because the extractor carried its own separate
+# alternation and nothing compared the two.
+NOUNS='skills?|checks?|agents?|subagents?|sub-agents?|commands?|hooks?|CLI wrappers?|cases?|MCP servers?|shell scripts?'
+
 errs=""
-for f in README.md .claude-plugin/marketplace.json claude/.claude-plugin/plugin.json \
+
+# Positive control. Every alternative the extractor looks for must resolve to a number, or the
+# check pulls claims out of the docs and then drops them on the floor without saying so.
+while IFS= read -r _n; do
+  [ -n "$_n" ] || continue
+  [ -n "$(want_for "$(printf '%s' "$_n" | tr '[:upper:]' '[:lower:]')")" ] \
+    || errs="$errs\ninternal: the extractor looks for '$_n' but want_for cannot resolve it"
+done <<EOF
+$(printf '%s' "$NOUNS" | tr '|' '\n' | sed 's/?$//')
+EOF
+
+for f in README.md CHANGELOG.md .claude-plugin/marketplace.json claude/.claude-plugin/plugin.json \
          claude/skills/ATTRIBUTION.md claude/CLAUDE.md docs/how-skills-fire.md tests/README.md; do
   [ -f "$f" ] || continue
-  norm=$(tr '\n' ' ' < "$f" | tr -s '[:space:]' ' ')
+  if [ "$f" = CHANGELOG.md ]; then
+    # Only the entries that describe what ships today: Unreleased, plus the section for the
+    # version the manifests declare. Older entries record what was true at that release and
+    # rewriting them to satisfy today's tree would be falsifying history. Taking simply "the top
+    # section" is not enough -- an Unreleased heading sits above the release, and slicing there
+    # stopped one section short of the claim that had gone stale.
+    _cv=$(jq -r '.version' claude/.claude-plugin/plugin.json 2>/dev/null)
+    norm=$(awk -v v="$_cv" '/^## /{ sec = ($2 == "Unreleased" || index($2, v) == 1) } sec' "$f" \
+             | tr '\n' ' ' | tr -s '[:space:]' ' ')
+  else
+    norm=$(tr '\n' ' ' < "$f" | tr -s '[:space:]' ' ')
+  fi
   while IFS= read -r ph; do
     [ -n "$ph" ] && norm=${norm//"$ph"/}
   done <<EOF
@@ -389,7 +426,7 @@ EOF
     [ -n "$want" ] && [ "$num" != "$want" ] \
       && errs="$errs\n$f: claims '$claim', tree has $want"
   done <<EOF
-$(printf '%s' "$norm" | grep -oE '[0-9]+ ((sub-?)?agents?|skills?|commands?|hooks?|cases?|checks?|MCP servers?|CLI wrappers?)' | sort -u)
+$(printf '%s' "$norm" | grep -oE "[0-9]+ ($NOUNS)" | sort -u)
 EOF
 
   # table form: "| Commands | 14 |"
@@ -940,11 +977,23 @@ if command -v git >/dev/null && command -v jq >/dev/null; then
   # quickstart pinned v1.4.0 while the manifests said v1.8.0, so anyone copy-pasting the "pin a
   # release" lane got a four-version-old payload and no error -- the tag resolves, the install
   # succeeds, and the only symptom is a setup that quietly disagrees with its own README.
+  #
+  # Agreeing with the manifest is not enough. The quickstart's "pin a release" lane pinned
+  # v1.8.0 while the manifests said v1.8.0 and no such tag existed, so the check was satisfied
+  # and the URL a stranger copy-pastes returned 404. A pin has to name a tag that is actually
+  # there. Only asserted where the checkout has tags at all -- a shallow clone has none, and the
+  # branch below already declines to measure in that case.
   pins=""
   for f in README.md docs/*.md; do
     [ -f "$f" ] || continue
     while IFS= read -r pv; do
-      [ -n "$pv" ] && [ "$pv" != "$mv_" ] && pins="$pins\n  $f pins v$pv"
+      [ -n "$pv" ] || continue
+      if [ "$pv" != "$mv_" ]; then
+        pins="$pins\n  $f pins v$pv"
+      elif [ -n "$(git tag -l 2>/dev/null | head -1)" ] \
+           && ! git rev-parse -q --verify "refs/tags/v$pv" >/dev/null 2>&1; then
+        pins="$pins\n  $f pins v$pv, which is not a tag in this repository (the URL 404s)"
+      fi
     done <<PINEOF
 $(grep -oE '(vstack/v|VSTACK_REF=v)[0-9]+\.[0-9]+\.[0-9]+' "$f" 2>/dev/null | sed -E 's/.*v//' | sort -u)
 PINEOF
@@ -961,7 +1010,12 @@ PINEOF
     # failure mode it was written to prevent, reproduced inside itself.
     skip "declared version matches what installs" "no tags in this checkout (shallow clone?), so there is nothing to compare against"
   elif ! git rev-parse -q --verify "refs/tags/v$mv_" >/dev/null 2>&1; then
-    ok "declared version matches what installs (v$mv_ not yet tagged)"
+    # A declared-but-untagged version has no payload to diff against, so this branch compares
+    # nothing. It used to print "ok", which is the same defect the tagless branch above guards
+    # against, one elif lower down: a green that measured nothing, hidden from the skip census
+    # because only skips are counted there. Say skip, and the release unit has to tag before the
+    # check starts measuring again.
+    skip "declared version matches what installs" "v$mv_ is declared by the manifests but not tagged, so there is no payload to compare it against — tag the release and this starts measuring"
   else
     # Everything a lane actually delivers. Docs, tests and CI are deliberately excluded: they
     # change without changing what a stranger receives.
@@ -1148,24 +1202,100 @@ fi
 
 # --- 29. every shell script passes shellcheck ------------------------------------------------
 #
-# This bundle is 27 shell scripts and almost nothing else, and the whole product is the claim
+# This bundle is shell scripts and almost nothing else, and the whole product is the claim
 # that they behave correctly on someone else's machine. The class of bug that keeps landing here
 # is not exotic -- an unquoted expansion, a pattern that can never match, a variable set for a
 # check nobody wrote -- and a linter finds all three for free.
 #
 # Warning level, not style: informational notes are opinions and this should fail on defects.
-# Where a warning is wrong the suppression carries a reason on the line above it, so the next
-# reader can see the argument rather than a bare disable.
+# Where a warning is wrong the suppression carries a reason, which check 30 enforces.
+#
+# Selected by shebang, the same way check 1 selects. It used to be the hand-maintained list
+# `git ls-files '*.sh' bin/doctor bin/vstack`, and bin/cloudflare-mcp -- a #!/bin/sh script with
+# no .sh suffix -- had never been on it. Appending an unquoted `$HOME/some path` to that file
+# left shellcheck exiting 1 on it while this check still printed "ok shellcheck clean (29
+# scripts)". A list you have to remember to update is a list that goes stale silently.
 if command -v shellcheck >/dev/null 2>&1; then
-  sc_out=$(git ls-files '*.sh' bin/doctor bin/vstack 2>/dev/null \
-    | while IFS= read -r f; do shellcheck -S warning -f gcc "$f" 2>/dev/null; done)
+  sc_files=$(git ls-files 2>/dev/null | while IFS= read -r f; do
+    grep -q '^#!.*sh' <<<"$(head -1 "$f" 2>/dev/null)" && printf '%s\n' "$f"
+  done)
+  sc_out=$(while IFS= read -r f; do
+    [ -n "$f" ] && shellcheck -S warning -f gcc "$f" 2>/dev/null
+  done <<<"$sc_files")
   [ -z "$sc_out" ] \
-    && ok "shellcheck clean ($(git ls-files '*.sh' bin/doctor bin/vstack 2>/dev/null | wc -l | tr -d ' ') scripts, warning level)" \
+    && ok "shellcheck clean ($(grep -c . <<<"$sc_files") scripts, warning level)" \
     || bad "shellcheck clean" "$(printf '%s' "$sc_out" | sed 's/^/  /' | head -20)"
 else
   skip "shellcheck clean" "shellcheck not installed (brew install shellcheck / apk add shellcheck)"
 fi
 
+# --- 30. every shellcheck suppression carries a reason ----------------------------------------
+#
+# Check 29's own header has said for several versions that a suppression carries its reason with
+# it, so the next reader sees the argument rather than a bare disable. Nothing enforced it, and
+# bootstrap.sh had carried a naked `# shellcheck disable=SC2086` since the lane was written. A
+# rule that lives only in prose is a rule that gets skipped by whoever did not read the prose,
+# which is the second time that has happened here -- the documented-count rule was the first.
+#
+# A reason counts if it is on the same line after the code list, which is how the other five
+# suppressions in this repo are written, or on the line immediately above. Both are readable at
+# the point of the disable; a reason three lines away is not.
+bare=""
+nsup=0
+while IFS= read -r f; do
+  [ -n "$f" ] || continue
+  grep -q '^#!.*sh' <<<"$(head -1 "$f" 2>/dev/null)" || continue
+  while IFS= read -r hit; do
+    [ -n "$hit" ] || continue
+    n=${hit%%:*}
+    nsup=$((nsup + 1))
+    line=$(sed -n "${n}p" "$f")
+    # whatever follows the comma-separated code list on the same line
+    tail_=$(sed -E 's/.*shellcheck[[:space:]]+disable=[A-Za-z0-9,]+//' <<<"$line")
+    above=$(sed -n "$((n - 1))p" "$f")
+    if ! grep -qE '[A-Za-z]{3}' <<<"$tail_" && ! grep -qE '^[[:space:]]*#.*[A-Za-z]{3}' <<<"$above"; then
+      bare="$bare\n  $f:$n"
+    fi
+    # A directive shellcheck honours is always its own comment line, so anchor on that. Matching
+    # the bare phrase also picked up this file's own prose about the rule and the mutation
+    # payload in tests/gate-falsifiability.sh, and reported 9 suppressions where there are 6.
+  done <<<"$(grep -nE '^[[:space:]]*#[[:space:]]*shellcheck[[:space:]]+disable=' "$f" 2>/dev/null)"
+done <<<"$(git ls-files 2>/dev/null)"
+[ -z "$bare" ] \
+  && ok "shellcheck suppressions carry a reason ($nsup suppressions)" \
+  || bad "shellcheck suppressions carry a reason" \
+         "$(printf 'a bare disable hides the argument from the next reader:%b' "$bare")"
+
+# --- 31. every shipped file has a referrer -----------------------------------------------------
+#
+# Check 28 does this for docs/, where a link to the containing directory counts. Everything else
+# in the tree had no such rule, and two files had been riding along for versions: a launchd
+# wrapper around the doctor that install.sh never installs and uninstall.sh never removes, and an
+# eval-loop driver nothing but its own header mentioned. Neither was reachable and neither was
+# visible to any check. They are not named here on purpose -- a basename in this comment is a
+# referrer as far as the grep below is concerned, which is exactly how the first draft of this
+# check passed over both of them.
+#
+# The limit, stated rather than hidden: a mention in prose counts. This finds files nothing points
+# at, not files pointed at only rhetorically.
+#
+# Skills, agents, commands and issue templates are excluded because the loader finds them by
+# directory: a referrer would be redundant there, not missing. docs/ is excluded because check 28
+# owns it with directory-link semantics this basename match cannot express.
+unref=""
+while IFS= read -r f; do
+  [ -n "$f" ] || continue
+  case "$f" in
+    claude/skills/*|claude/agents/*|claude/commands/*|.github/ISSUE_TEMPLATE/*|docs/*) continue ;;
+  esac
+  [ -n "$(git grep -l -F "${f##*/}" -- . ":(exclude)$f" 2>/dev/null | head -1)" ] \
+    || unref="$unref\n  $f"
+done <<<"$(git ls-files 2>/dev/null)"
+[ -z "$unref" ] \
+  && ok "every shipped file has a referrer ($(git ls-files | grep -cvE '^(claude/(skills|agents|commands)/|\.github/ISSUE_TEMPLATE/|docs/)') outside the load-by-directory trees)" \
+  || bad "every shipped file has a referrer" \
+         "$(printf 'nothing in this repository names:%b\n  delete it, or give it a referrer -- a file nobody can find is a file nobody maintains' "$unref")"
+
 echo
 # Accounting. Every declared check must have reported either a result or a skip. A check
 # that throws a shell error mid-body, or is wrapped in a conditional with no else, silently

CHANGELOG.md

@@ -6,6 +6,83 @@ Versions follow [semver](https://semver.org). The version lives in two manifests
 
 ## Unreleased
 
+## 1.9.1 — 2026-08-22
+
+**A fresh bootstrap ended on a red line.** setup-machine.sh installs claude-mem, bin/doctor has
+checked that the plugin's UserPromptSubmit hooks are async for several versions, and nothing ever
+set the flag. So the lane installed the plugin and then left the machine in a state its own doctor
+called drift, telling the operator to re-apply something that had never been applied once.
+
+The install-matrix doctor-stranger case could not see it. That case exercises install.sh, and the
+plugin only arrives through setup-machine.sh, which only the bootstrap lane runs. It was found by
+running the README quickstart verbatim into a scratch HOME, which is the point of running it
+verbatim rather than reading it.
+
+setup-machine.sh now sets the flag, idempotently: it reads first and rewrites only when the flag
+is not already set, because claude-mem auto-updates rewrite hooks.json and revert it. Measured on
+a scratch HOME: doctor goes from one red line and DRIFT to 23 ok, 0 red, 6 notes.
+
+**doctor --drift deleted an unpushed release tag.** It ran a bare `git fetch` in the vstack
+checkout to work out how far behind the remote it was. A bare fetch is not read-only: it does
+whatever ~/.gitconfig says, and with fetch.prune and fetch.pruneTags set true it deletes every
+local tag and remote-tracking branch the remote does not have. During this audit it destroyed the
+v1.9.1 tag seconds after it was created, and the release check then reported ok for a version
+whose tag was already gone.
+
+Every flag is spelled out now, so ambient config cannot turn an inspection into an edit. The new
+doctor-no-mutate case in the install matrix clones a real checkout, sets that config pairing
+locally, plants an unpushed tag and asserts it survives. It carries three controls, because the
+first version of the case bailed before reaching the fetch and passed against the unfixed doctor.
+
+v1.9.0 is tagged and describes a payload carrying the claude-mem defect. It is left in place
+rather than moved, because a tag somebody may have fetched is not a thing to rewrite. Use v1.9.1.
+
+## 1.9.0 — 2026-08-22
+
+An audit pass. Every finding below is a green that measured nothing, which is the fifth time
+that class has shipped here, so each one leaves behind a check and a mutation row rather than
+just a fix.
+
+**Check 24 said ok over a comparison it never ran.** A version declared by the manifests but not
+yet tagged has no payload to diff against, and that branch printed ok. The tagless branch one
+elif below already knew better. It now skips with a reason, so the skip census can see it.
+
+**A pinned quickstart that 404s.** The README's "pin a release" lane pinned v1.8.0, the manifests
+said v1.8.0, and no such tag existed. The check compared the two strings, found them equal, and
+was satisfied. Measured: HTTP 404. A pin now has to name a tag that is actually there.
+
+**shellcheck was linting a hand-maintained list.** `git ls-files '*.sh' bin/doctor bin/vstack`
+never included bin/cloudflare-mcp, a #!/bin/sh script with no .sh suffix. An unquoted expansion
+appended to it made shellcheck exit 1 while the gate printed "ok shellcheck clean (29 scripts)".
+Selection is by shebang now, the way check 1 already did it, and the count is 30. Row 29 mutates
+that file specifically, so it proves the linter runs over everything rather than that it runs.
+
+**The count check dropped nouns on the floor.** `want_for()` resolved eight nouns and the
+extractor carried a separate grep alternation, so a claim could be extractable-but-unresolvable
+or the reverse, silently either way. Both come from one list now, with a positive control that
+fails if the extractor looks for a noun `want_for` cannot resolve. Adding "shell scripts"
+surfaced a stale CHANGELOG claim, and CHANGELOG's current-version section is now in the scan.
+
+**A suppression-reason rule that only lived in a comment.** Check 29's header had claimed it for
+several versions while bootstrap.sh carried a naked disable=SC2086. Check 30 enforces it.
+
+**Two files nothing pointed at.** A launchd wrapper around the doctor, which install.sh never
+installed and uninstall.sh never removed, and the eval-loop driver. The wrapper is deleted; the
+driver now has a real referrer in tests/README.md. Check 31 makes an unreferenced file a failure.
+
+**An uninstall that left Conductor pinning policy.** install.sh writes ~/.conductor/settings.toml
+and settings.managed.toml; uninstall.sh had no reference to conductor at all, so both survived
+removal, and the managed file is the one that pins models and plan mode.
+
+**The 141 that hid all of it.** tests/gate-falsifiability.sh probed for a check's skip with
+`verify.sh | grep -q`. Under `set -o pipefail` grep -q exits on the first match, verify dies of
+SIGPIPE, and the pipeline returns 141, which reads as "did not skip". Measured: rc=141 with
+pipefail, rc=0 without. Four sites now capture first and grep a here-string.
+
+The gate is 33 checks. Two of the new ones defeated themselves before they worked: naming a file
+in the check that hunts unnamed files gives it a referrer, and so does naming the probe in the
+mutation row.
+
 ## 1.8.0 — 2026-08-22
 
 **Two skill routings are mandatory now, not merely instructed.** Everything vstack did to route
@@ -26,7 +103,7 @@ names in CI and the platform names in the README to be the same set, in both dir
 Codex, not a local model behind a compatibility shim. Every mechanism here is Claude Code's own
 and there is no adapter layer.
 
-**shellcheck is a gate.** This bundle is 29 shell scripts and almost nothing else. Warning level,
+**shellcheck is a gate.** This bundle is shell scripts and almost nothing else. Warning level,
 and where a warning is wrong the suppression carries its reason on the line above. It found a
 pattern in the destructive guard that could never match, a variable in `doctor` computed for a
 check nobody ever wrote, and two dead assignments.