One failed fetch retired vstack update on a pinned install #181
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: verify | |
| on: | |
| push: | |
| branches: [main] | |
| pull_request: | |
| # Re-runnable by hand against whatever main points at now. Without this, the only way to get a | |
| # fresh verdict for a commit already on main was `gh run rerun <id>` against one specific past | |
| # run -- which is not available while that run is still in flight, and re-checks out the tree | |
| # at the moment the rerun starts. On 2026-08-27 that cost two deleted tags: check 24 and | |
| # bin/doctor's fetchable check are both required to be red until the release tag is on origin, | |
| # so the tag has to land and the checks have to be asked again, in that order. Being able to | |
| # ask again is the whole operation. | |
| workflow_dispatch: | |
| jobs: | |
| verify: | |
| runs-on: ubuntu-latest | |
| steps: | |
| # fetch-depth 0 because check 24 compares the installable payload against the tag whose | |
| # version the manifests declare, and the default shallow checkout carries no tags at all. | |
| # Without this the check cannot measure and skips, which is honest but proves nothing on | |
| # the one machine that gates every push. | |
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | |
| with: | |
| fetch-depth: 0 | |
| # The gate needs jq for the JSON checks and the install dry run. shellcheck backs check 29, | |
| # which skips without it -- and a check that skips on the one machine gating every push is | |
| # a green measuring nothing, which is the failure mode this repository keeps rediscovering. | |
| # `shellcheck --version` runs so a broken install stops the run here rather than being | |
| # found later as a check that quietly proved nothing. | |
| - name: Install jq and shellcheck | |
| run: | | |
| sudo apt-get update -qq && sudo apt-get install -y -qq jq shellcheck | |
| shellcheck --version | |
| # tests/dispatch-static.sh parses every skill fixture in the language the fixture claims | |
| # to be. Before 1.46.0 a missing parser was a silent SKIP that still printed ok, so 28 of | |
| # 35 fixtures were never parsed on a runner without these four binaries -- the suite's own | |
| # fake green. It now FAILs naming the missing validator, which means CI must actually | |
| # carry them. `--version` on each so a broken install stops here rather than being read | |
| # later as a fixture defect. | |
| - name: Install fixture validators (tsc, csslint, xmllint, gofmt) | |
| run: | | |
| set -eu | |
| sudo apt-get install -y -qq libxml2-utils | |
| npm install -g typescript@5 csslint@1 | |
| tsc --version | |
| csslint --version | |
| xmllint --version | |
| # gofmt ships with the Go toolchain, preinstalled on ubuntu-latest. Assert rather than | |
| # assume: if a future image drops it, fail here naming it instead of in the suite. | |
| gofmt --help >/dev/null 2>&1 || { echo "gofmt not on PATH"; exit 1; } | |
| # Check 19 runs `claude plugin validate --strict`; without a working CLI it skips, and the | |
| # falsifiability suite skips its case 19 with it. Install it so CI proves the real thing. | |
| # | |
| # `npm install -g` alone was not enough and produced a false green for as long as the step | |
| # has existed. The npm package is a shim that fetches a platform-native binary in its | |
| # postinstall; that did not land here, so every `claude plugin validate` exited non-zero | |
| # with "claude native binary not installed" — and check 19's old output parsing read that | |
| # error as no findings and printed ok. CI has never once validated these manifests. | |
| # | |
| # So install it, then prove the binary answers before trusting the step. `claude | |
| # --version` failing here is a setup failure and should stop the run, not be discovered | |
| # later as a check that silently measures nothing. | |
| - name: Install Claude Code CLI | |
| run: | | |
| set -u | |
| # The npm package is a shim; the real binary arrives as a platform-specific optional | |
| # dependency. Installing the shim alone produced a false green for as long as this | |
| # step existed: the optional dep never landed, every `claude plugin validate` exited | |
| # with "native binary not installed", and check 19's old output parsing read that | |
| # error as "no findings" and printed ok. CI validated these manifests exactly zero | |
| # times before this was found. | |
| # | |
| # The reason the optional dep never landed is worth naming, because it will recur: | |
| # the shim pins its platform deps to its own version, and the platform packages are | |
| # published on a lag. At the time of writing the shim was 2.1.237 and linux-x64 was | |
| # 2.1.236, so npm resolved nothing and said nothing. Ask for the platform package by | |
| # name at whatever version is actually published, and use the binary it ships — it is | |
| # self-contained and needs no shim. | |
| npm install -g @anthropic-ai/claude-code@2.1.243 || true | |
| case "$(uname -s)-$(uname -m)" in | |
| Linux-x86_64) PKG=linux-x64 ;; | |
| Linux-aarch64) PKG=linux-arm64 ;; | |
| Darwin-arm64) PKG=darwin-arm64 ;; | |
| Darwin-x86_64) PKG=darwin-x64 ;; | |
| *) PKG="" ;; | |
| esac | |
| if [ -n "$PKG" ]; then | |
| PREFIX="$RUNNER_TEMP/ccbin" | |
| npm install --prefix "$PREFIX" "@anthropic-ai/claude-code-$PKG@2.1.243" || true | |
| BIN="$PREFIX/node_modules/@anthropic-ai/claude-code-$PKG/claude" | |
| if [ -x "$BIN" ] && "$BIN" --version >/dev/null 2>&1; then | |
| sudo ln -sf "$BIN" /usr/local/bin/claude | |
| fi | |
| fi | |
| # Report rather than fail. Check 19 runs its own two-way control and skips with a | |
| # reason when the validator is unusable, so an unavailable CLI degrades to a visible | |
| # skip in the gate output instead of a red run over something this repo does not own. | |
| claude --version || echo "::warning::claude CLI unusable on this runner; check 19 will skip" | |
| - name: The manifests were actually validated, not skipped | |
| run: | | |
| # Check 19 is allowed to skip, so a green gate does not prove it ran. On this runner | |
| # it must run: that is the whole point of installing the CLI above, and a silent | |
| # regression back to skipping is exactly how it went unnoticed before. | |
| ./.claude/verify.sh > "$RUNNER_TEMP/gate.txt" 2>&1 || true | |
| if grep -q '^skip plugin manifests valid' "$RUNNER_TEMP/gate.txt"; then | |
| grep '^skip plugin manifests valid' "$RUNNER_TEMP/gate.txt" | |
| echo "check 19 skipped on a runner where the CLI was installed on purpose" | |
| exit 1 | |
| fi | |
| grep '^ok plugin manifests valid' "$RUNNER_TEMP/gate.txt" | |
| # This lane has every validator installed on purpose (jq, shellcheck, the CLI), so the | |
| # only skip it should ever see is the version-tag comparison before the commit that gets | |
| # tagged. Reuses the capture from the step above rather than running the gate a third | |
| # time in one job. | |
| - name: No unexpected skips in this lane's gate output | |
| run: | | |
| bash .github/scripts/require-no-unexpected-skips.sh "$RUNNER_TEMP/gate.txt" \ | |
| "declared version matches what installs" | |
| - name: Run the verification gate | |
| run: ./.claude/verify.sh | |
| # A gate that cannot fail proves nothing. This used to break one skill description and | |
| # call it done, which left the other checks unproven — and two of them were not enforcing | |
| # anything at the time. Every declared check now gets its own mutation, and check 16 of | |
| # the gate fails if a check is added without one. | |
| - name: Prove every check is falsifiable | |
| run: ./tests/gate-falsifiability.sh | |
| # The comparison the README cites. It asserts the decision each mechanism produces, so a | |
| # regression turns it red rather than quietly changing a number in a table nobody re-runs. | |
| - name: The mechanisms still decide what the README says they do | |
| run: ./tests/compare-baseline.sh | |
| # A standalone reproduction of the delegation mandate in skill-mandate.sh, offline and | |
| # under 2s. It is not a pass/fail harness on its own -- check 27 of the gate above is the | |
| # thing that actually enforces the four cases it drives -- but it must at least run clean | |
| # so its raw stdout/exit code stays trustworthy for anyone reaching for it by hand. | |
| - name: skill-mandate.sh breadth reproduction runs clean | |
| run: ./tests/test-breadth-mandate.sh | |
| # Zero model calls, fixture integrity only. auto-trigger.sh and team-gating.sh need a | |
| # billed, authenticated `claude` session the user has declined to give CI, so this is the | |
| # one thing about skill dispatch this pipeline CAN check for free: that the fixtures those | |
| # suites depend on (skill names, setup_* functions, the files they write, the case count | |
| # their own docs claim) are still intact. See the header of dispatch-static.sh for what | |
| # this does and does not prove -- it does not prove a skill fires. | |
| - name: auto-trigger.sh fixtures are still intact (no model calls) | |
| run: ./tests/dispatch-static.sh | |
| # The three bin/ scripts install.sh ships into every user's PATH (claude-bg.sh, | |
| # claude-task.sh, deploy-auto.sh) were syntax-checked by checks 1 and 29 of the gate but | |
| # never actually run before this suite existed -- see tests/bin-scripts.sh's own header. | |
| # Zero model calls: every `claude` this suite touches is a local stub. | |
| # The release gate's check-run selection, exercised against fixtures. It runs the real | |
| # .github/scripts/latest-check-run.jq rather than a copy of the rule, because a test that | |
| # restates the logic it tests agrees with itself forever. The defect it was written for | |
| # shipped in this repository: the selection sorted on a key that was constant across the | |
| # set it sorted, so it returned whatever order the API happened to use. | |
| - name: tests/require-checks-green.sh proves the release gate picks the right check-run | |
| run: ./tests/require-checks-green.sh | |
| - name: tests/bin-scripts.sh runs the shipped bin/ scripts, not just lints them | |
| run: ./tests/bin-scripts.sh | |
| - name: Confirm the tree is restored | |
| run: git diff --exit-code | |
| # Everything above runs the gate. This installs for real, on Linux, into a disposable HOME. | |
| # It exists because the worst bug this repo has shipped was Linux-only and invisible on the | |
| # machine it was written on: hooks called /usr/bin/jq by absolute path, which is a macOS | |
| # path, so off macOS the verify gate silently stopped blocking and the session hook injected | |
| # nothing at all. A dry run cannot catch that. Only installing and firing the hooks can. | |
| install-linux: | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | |
| - name: Install jq | |
| run: sudo apt-get update -qq && sudo apt-get install -y -qq jq | |
| # Runs install.sh into throwaway HOMEs and asserts the resulting tree each time: a plain | |
| # install, CLAUDE_CONFIG_DIR pointed elsewhere, a home path with a space, jq removed from | |
| # PATH, a bash user, a second run for idempotency, both uninstall paths, and an install | |
| # over a home that already holds somebody else's skills, settings and MCP servers. Two | |
| # further cases exercise the curl bootstrap and the plugin marketplace against the | |
| # published repo. This runner is the environment the matrix exists for — bash by default, | |
| # no Homebrew, the shape of a cloud VM or a container. | |
| - name: Install Claude Code CLI | |
| run: | | |
| set -u | |
| # The npm package is a shim; the real binary arrives as a platform-specific optional | |
| # dependency. Installing the shim alone produced a false green for as long as this | |
| # step existed: the optional dep never landed, every `claude plugin validate` exited | |
| # with "native binary not installed", and check 19's old output parsing read that | |
| # error as "no findings" and printed ok. CI validated these manifests exactly zero | |
| # times before this was found. | |
| # | |
| # The reason the optional dep never landed is worth naming, because it will recur: | |
| # the shim pins its platform deps to its own version, and the platform packages are | |
| # published on a lag. At the time of writing the shim was 2.1.237 and linux-x64 was | |
| # 2.1.236, so npm resolved nothing and said nothing. Ask for the platform package by | |
| # name at whatever version is actually published, and use the binary it ships — it is | |
| # self-contained and needs no shim. | |
| npm install -g @anthropic-ai/claude-code@2.1.243 || true | |
| case "$(uname -s)-$(uname -m)" in | |
| Linux-x86_64) PKG=linux-x64 ;; | |
| Linux-aarch64) PKG=linux-arm64 ;; | |
| Darwin-arm64) PKG=darwin-arm64 ;; | |
| Darwin-x86_64) PKG=darwin-x64 ;; | |
| *) PKG="" ;; | |
| esac | |
| if [ -n "$PKG" ]; then | |
| PREFIX="$RUNNER_TEMP/ccbin" | |
| npm install --prefix "$PREFIX" "@anthropic-ai/claude-code-$PKG@2.1.243" || true | |
| BIN="$PREFIX/node_modules/@anthropic-ai/claude-code-$PKG/claude" | |
| if [ -x "$BIN" ] && "$BIN" --version >/dev/null 2>&1; then | |
| sudo ln -sf "$BIN" /usr/local/bin/claude | |
| fi | |
| fi | |
| # Report rather than fail. Check 19 runs its own two-way control and skips with a | |
| # reason when the validator is unusable, so an unavailable CLI degrades to a visible | |
| # skip in the gate output instead of a red run over something this repo does not own. | |
| claude --version || echo "::warning::claude CLI unusable on this runner; check 19 will skip" | |
| - name: Install matrix (all environments, plus the network lanes) | |
| run: ./tests/install-matrix.sh | |
| # The marketplace case is allowed to skip when there is no CLI, which meant it skipped in | |
| # every job: the CLI was installed only in the verify job, and the matrix ran only in the | |
| # others. One of the three advertised install lanes was therefore unproven while every | |
| # required check stayed green. The CLI is installed above on purpose, so a skip here is a | |
| # regression, not an environment fact. | |
| - name: The marketplace lane actually ran | |
| run: | | |
| ./tests/install-matrix.sh marketplace > "$RUNNER_TEMP/mkt.txt" 2>&1 || true | |
| cat "$RUNNER_TEMP/mkt.txt" | |
| if grep -q '^skip plugin marketplace lane' "$RUNNER_TEMP/mkt.txt"; then | |
| echo "marketplace case skipped on a runner where the CLI was installed on purpose" | |
| exit 1 | |
| fi | |
| grep -q '^ok plugin marketplace lane' "$RUNNER_TEMP/mkt.txt" | |
| - name: install.sh, for real | |
| run: ./install.sh | |
| - name: The installed tree matches the repo | |
| run: ./bin/doctor --drift | |
| # Expected counts are derived from the repo at run time, not hardcoded — a literal here | |
| # went stale the first time a skill was added and failed a green tree. | |
| - name: Everything landed | |
| run: | | |
| set -e | |
| count() { ls -d $1 2>/dev/null | wc -l | tr -d ' '; } | |
| for spec in "$HOME/.claude/skills/*/:claude/skills/*/" \ | |
| "$HOME/.claude/agents/*.md:claude/agents/*.md" \ | |
| "$HOME/.claude/commands/*.md:claude/commands/*.md" \ | |
| "$HOME/.claude/hooks/*.sh:claude/hooks/*.sh"; do | |
| pat="${spec%:*}"; src="${spec##*:}" | |
| got=$(count "$pat"); want=$(count "$src") | |
| [ "$got" = "$want" ] && [ "$want" != 0 ] || { echo "$pat: got $got, repo has $want"; exit 1; } | |
| echo "ok $pat -> $got (repo: $want)" | |
| done | |
| test -x "$HOME/.claude/hooks/verify-gate.sh" | |
| - name: The Stop gate blocks on Linux, with and without jq on PATH | |
| run: | | |
| set -e | |
| mkdir -p /tmp/g/repo/.claude /tmp/g/home/.config/agents /tmp/g/tmp /tmp/g/bin | |
| printf '#!/usr/bin/env bash\necho "seeded failure"\nexit 1\n' > /tmp/g/repo/.claude/verify.sh | |
| chmod +x /tmp/g/repo/.claude/verify.sh | |
| printf '%s %s\n' "$(sha256sum /tmp/g/repo/.claude/verify.sh | cut -d' ' -f1)" \ | |
| /tmp/g/repo/.claude/verify.sh > /tmp/g/home/.config/agents/verify-trust | |
| for t in bash sh cat cut grep sed awk tr rm mkdir env dirname basename sha256sum; do | |
| ln -sf "$(command -v $t)" /tmp/g/bin/$t | |
| done | |
| out=$(printf '{"session_id":"ci"}' | env HOME=/tmp/g/home TMPDIR=/tmp/g/tmp \ | |
| CLAUDE_PROJECT_DIR=/tmp/g/repo bash claude/hooks/verify-gate.sh) | |
| echo "$out" | jq -e '.decision=="block"' >/dev/null || { echo "did not block with jq"; exit 1; } | |
| echo "ok blocks with jq" | |
| sed 's#/usr/bin/jq#/nonexistent/jq#' claude/hooks/verify-gate.sh > /tmp/g/nojq.sh | |
| out=$(printf '{"session_id":"ci2"}' | env PATH=/tmp/g/bin HOME=/tmp/g/home \ | |
| TMPDIR=/tmp/g/tmp CLAUDE_PROJECT_DIR=/tmp/g/repo bash /tmp/g/nojq.sh) | |
| echo "$out" | jq -e '.decision=="block"' >/dev/null || { echo "did not block without jq"; exit 1; } | |
| echo "ok blocks with no jq reachable" | |
| - name: The session hook injects the routing block on Linux | |
| run: | | |
| set -e | |
| n=$(printf '{"hook_event_name":"SessionStart"}' | bash claude/hooks/inject-session-context.sh | wc -c) | |
| [ "$n" -gt 2000 ] || { echo "session hook injected only $n bytes"; exit 1; } | |
| echo "ok session hook injected $n bytes" | |
| - name: uninstall.sh --dry-run | |
| run: ./uninstall.sh --dry-run | |
| # macOS is the platform this repo calls primary, and until now CI never touched it. Every | |
| # macOS-only assumption — BSD sed and find, `shasum` instead of `sha256sum`, `stat -f` — was | |
| # only ever exercised on the author's laptop, which is the definition of works-on-my-machine. | |
| install-macos: | |
| runs-on: macos-latest | |
| steps: | |
| # fetch-depth 0 for the same reason as the verify job above: without it check 24 | |
| # ("declared version matches what installs") cannot see any tags and reports the "shallow | |
| # clone" skip, which read as an environment fact for as long as this checkout omitted it | |
| # and was really just this job never asking for the tags it needed. | |
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | |
| with: | |
| fetch-depth: 0 | |
| - name: Install matrix on macOS | |
| run: ./tests/install-matrix.sh | |
| # check 29 ("shellcheck clean") skips without a validator on PATH, and a skip is not a pass. | |
| # macos-latest stopped shipping shellcheck, so this lane silently stopped linting all 71 | |
| # scripts on the one platform whose BSD tools this job exists to exercise. Installed here | |
| # rather than added to the approved-skip list below, for the same reason the alpine lane | |
| # installs it: approving the skip would make the check disappear and the lane still green. | |
| - name: shellcheck for macOS, or fail naming it | |
| run: | | |
| brew install shellcheck | |
| shellcheck --version | |
| # NOT piped into tee. Under this workflow's default `bash -e` shell there is no pipefail, so | |
| # `verify.sh | tee` returns tee's status and a red gate exits 0. Measured on 2026-08-27: | |
| # `bash -e -c 'red-gate | tee f'` exits 0, the same command unpiped exits 1, and the skip | |
| # audit below exits 0 on a log full of FAIL lines because it only reads skip lines. A failing | |
| # gate on this lane has therefore never failed this job. The redirect keeps the log for the | |
| # skip audit, `|| rc=$?` keeps `-e` from exiting before the code is read, and the exit is on | |
| # its own line where it can be seen. | |
| - name: Run the verification gate | |
| run: | | |
| rc=0 | |
| ./.claude/verify.sh > "$RUNNER_TEMP/gate-macos.txt" 2>&1 || rc=$? | |
| cat "$RUNNER_TEMP/gate-macos.txt" | |
| exit "$rc" | |
| # This lane declares, out loud, the only two checks it accepts as skips: the CLI is not | |
| # installed here on purpose (only the ubuntu `verify` job proves check 19 runs, per the | |
| # comment on that job's own CLI-install step), and the version-tag comparison is expected | |
| # to skip on every commit before the one that gets tagged. Anything else that skips here | |
| # is a regression this lane is supposed to catch, not an environment fact to shrug at. | |
| - name: No unexpected skips on macOS | |
| run: | | |
| bash .github/scripts/require-no-unexpected-skips.sh "$RUNNER_TEMP/gate-macos.txt" \ | |
| "plugin manifests valid" \ | |
| "declared version matches what installs" | |
| # Alpine, because BusyBox is not GNU. Its sed, grep, find and stat take different flags, and | |
| # there is no bash until you install it — which is exactly the shape of a slim container | |
| # image. Nothing else in this workflow would catch a GNU-ism. | |
| install-alpine: | |
| runs-on: ubuntu-latest | |
| container: alpine:latest | |
| steps: | |
| - name: Tools a slim image does not ship | |
| run: apk add --no-cache bash git jq curl findutils coreutils grep sed nodejs npm | |
| # check 29 ("shellcheck clean") skips without a validator on PATH, and a check that skips | |
| # on a lane specifically kept around to catch BusyBox-vs-GNU differences is a green | |
| # measuring nothing on the one image most likely to carry a GNU-ism shellcheck would | |
| # catch. Install it for real and fail here, by name, if the package this image ships does | |
| # not resolve -- not a silent fallback to running without it. | |
| - name: shellcheck for a slim image, or fail naming it | |
| run: | | |
| apk add --no-cache shellcheck | |
| shellcheck --version | |
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | |
| with: | |
| fetch-depth: 0 | |
| # The container runs as root against a checkout owned by another uid, so git refuses to | |
| # operate on it at all. That is a property of running in a container, not of this repo, | |
| # but it fails loudly and confusingly: overlay.sh dies mid-run and check 17 reports the | |
| # overlay as broken. Declare the checkout safe so the gate measures the repo rather than | |
| # the runner's file ownership. | |
| - name: Let git operate on a checkout root does not own | |
| run: git config --global --add safe.directory "$GITHUB_WORKSPACE" | |
| - name: What this container actually has | |
| run: | | |
| bash --version | head -1 | |
| for t in jq git sed find cp chmod mktemp sha256sum shasum; do | |
| printf '%-12s %s\n' "$t" "$(command -v $t || echo MISSING)" | |
| done | |
| - name: Install matrix on Alpine | |
| run: bash ./tests/install-matrix.sh | |
| - name: Run the verification gate | |
| # Not piped, for the reason spelled out on the macOS lane above: no pipefail under this | |
| # shell means `gate | tee` reports tee's status and a red gate passes. | |
| run: | | |
| rc=0 | |
| bash ./.claude/verify.sh > "$RUNNER_TEMP/gate-alpine.txt" 2>&1 || rc=$? | |
| cat "$RUNNER_TEMP/gate-alpine.txt" | |
| exit "$rc" | |
| # Same declaration as the macOS lane: the CLI is deliberately absent here too, and the | |
| # version-tag check is expected to skip pre-release. shellcheck is no longer on this list | |
| # -- the step above installs it for real, so a skip of check 29 here now means the | |
| # install silently stopped working, and that must fail the lane, not slide past it. | |
| - name: No unexpected skips on Alpine | |
| run: | | |
| bash .github/scripts/require-no-unexpected-skips.sh "$RUNNER_TEMP/gate-alpine.txt" \ | |
| "plugin manifests valid" \ | |
| "declared version matches what installs" |