Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changes/animation/test-selection-audit-gaps-d611396c.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
bump: none
---

No release needed (test): declare the files 22 suites read that the selector missed
5 changes: 5 additions & 0 deletions .changes/bash-format/test-selection-audit-gaps-34742c5b.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
bump: none
---

No release needed (test): declare the files 22 suites read that the selector missed
5 changes: 5 additions & 0 deletions .changes/code-metrics/test-selection-audit-gaps-d85a009a.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
bump: none
---

No release needed (test): declare the files 22 suites read that the selector missed
5 changes: 5 additions & 0 deletions .changes/evals/test-selection-audit-gaps-13d22281.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
bump: none
---

No release needed (test): declare the files 22 suites read that the selector missed
5 changes: 5 additions & 0 deletions .changes/guardrails/test-selection-audit-gaps-e7b7681c.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
bump: none
---

No release needed (test): declare the files 22 suites read that the selector missed
5 changes: 5 additions & 0 deletions .changes/planning/test-selection-audit-gaps-1b9c49e4.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
bump: none
---

No release needed (test): declare the files 22 suites read that the selector missed
5 changes: 5 additions & 0 deletions .changes/review/test-selection-audit-gaps-d928980e.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
bump: none
---

No release needed (test): declare the files 22 suites read that the selector missed
5 changes: 5 additions & 0 deletions .changes/session-flow/test-selection-audit-gaps-4e443754.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
bump: none
---

No release needed (test): declare the files 22 suites read that the selector missed
5 changes: 5 additions & 0 deletions .changes/source-control/test-selection-audit-gaps-211b5d17.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
bump: none
---

No release needed (test): declare the files 22 suites read that the selector missed
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
#!/usr/bin/env bash
# test-scope: .github/*.cjs .github/*.mjs .github/*.sh .github/*.bash .github/workflows/*.yml .github/actions/*/action.yml
# test-scope: .github/actionlint.yaml .github/dependabot.yml .github/actions/resolve-config/fixtures/*.yaml .github/scripts/intake-triage/reentry-eval/cases/*/case.yaml
# test-scope: plugins/*.js plugins/*.cjs plugins/*.mjs plugins/*.sh plugins/*.yml plugins/*.yaml plugins/*.py .github/*.py
# Owns graphql-operations.test.cjs, which reads every tracked source under
# .github/ and plugins/ and loads graphql from the root node_modules, so CI
# runs it in test-bash after the root npm ci. Under .github/ the YAML scope is
# the workflows and composite actions, the YAML that runs; the rest is config
# and fixtures, which stay a no-suite class.
# the workflows, composite actions, and the config and fixture files the
# selection audit traced it reading; other .github YAML stays a no-suite class.
set -euo pipefail
exec node --test "$(dirname "${BASH_SOURCE[0]}")/graphql-operations.test.cjs"
4 changes: 2 additions & 2 deletions .github/workflows/maintenance-audit-test-selection.yml
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ jobs:
SHFMT_SHA256: 76e77641faa025814b77f153b29796b8e6fa2fca03e0c76a691608b86c7ea7bf
run: |
mkdir -p "$HOME/.local/bin"
curl -fsSL -o "$HOME/.local/bin/shfmt" \
curl -fsSL --retry 7 --retry-all-errors -o "$HOME/.local/bin/shfmt" \
"https://github.com/mvdan/sh/releases/download/$SHFMT_VERSION/shfmt_${SHFMT_VERSION}_linux_amd64"
echo "$SHFMT_SHA256 $HOME/.local/bin/shfmt" | sha256sum -c -
chmod +x "$HOME/.local/bin/shfmt"
Expand All @@ -118,7 +118,7 @@ jobs:
DUCKDB_SHA256: c61f21485e6e41d3a0c28ce9904ea18346309cf427b4cf9479bc3564348dc885
run: |
mkdir -p "$RUNNER_TEMP/duckdb"
curl -fsSL -o "$RUNNER_TEMP/duckdb/duckdb.gz" \
curl -fsSL --retry 7 --retry-all-errors -o "$RUNNER_TEMP/duckdb/duckdb.gz" \
"https://github.com/duckdb/duckdb/releases/download/$DUCKDB_VERSION/duckdb_cli-linux-amd64.gz"
echo "$DUCKDB_SHA256 $RUNNER_TEMP/duckdb/duckdb.gz" | sha256sum -c -
gunzip "$RUNNER_TEMP/duckdb/duckdb.gz"
Expand Down
1 change: 1 addition & 0 deletions lib/exec-bash.seed.test.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# test-scope: lib/hook-utils.sh
# Owns lib/exec-bash.seed.test.mjs for the plugin test lane.
# scripts/run-outside-node-suites.sh treats a sibling .test.sh as the runner.
set -euo pipefail
Expand Down
1 change: 1 addition & 0 deletions lib/session-bridge/test_session_bridge.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# test-scope: lib/session-bridge/view_bridge.py
# test-scope: lib/session-bridge/watch.sh lib/session-bridge/wake.sh
"""Tests for session-bridge: the Transport port, the loopback adapter's server and client halves,
and watch.sh and wake.sh against a toy app built on the bridge.
Expand Down
1 change: 1 addition & 0 deletions plugins/animation/scripts/test_render.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# test-scope: plugins/animation/.claude-plugin/plugin.json
"""render.py and capture.mjs over a real headless render. Needs node and playwright-core with Chromium (the
capture.mjs --probe lookup) and skips without them, or fails with ANIMATION_REQUIRE_DEPS=1."""
import json
Expand Down
1 change: 1 addition & 0 deletions plugins/bash-format/git-hooks/shell-check.test.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# test-scope: plugins/bash-format/.claude-plugin/plugin.json
# Contract tests for the bash-format pre-commit entry (git-hooks/shell-check.sh):
# staged shell scripts that an .editorconfig section governs are formatted and
# re-staged, a partly staged one is refused untouched, a ShellCheck finding blocks
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# test-scope: plugins/code-metrics/scripts/yaml_subset.py
# Regression tests for the audit-duplication entry point
# (audit-duplication.sh): the sanctioned-replication exclusion, the tunables it
# exports for the collector adapters, option parsing, and exit codes.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
# test-scope: plugins/*/evals/*/case.yaml plugins/*/evals/*/matchers.json plugins/*/evals/*/samples/*
"""Tests for check-matchers.py: rendering, drift, sample registration, both
regex engines, and the live fragments file with every suite in the repository."""

Expand Down
1 change: 1 addition & 0 deletions plugins/guardrails/git-hooks/pre-push-content.test.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# test-scope: plugins/guardrails/hooks/*.sh plugins/guardrails/hooks/yaml-subset.awk plugins/guardrails/lib/verification/*.sh
# Contract tests for the guardrails pre-push entry git-hooks/pre-push-content.sh:
# a secret or machine-specific path that a pushed commit adds blocks with
# "<path>:<line>: <message> (commit <id>)", also when a later commit of the same
Expand Down
1 change: 1 addition & 0 deletions plugins/guardrails/hooks/git-hooks-notice.test.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# test-scope: plugins/guardrails/git-hooks/content-scan.sh plugins/guardrails/git-hooks/pre-push-content.sh plugins/guardrails/hooks/yaml-subset.awk
# The SessionStart probe that names /guardrails:setup apply install-git-hooks when this clone
# lacks the guardrails git-hook entries: run as harness-ops' prerequisites-fleet.mjs runs it
# (this plugin's own lib/prerequisites.mjs probe), in a fresh clone and with every entry
Expand Down
1 change: 1 addition & 0 deletions plugins/guardrails/hooks/onfailure-rows.test.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# test-scope: plugins/*/hooks/hooks.json
# Pins which hooks.json rows set "onFailure" (#9982). The hook-budget contract,
# "Which rows set onFailure: block", names exactly the rows that set the key;
# every other row, the advisory and SessionStart rows included, leaves it unset.
Expand Down
1 change: 1 addition & 0 deletions plugins/planning/shared/tests/align-c.test.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# test-scope: plugins/planning/shared/*.py
# Part c of the page alignment check, the sizes align.js PARTS puts in c; align-a.test.sh and align-b.test.sh run
# the rest, and align-run.sh runs each part.
# bash plugins/planning/shared/tests/align-c.test.sh
Expand Down
1 change: 1 addition & 0 deletions plugins/planning/shared/tests/tracer.test.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# test-scope: plugins/planning/surface/schema/questions.schema.json
# The shared session end to end, CI-safe: the fake cloudflared stands in for the tunnel.
# bash tracer.test.sh
# ensure-running -> two guests join over HTTP -> apply an open-round -> both answer -> the owner
Expand Down
1 change: 1 addition & 0 deletions plugins/planning/surface/test_interview_server.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# test-scope: plugins/planning/lib/resolve-convention-home.sh plugins/planning/shared/schema/*.schema.json
# test-scope: plugins/planning/surface/tests/fixtures/*.json
# test-scope: plugins/planning/surface/schema/*.schema.json plugins/planning/.claude-plugin/plugin.json
"""Tests for the interview surface server and its lifecycle commands.
Expand Down
1 change: 1 addition & 0 deletions plugins/planning/tests/decision-levels.test.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# test-scope: plugins/planning/skills/*/SKILL.md
# Tripwire contract test: each planning stage names the one level it decides
# (behavior, interfaces and contracts, files and symbols). This test fails
# loudly if a prose edit drops, duplicates or swaps one of those lines.
Expand Down
1 change: 1 addition & 0 deletions plugins/review/skills/setup/scripts/setup-apply.test.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# test-scope: plugins/review/.claude-plugin/plugin.json plugins/review/lib/yaml-subset.awk
# Tests for setup-apply.mjs: the narrow write of docs/conventions/review.yaml.
# Each case builds its own repository root in a temp directory; expected values
# come from the schema's allowed values (run, report; lens, model, codex; a whole number of at
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# test-scope: plugins/session-flow/skills/workflow/SKILL.md
"""Tripwire: the workflow skill's stage list and detection list agree with steps.md, interview first."""

from __future__ import annotations
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# test-scope: .github/workflows/*.yml
# test-scope: plugins/source-control/lib/parse-concern-value.sh
"""The post-merge check and its remediation (`babysit_postmerge`).

Expand Down
1 change: 1 addition & 0 deletions scripts/check-data-root.test.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# test-scope: docs/conventions/attribution.yaml
# Contract test for CHECK_DATA_ROOT in the spoke-plugin-root, plugin-options-docs,
# changelog-parity and cross-plugin-source-drift checks: each reads the tree the
# variable names, not the one holding the script. scripts/check-affected.sh relies
Expand Down
1 change: 1 addition & 0 deletions scripts/check-script-contract.test.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# test-scope: docs/upstream/*.md
# Contract suite for the scripts/check-*.sh family: the part a CALLER must know
# is the same for every member, and this is what holds it.
#
Expand Down
1 change: 1 addition & 0 deletions scripts/ci-queued-pr.test.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# test-scope: .github/actions/log-api-budget/api-budget.bash
# Runs the `Resolve the queued pull request` step body of
# .github/workflows/pr-require-checks.yml under bash -e, against stub `gh` and
# `git`.
Expand Down
1 change: 1 addition & 0 deletions scripts/post-merge-demotion-alert.test.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# test-scope: .github/scripts/post-merge-demotion-alert/newer-green.bash
# Runs the `File or comment on the demotion issue` step body of
# .github/workflows/post-merge-demotion-alert.yml under the shell Actions gives
# a step with no `shell:` (bash -e), against a stub `gh`.
Expand Down
6 changes: 4 additions & 2 deletions scripts/selection-audit.sh
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,8 @@
# Four root files are read by almost every suite through its tools, not its
# logic: node reads package.json for the module type, the version manager reads
# .node-version, and git reads .gitignore and .gitattributes. A read of one is no gap; a change to
# one is the toolchain's to gate, and as a scope it would select every suite.
# one is the toolchain's to gate, and as a scope it would select every suite. The same holds for a
# .gitignore or .gitattributes in any directory, which git reads when a suite walks the tree.
# So is .github/npm-fetch.npmrc: npm reads it at startup in any suite that runs
# npm or npx, through npm_config_globalconfig, and callers.test.mjs gates it.
#
Expand Down Expand Up @@ -148,7 +149,8 @@ trace_one() {
# -k: a daemon a suite leaves behind stays traced, and strace outlives a TERM waiting for it.
timeout -k 10 600 strace -f -y -z -qq --seccomp-bpf -e trace=open,openat -e signal=none \
-o "$out/strace.$key" "${runner[@]}" >"$out/logs/$key.log" 2>&1 || rc=$?
reads "$out/strace.$key" "$ROOT" | grep -vxF -e "$suite" -e package.json -e .node-version -e .gitignore -e .gitattributes -e .github/npm-fetch.npmrc |
reads "$out/strace.$key" "$ROOT" | grep -vxF -e "$suite" -e package.json -e .node-version -e .github/npm-fetch.npmrc |
grep -vE '(^|/)\.git(ignore|attributes)$' |
Comment thread
kyle-sexton marked this conversation as resolved.
Resolved
awk 'NR == FNR { t[$0] = 1; next } t[$0]' "$out/tracked" - >"$out/reads/$key"
rm -f "$out/strace.$key"
printf '%s\t%s\t%s\t%s\n' "$suite" "$rc" "$(($(date +%s) - start))" "$(wc -l <"$out/reads/$key")" >"$out/suites/$key"
Expand Down
7 changes: 5 additions & 2 deletions scripts/selection-audit.test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -81,10 +81,13 @@ else
echo 22 >"$t/.node-version"
echo '*.log' >"$t/.gitignore"
echo '* text=auto' >"$t/.gitattributes"
# So are the nested copies git reads when a suite walks the tree.
echo '*.tmp' >"$t/data/.gitignore"
echo '*.bin binary' >"$t/data/.gitattributes"
mkdir -p "$t/.github"
echo 'fetch-retries=1' >"$t/.github/npm-fetch.npmrc"
printf '%s\n' '#!/usr/bin/env bash' 'cat data/hidden.txt data/recorded.txt data/named.txt data/unmapped.txt data/untracked.txt data/broken.txt' \
'cat package.json .node-version .gitignore .gitattributes .github/npm-fetch.npmrc' >"$t/suites/reader.test.sh"
'cat package.json .node-version .gitignore .gitattributes data/.gitignore data/.gitattributes .github/npm-fetch.npmrc' >"$t/suites/reader.test.sh"
printf '%s\n' '#!/usr/bin/env bash' 'true' >"$t/suites/quiet.test.sh"
# Reads the unmapped file too, but the fallback corpus does not run it.
printf '%s\n' '#!/usr/bin/env bash' 'cat data/unmapped.txt' >"$t/suites/zz-outside.test.sh"
Expand Down Expand Up @@ -118,7 +121,7 @@ EOF
fail "trace: exit $rc, want 1: $(cat "$TMP_ROOT/trace.log")"
fi
if [[ "$gaps" == "$want" ]]; then
ok "trace: gaps are the read that selects no suite, the unmapped-to-suite read and the unmapped read of a suite the fallback does not run (mapped, recorded no-suite, fallback-run, unchecked, untracked, self and root toolchain files are not)"
ok "trace: gaps are the read that selects no suite, the unmapped-to-suite read and the unmapped read of a suite the fallback does not run (mapped, recorded no-suite, fallback-run, unchecked, untracked, self, root toolchain files and nested .gitignore/.gitattributes are not)"
else
fail "trace: gaps.tsv [$gaps] want [$want]"
fi
Expand Down
1 change: 1 addition & 0 deletions scripts/validate-plugin-contracts.test.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# test-scope: .claude-plugin/marketplace.json
# Black-box contract test for the check-only carve-out assertions in
# validate-plugin-contracts.mjs.
#
Expand Down
Loading