[kyle-sexton]

No related issue: owner decision Q14 (failing workflows). The owner decided to investigate and fix each failing scheduled workflow; this PR covers the daily test-selection audit, which has failed every scheduled run since 2026-10-26.

Summary

maintenance-audit-test-selection.yml fails its scheduled run whenever the day's shard finds a gap: a file a suite read whose one-file diff does not select that suite. The monitoring lane's signal 6 reads that red as findings and keeps its bot-filed incident (issue 14294) open, so the red is the intended signal and the fix is to close the gaps, not to silence the run. One run (shard 3, 42472875440) failed on a different cause: curl: (22) The requested URL returned error: 503 downloading shfmt.

Fix

• Replayed every gap row from the last seven scheduled runs against main's selector: 413 rows in 23 suites still missed. 22 suites now declare those files in a # test-scope: header, as test: declare test scopes the selector missed #8617 did; the selector logic is unchanged. The largest class is test_check_matchers.py reading every eval case's case.yaml, matchers.json and samples/ (332 rows). graphql-operations.test.sh now declares the .github/ config and fixture YAML the audit traced it reading; other .github YAML stays a no-suite class, as scripts/affected-tests-live.sh requires.
• selection-audit.sh skips a .gitignore or .gitattributes read in any directory, as it already skipped the root copies: git reads them when a suite walks the tree. That clears the one unmapped row (identity-domains.test.sh reading plugins/ai-briefing/.gitignore).
• scripts/selection-audit.test.sh's trace fixture reads nested .gitignore/.gitattributes, so anchoring the skip back on the root fails it (PASS=18 FAIL=2 with a root-only pattern).
• The workflow's shfmt and DuckDB downloads retry with --retry 7 --retry-all-errors, matching the other pinned-tool downloads (ci: retry pinned tool downloads 7 times so a short 504 window passes #14378).

Done when

• Every gap row from the last seven scheduled shards selects its suite on this branch, or is a read the audit now skips by rule.
• scripts/affected-tests-live.sh and scripts/selection-audit.test.sh pass.
• A dispatched audit run on main tracing the 22 suites concludes success.

Verification

• Replay of the 413 rows on this branch: 412 now select their suite; the remaining row is the .gitignore read the audit now skips.
• scripts/selection-audit.test.sh: PASS=20 FAIL=0. scripts/affected-tests-live.sh: PASS=39 FAIL=0. scripts/affected-tests.test.sh: PASS=147 FAIL=0.
• scripts/check-affected.sh --tests: 15 passed, 39 skipped. actionlint on the workflow: clean. Changelog fragments: --check-required passes for the 9 plugins touched.
• After merge: a dispatch with the suites: input naming the fixed suites, then each shard's next scheduled run.

Related

• Monitor incident issue 14294 (the monitor closes it once the signal stays clear), test: declare test scopes the selector missed #8617 and fix(ci): select suites for the files they read (#8410) #8411 (same fix shape).
• ci: name every workflow job in Title Case and run the org naming lint #15404 renames job names in this workflow; this PR edits only two curl lines in its steps.

Follow-ups

• Pre-existing (advisory): on 2026-11-11 the trace read 7996 tracked files and checked only 711 against the selector; 7285 were cut off by the 1500 s budget or a selector error. A green shard can still hide gaps; the selector queries need to be cheaper or batched before the audit can see them all.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VQb9Q2ceZLXhTR4d1vkTCe

[chatgpt-codex-connector]

Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-12T00:56:55.067213Z	380a275	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[github-actions]

claude-security-review reviewed 4071b1e757d150d2504e197d4fc399ba2cb2c5a1: read 34 of 34 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #15479

I found no security issues in this change. This lane already reviewed an earlier head (380a2757), so this pass reported CRITICAL and IMPORTANT findings only. It found none.

What I read: every hunk of all 34 changed files. The 9 changelog fragments are identical bump: none notes, so I read the first and skipped its copies. They contain no code, and their paths under .changes/ change nothing about what they do. None of the changed files matches a trusted-path glob.

What changed:

• # test-scope: headers in 22 suites. Each header only adds files that cause its suite to be selected, so more suites run on more diffs. This removes no selection and changes no test logic, hook, gate or trust control. Some headers name plugin.json, marketplace.json, hooks.json, .github/workflows/*.yml or api-budget.bash. They only make the tests run when those files change and do not modify the files themselves.
• .github/workflows/maintenance-audit-test-selection.yml. The shfmt and DuckDB curl downloads add --retry 7 --retry-all-errors. Each download is still pinned to a version and checked with sha256sum -c before use, so a retry cannot substitute an unverified binary. No trigger, permissions: block or token use changed.
• scripts/selection-audit.sh. The trace filter now skips .gitignore and .gitattributes reads in any directory, not just the root copies. This only changes which reads the audit reports as test-selection gaps. It gates no merge, trust or credential path.
• scripts/selection-audit.test.sh. The new fixture adds data/.gitignore and data/.gitattributes, which pins the nested skip. This addresses the settled thread on scripts/selection-audit.sh line 153, and I did not raise that point again.

Instruction surface: the PR removes no standing instruction, rule, hook block list or allowlist. The only comment it rewrites, in graphql-operations.test.sh, describes the wider YAML scope.

Plugin manifests: the PR changes no .claude-plugin/plugin.json or marketplace.json.

Spend: a few more suites may run on diffs that touch the newly declared files, and downloads may retry. Both use Actions minutes the plan already covers. This is not a hold and not a finding.

The PR context contains no embedded instructions aimed at this lane.

critical-findings: 0

[github-actions]

claude-review: 1 inline finding(s) on 380a27577301a956a6812970862f9f6e533541bf; the summary is in this lane's pull request comment.

claude-lane: claude-review

[github-actions]

claude-review reviewed 4071b1e757d150d2504e197d4fc399ba2cb2c5a1: read 34 of 34 changed file(s); 0 finding(s), 0 as inline comments.

Code review: PR #15479, test-scope declarations for the selection audit (blocking only)

This lane already reviewed an earlier head (380a2757), so this pass reports only blocking findings. I found none.

I read every hunk of all 34 changed files. The nine .changes/* fragments are identical, so I read the first one and counted the copies as read. All nine are added at the same .changes/<plugin>/ location, so none of them behaves differently because of its path.

Since the last review

• Nested .gitignore/.gitattributes test (scripts/selection-audit.test.sh). This was the one finding from last round. It was resolved with an owner reply, and the code under that thread has not changed since. The trace fixture now writes data/.gitignore and data/.gitattributes after mkdir -p "$t/data" (L73). git add -A at L100 tracks both files, and reader.test.sh reads both. The stub selector maps neither file to a suite, so if the new grep -vE '(^|/)\.git(ignore|attributes)$' filter were removed, both would show up in gaps.tsv as selects no suite. The existing want therefore pins the new skip. I did not re-raise the settled point.
• The rest of the change has the same shape I reviewed last round. The # test-scope: headers sit in the leading comment block, after the shebang or before the docstring. The selector logic is unchanged. The shfmt and DuckDB downloads now retry with --retry 7 --retry-all-errors.

Not verified by me: the 413-row replay, the PASS counts, the actionlint result and the changelog check are what the author reports. I had no shell, so I ran nothing.

Spend (not a finding): more pull requests will now select graphql-operations.test.sh (for the newly declared .github/ config and fixture YAML) and test_babysit_postmerge.py (for any workflow change). That means somewhat longer test runs on the existing CI runners, with no billing change.

No plugin manifests changed. No path-scoped rule applies to these header-only edits, and the PR context contains no embedded instructions.