Skip to content

test(scripts): cover the dry-run/apply mutation boundary of apply-repo-settings - #128

Merged
bketelsen merged 2 commits into
mainfrom
ci-gap/apply-repo-settings-hermetic-test
Aug 28, 2026
Merged

test(scripts): cover the dry-run/apply mutation boundary of apply-repo-settings#128
bketelsen merged 2 commits into
mainfrom
ci-gap/apply-repo-settings-hermetic-test

Conversation

@bketelsen

Copy link
Copy Markdown
Contributor

Summary

scripts/apply-repo-settings.sh is dry-run by default and mutates GitHub only
with --apply (ADR-0040). Nothing pinned that boundary, so a refactor could
have made the default invocation write to a repository with no test failing.

Adds test/apply-repo-settings.test.mjs, a hermetic Node test that runs the
real script against a fake gh placed on PATH. The stub answers the
contract's read endpoints from temporary JSON fixtures and records every
mutating call instead of making one; an endpoint the script was not expected
to reach fails loudly rather than returning an empty body that would look like
conformance. No network, no credential, no real repository.

Three cases, all on one representative repository-object drift path
(.repository.delete_branch_on_merge), scoped to the command contract with
--no-rulesets:

  • default invocation reports WOULD PATCH … and 1 change(s) planned, and
    the call log contains no -X PATCH|PUT|POST|DELETE and the mutation log is
    empty;
  • --apply records exactly one mutation — PATCH repos/<repo> with body
    {"delete_branch_on_merge": true} — and reports 1 change(s) applied;
  • an already-conformant fixture reports already matches the repository settings contract and mutates nothing, in both modes.

Fixtures are derived from
organization/contracts/repository-settings/v1.json itself, so a contract
change cannot silently turn the conformant case into a drift case. No
production code changed.

Risk tier

Risk tier: 1 — test-only change; no production script, workflow, policy, or
contract file is touched.

Docs housekeeping

No docs touched — this change adds a single test file.

Verification

  • node scripts/check-docs.mjs green (run as part of npm run verify):
ok   docs_index_coverage: 1.000 (required 1)
ok   link_integrity: 1.000 (required 1)
ok   symlink_resolution: 1.000 (required 1)
  • npm run verify green (exit 0) — check:docs, check:organization,
    check:scripts (ok bash_syntax: 6/6), and npm test:
1..42
# tests 42
# pass 42
# fail 0

The three new cases are picked up by the existing node --test test/*.test.mjs
signal:

ok 1 - the default invocation plans a drifted setting and issues no mutating call
ok 2 - --apply issues exactly the expected mutation
ok 3 - an already-conformant repository is idempotent in both modes
  • The new tests were proved to catch regressions, then the script was
    restored with no diff:
    • plan() forced to run gh api regardless of --apply → case 1 fails
      (# pass 2 / # fail 1).
    • the repository-object drift comparison forced to always report drift →
      cases 2 and 3 fail (# pass 1 / # fail 2).
  • Scaffold unchanged — .agents/skills/frostyard-docs-site/scaffold/ not
    touched.
  • Workflows unchanged — no file under .github/workflows/ is touched.

bketelsen and others added 2 commits August 27, 2026 21:28
…o-settings

Adds test/apply-repo-settings.test.mjs, a hermetic Node test that runs
scripts/apply-repo-settings.sh against a fake `gh` on PATH. The stub answers
the repository-settings contract's read endpoints from temporary JSON
fixtures and records every mutating call instead of making one, so the test
needs no network, no credential, and no real repository.

The fixtures are derived from
organization/contracts/repository-settings/v1.json itself, so a contract
change cannot silently turn the "already conformant" case into a drift case.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UigLsmd17TVDuaaSN4yfdU
Real `gh api` sends POST whenever a request body is supplied (--input,
-f/--raw-field, -F/--field), so the harness's old rule — mutation only on an
explicit non-GET -X — let a dry-run regression mutate GitHub while the
boundary test still passed.

The fake gh now derives the effective method the way gh does: an explicit
-X/--method wins, otherwise a request body makes it a POST, otherwise GET.
Every effective non-GET call is recorded to the mutations log instead of being
answered from a fixture, so the existing dry-run assertions (empty mutations
log) actually bind. A fourth case pins the derivation itself for both the
--input and the -f form.

No production code changed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UigLsmd17TVDuaaSN4yfdU
@bketelsen
bketelsen marked this pull request as ready for review August 28, 2026 02:07
@bketelsen
bketelsen added this pull request to the merge queue Aug 28, 2026
Merged via the queue into main with commit 12beba6 Aug 28, 2026
3 checks passed
@bketelsen
bketelsen deleted the ci-gap/apply-repo-settings-hermetic-test branch August 28, 2026 02:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant