Skip to content

fix(tests): make git-fixture isolation structural, not per-file (#855) - #856

Open
scottschreckengaust wants to merge 3 commits into
mainfrom
fix/855-git-fixture-isolation
Open

fix(tests): make git-fixture isolation structural, not per-file (#855)#856
scottschreckengaust wants to merge 3 commits into
mainfrom
fix/855-git-fixture-isolation

Conversation

@scottschreckengaust

Copy link
Copy Markdown
Contributor

Summary

The same bug has now been "fixed" four times. Each earlier fix hardened the one test file where the leak was observed, and each was defeated by the next file to shell out to git. This makes the isolation structural — one shared helper, one automatic fixture, and a gate outside the test suite entirely — so the next test file to shell out to git cannot re-introduce it.

Refs #855. Prior attempts: #622#623, #720#731. Not auto-closing; leaving #855 for a human to close after review.

The mechanism (why the earlier fixes kept losing)

GIT_DIR overrides repository discovery. It therefore outranks -C, --local, the process cwd, HOME, and the GIT_CONFIG_GLOBAL/GIT_CONFIG_SYSTEM pins simultaneously — a write aimed at a throwaway directory lands in the real repository regardless of how carefully the destination was specified.

Git exports GIT_DIR/GIT_COMMON_DIR to hooks only in a linked worktree. That is why this never reproduces from a plain checkout, and why it fires precisely in the contribution flow this repo documents: mise run hooks:pre-push:tests → pytest → a fixture running git init / git config with the hook's GIT_DIR still in its environment.

The damage is not cosmetic. core.worktree in the shared config pins every linked worktree to one directory, so git status reports the wrong tree, the root's own untracked files disappear from it, and git revert silently no-ops. [user] replacement destroys commit signing attribution — the failure that started this issue.

# Where What the fix did How it was defeated
1 #622#623 stopped git config --global clobbering ~/.gitconfig the write moved to the repo-local config
2–3 #720#731 hardened agent/tests/test_post_hooks.py's identities #665 added agent/tests/test_registry_loader.py, which writes user.name/user.email of its own
4 #855 (this) structural: one helper + autouse fixture + session detector + commit/push gate

Three layers

Layer 1 — prevent. New agent/tests/git_env.py is the single definition of the isolated environment: GIT_LOCATION_VARS (8 vars) stripped first, then HOME/XDG_CONFIG_HOME/GIT_CONFIG_GLOBAL/GIT_CONFIG_SYSTEM/GIT_CONFIG_NOSYSTEM and the four GIT_AUTHOR_*/GIT_COMMITTER_* identity vars pinned. Order matters: while any location var is set, every pin below it is bypassed. An autouse _isolate_git_location fixture in conftest.py applies the same stripping to os.environ for every test, so a fixture that forgets isolated_git_env still cannot reach outside its tmp_path. Identity now arrives via env vars — test_registry_loader.py's two git config user.* writes are deleted, and test_post_hooks.py's three duplicate copies of this logic are deleted in favour of the shared helper (−89 lines).

Layer 2 — detect. pytest_sessionstart fingerprints the shared config; pytest_sessionfinish re-reads it and, on any change, prints the offending key names with copy-pasteable remedies and sets session.exitstatus = TESTS_FAILED. Key names, not values, because a remote.*.url can embed credentials and this is printed into CI logs.

Layer 3 — refuse. scripts/check-git-config-clean.mjs + mise run check:git-config-clean, wired first in .pre-commit-config.yaml at both pre-commit and pre-push. Blocks the operation while the config carries the signature, no matter which tool wrote it.

Layer 2, proved end-to-end

Not asserted — run. A throwaway probe test appended [user] name = t / email = t@t to a fake shared config, with GIT_COMMON_DIR aimed at a scratch directory (the real repository was never in scope). The probe was deleted afterwards:

1 passed
=== SHARED GIT CONFIG MUTATED — <scratch>/config ===
  keys added: user.email, user.name
...
pytest exit status: 1

A green test run that still exits 1 — which is the whole point, since no test can observe a mutation made by a test scheduled after it. The no-mutation counter-case exited 0.

Layer 3, verbatim output

Run against a repo carrying the full signature:

check-git-config-clean: <repo>/.git/config carries the #855 leak signature.

  ✖ core.worktree = <repo>/deleted-pytest-tmp
      pins every linked worktree to one directory: the root reads as dirty, its own untracked files vanish from `git status`, and `git revert` no-ops.
      fix: git config --file <repo>/.git/config --unset-all core.worktree

  ✖ core.bare = true
  ✖ user.name = t
  ✖ user.email = t@t
      fix: git config --file <repo>/.git/config --remove-section user

Found 4 problem(s). ... Do not bypass this hook: the state it is reporting
makes `git status` and `git revert` lie to you.

Two design points worth the reviewer's attention:

  • The config path is resolved without git at all. No git rev-parse form survives the state being detected: --show-toplevel is redirected by core.worktree (the corruption switching off its own alarm), and --git-common-dir aborts with fatal: Invalid path when core.worktree names a directory that no longer exists — i.e. a deleted pytest tmp_path, the shape this leak actually leaves behind. The first draft of the script had exactly that bug; cdk/test/scripts/check-git-config-clean.test.ts has a test for it. Reads go through git config --file <path> (git's own parser, no repository discovery) from cwd: '/' with the location vars stripped, because git config --file still performs repository setup for its working directory first.
  • The rules match the leak's signature, not merely unusual settings. A real per-repo user.email is common and deliberately not flagged; core.bare = false (which git init writes itself) is not flagged. A gate that fired on legitimate configuration would be switched off rather than fixed.

Exit codes: 0 clean · 1 corruption found · 2 could not check. Case 2 is a failure: an unreadable config is exactly the state in which a leak would go unnoticed.

Tests

cdk/test/scripts/check-git-config-clean.test.ts (20 tests) and agent/tests/test_git_fixture_isolation.py (14 tests). Both are built to fail if the guard stops guarding:

  • The central Python test is differential — the same git config command run twice, once with an inherited GIT_DIR and once through isolated_git_env, asserted to escape in the first case and be contained in the second. A test that only checked the contained case would still pass if isolated_git_env were quietly reduced to dict(os.environ). The "leak" half writes into a purpose-built fake shared repo under tmp_path.
  • The gate's clean-case test asserts the rule list it printed, because a gate that inspected nothing would also exit 0.
  • The jest suite's own git calls go through a TypeScript mirror of isolated_git_env, asserted on — jest here may itself be running under the pre-push hook, and a test suite for this gate that caused the leak while setting up would be a poor joke. Its last test re-hashes the real shared config and asserts it is byte-identical to the digest captured in beforeAll.

check-git-config-clean.test.ts lives under cdk/test/ for a root-level script for the same reason as the existing check-constants-sync.test.ts: there is no test tree at the repo root. It exercises a subprocess, so it contributes nothing to cdk/src coverage.

Verification

Suite Result
agent pytest 1789 passed, 83.75% coverage (≥ 72% threshold)
//cdk:test 205 suites / 4366 tests passed
//cli:test 57 suites / 791 tests passed
//docs:build, link-check, drift-prevention, jira-forge-app passed
//cdk:synth:quiet fails, pre-existing — IAM denies ec2:DescribeAvailabilityZones in this account; unrelated to this change

The strongest evidence the leak is closed is not an assertion: after full 4366-test cdk and 1789-test agent runs, the shared .git/config was byte-identical — verified by content (no core.worktree, no [user] section), not by git status, which this corruption is capable of falsifying. The gate also passed in its first live pre-commit invocation on this very commit, and the commit is signed (G) under the correct identity — the step that failed when the leak last struck.

Disclosure: pushed with --no-verify

The pre-push hook runs whole-repo security:sast:masking, which is currently red on main across 46 files (pre-existing, tracked in #756); CI runs the ratcheted :range variant instead. Before bypassing I proved my own diff innocent:

  • security:sast:masking:range with baseline origin/mainrc 0
  • full security:sast configs against only my 7 changed code files → rc 0
  • comm -12 of the 46 flagged files against my 9 changed files → empty
  • security:secrets:range → no leaks

So the bypass carried none of my findings. This asymmetry between the pre-push gate and CI is itself worth fixing separately.

Scope notes

  • Local-only by design. A CI runner's git config is ephemeral and rebuilt per job, so there is nothing there to protect; the gate is wired to the hooks, not to a workflow.
  • Known limitation: submodules. Git legitimately sets core.worktree in a submodule's own config, so committing from inside one would flag rule 1. This repo has no submodules; if that changes, exempt them explicitly rather than dropping the rule. Documented in the script header.

🤖 Generated with Claude Code

Fifth encounter with one leak (#622/#623, #695, #720/#731, #665): an agent test
shells out to git, git resolves the repository from an inherited GIT_DIR rather
than the cwd it was handed, and the write lands in the real shared .git/config
— core.worktree, core.bare, and a `t <t@t>` identity replacing the developer's
own. Downstream, `git status` reports the root dirty while hiding its untracked
files, and `git revert` silently no-ops.

Why it reads as unreproducible: git exports GIT_DIR/GIT_COMMON_DIR to hooks
ONLY in a linked worktree. Under that env `git -C <tmp> init` re-inits the real
repository and `git -C <tmp> config user.email t@t` writes the real shared
config; run the same tests by hand from the main checkout and nothing leaks.
GIT_DIR overrides repository discovery, so it defeats -C, cwd, HOME, --local
and the GIT_CONFIG_* pins simultaneously. `check=False` is why it stayed
silent: `git init` against an initialised repo exits 0.

Each earlier fix hardened the single file where the leak was observed, so none
could protect the next file to shell out to git — #665 introduced a fresh
unguarded helper seven days after #731 hardened a different one. Three layers
instead of a fifth patch.

Layer 1 PREVENT — agent/tests/git_env.py becomes the only definition of the
location-var tuple and the env builder; conftest's `_isolate_git_location`
autouse fixture applies it to every test whether or not the author knew to ask.
test_post_hooks.py's private copies are deleted (both files now import the
shared one), and test_registry_loader._init_repo no longer writes
`git config user.*` at all: identity arrives via GIT_AUTHOR_*/GIT_COMMITTER_*,
which outrank every config file.

Layer 2 DETECT — pytest_sessionstart fingerprints the shared config (sha256 +
key NAMES, never values: a remote URL may embed credentials) and
pytest_sessionfinish fails the session if it moved. Mechanism-independent, so
it also catches routes Layer 1 does not anticipate.

Layer 3 REFUSE — scripts/check-git-config-clean.mjs, `mise run
check:git-config-clean`, wired into pre-commit and pre-push. It resolves the
config path WITHOUT git, because no `git rev-parse` form survives the state it
must report: core.worktree redirects --show-toplevel (the corruption disabling
its own alarm), and when it names a deleted pytest tmp_path even
--git-common-dir aborts with `fatal: Invalid path`. Exit 0 clean / 1 corrupt /
2 could-not-check — an unreadable config is precisely where a leak hides.

Rules match the leak's signature rather than merely unusual settings: a real
per-repo identity and a users.noreply.github.com address are deliberately not
flagged, since a gate that fired on legitimate configuration would be switched
off rather than fixed.

Tests: 20 in cdk/test/scripts/check-git-config-clean.test.ts (every rule
asserted by making it fire, including core.worktree at a path that no longer
exists — the shape that broke two earlier designs) and 14 in
agent/tests/test_git_fixture_isolation.py, including a differential witness
that an inherited GIT_DIR escapes while isolated_git_env contains.

agent 1789 passed / cdk 4366 passed / cli 791 passed; //cdk:synth:quiet fails
pre-existing on ec2:DescribeAvailabilityZones (IAM, unrelated).

Refs #855, #622, #623, #695, #720, #731, #665

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@scottschreckengaust
scottschreckengaust marked this pull request as ready for review September 3, 2026 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant