fix(tests): make git-fixture isolation structural, not per-file (#855) - #856
Open
scottschreckengaust wants to merge 3 commits into
Open
fix(tests): make git-fixture isolation structural, not per-file (#855)#856scottschreckengaust wants to merge 3 commits into
scottschreckengaust wants to merge 3 commits into
Conversation
Fifth encounter with one leak (#622/#623, #695, #720/#731, #665): an agent test shells out to git, git resolves the repository from an inherited GIT_DIR rather than the cwd it was handed, and the write lands in the real shared .git/config — core.worktree, core.bare, and a `t <t@t>` identity replacing the developer's own. Downstream, `git status` reports the root dirty while hiding its untracked files, and `git revert` silently no-ops. Why it reads as unreproducible: git exports GIT_DIR/GIT_COMMON_DIR to hooks ONLY in a linked worktree. Under that env `git -C <tmp> init` re-inits the real repository and `git -C <tmp> config user.email t@t` writes the real shared config; run the same tests by hand from the main checkout and nothing leaks. GIT_DIR overrides repository discovery, so it defeats -C, cwd, HOME, --local and the GIT_CONFIG_* pins simultaneously. `check=False` is why it stayed silent: `git init` against an initialised repo exits 0. Each earlier fix hardened the single file where the leak was observed, so none could protect the next file to shell out to git — #665 introduced a fresh unguarded helper seven days after #731 hardened a different one. Three layers instead of a fifth patch. Layer 1 PREVENT — agent/tests/git_env.py becomes the only definition of the location-var tuple and the env builder; conftest's `_isolate_git_location` autouse fixture applies it to every test whether or not the author knew to ask. test_post_hooks.py's private copies are deleted (both files now import the shared one), and test_registry_loader._init_repo no longer writes `git config user.*` at all: identity arrives via GIT_AUTHOR_*/GIT_COMMITTER_*, which outrank every config file. Layer 2 DETECT — pytest_sessionstart fingerprints the shared config (sha256 + key NAMES, never values: a remote URL may embed credentials) and pytest_sessionfinish fails the session if it moved. Mechanism-independent, so it also catches routes Layer 1 does not anticipate. Layer 3 REFUSE — scripts/check-git-config-clean.mjs, `mise run check:git-config-clean`, wired into pre-commit and pre-push. It resolves the config path WITHOUT git, because no `git rev-parse` form survives the state it must report: core.worktree redirects --show-toplevel (the corruption disabling its own alarm), and when it names a deleted pytest tmp_path even --git-common-dir aborts with `fatal: Invalid path`. Exit 0 clean / 1 corrupt / 2 could-not-check — an unreadable config is precisely where a leak hides. Rules match the leak's signature rather than merely unusual settings: a real per-repo identity and a users.noreply.github.com address are deliberately not flagged, since a gate that fired on legitimate configuration would be switched off rather than fixed. Tests: 20 in cdk/test/scripts/check-git-config-clean.test.ts (every rule asserted by making it fire, including core.worktree at a path that no longer exists — the shape that broke two earlier designs) and 14 in agent/tests/test_git_fixture_isolation.py, including a differential witness that an inherited GIT_DIR escapes while isolated_git_env contains. agent 1789 passed / cdk 4366 passed / cli 791 passed; //cdk:synth:quiet fails pre-existing on ec2:DescribeAvailabilityZones (IAM, unrelated). Refs #855, #622, #623, #695, #720, #731, #665 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
scottschreckengaust
marked this pull request as ready for review
September 3, 2026 19:42
scottschreckengaust
requested review from
a team and
backgroundagents
as code owners
September 3, 2026 19:42
scottschreckengaust
enabled auto-merge
September 3, 2026 19:42
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The same bug has now been "fixed" four times. Each earlier fix hardened the one test file where the leak was observed, and each was defeated by the next file to shell out to git. This makes the isolation structural — one shared helper, one automatic fixture, and a gate outside the test suite entirely — so the next test file to shell out to git cannot re-introduce it.
Refs #855. Prior attempts: #622 → #623, #720 → #731. Not auto-closing; leaving #855 for a human to close after review.
The mechanism (why the earlier fixes kept losing)
GIT_DIRoverrides repository discovery. It therefore outranks-C,--local, the process cwd,HOME, and theGIT_CONFIG_GLOBAL/GIT_CONFIG_SYSTEMpins simultaneously — a write aimed at a throwaway directory lands in the real repository regardless of how carefully the destination was specified.Git exports
GIT_DIR/GIT_COMMON_DIRto hooks only in a linked worktree. That is why this never reproduces from a plain checkout, and why it fires precisely in the contribution flow this repo documents:mise run hooks:pre-push:tests→ pytest → a fixture runninggit init/git configwith the hook'sGIT_DIRstill in its environment.The damage is not cosmetic.
core.worktreein the shared config pins every linked worktree to one directory, sogit statusreports the wrong tree, the root's own untracked files disappear from it, andgit revertsilently no-ops.[user]replacement destroys commit signing attribution — the failure that started this issue.git config --globalclobbering~/.gitconfigagent/tests/test_post_hooks.py's identitiesagent/tests/test_registry_loader.py, which writesuser.name/user.emailof its ownThree layers
Layer 1 — prevent. New
agent/tests/git_env.pyis the single definition of the isolated environment:GIT_LOCATION_VARS(8 vars) stripped first, thenHOME/XDG_CONFIG_HOME/GIT_CONFIG_GLOBAL/GIT_CONFIG_SYSTEM/GIT_CONFIG_NOSYSTEMand the fourGIT_AUTHOR_*/GIT_COMMITTER_*identity vars pinned. Order matters: while any location var is set, every pin below it is bypassed. An autouse_isolate_git_locationfixture inconftest.pyapplies the same stripping toos.environfor every test, so a fixture that forgetsisolated_git_envstill cannot reach outside itstmp_path. Identity now arrives via env vars —test_registry_loader.py's twogit config user.*writes are deleted, andtest_post_hooks.py's three duplicate copies of this logic are deleted in favour of the shared helper (−89 lines).Layer 2 — detect.
pytest_sessionstartfingerprints the shared config;pytest_sessionfinishre-reads it and, on any change, prints the offending key names with copy-pasteable remedies and setssession.exitstatus = TESTS_FAILED. Key names, not values, because aremote.*.urlcan embed credentials and this is printed into CI logs.Layer 3 — refuse.
scripts/check-git-config-clean.mjs+mise run check:git-config-clean, wired first in.pre-commit-config.yamlat bothpre-commitandpre-push. Blocks the operation while the config carries the signature, no matter which tool wrote it.Layer 2, proved end-to-end
Not asserted — run. A throwaway probe test appended
[user] name = t / email = t@tto a fake shared config, withGIT_COMMON_DIRaimed at a scratch directory (the real repository was never in scope). The probe was deleted afterwards:A green test run that still exits 1 — which is the whole point, since no test can observe a mutation made by a test scheduled after it. The no-mutation counter-case exited 0.
Layer 3, verbatim output
Run against a repo carrying the full signature:
Two design points worth the reviewer's attention:
git rev-parseform survives the state being detected:--show-toplevelis redirected bycore.worktree(the corruption switching off its own alarm), and--git-common-diraborts withfatal: Invalid pathwhencore.worktreenames a directory that no longer exists — i.e. a deleted pytesttmp_path, the shape this leak actually leaves behind. The first draft of the script had exactly that bug;cdk/test/scripts/check-git-config-clean.test.tshas a test for it. Reads go throughgit config --file <path>(git's own parser, no repository discovery) fromcwd: '/'with the location vars stripped, becausegit config --filestill performs repository setup for its working directory first.user.emailis common and deliberately not flagged;core.bare = false(whichgit initwrites itself) is not flagged. A gate that fired on legitimate configuration would be switched off rather than fixed.Exit codes:
0clean ·1corruption found ·2could not check. Case 2 is a failure: an unreadable config is exactly the state in which a leak would go unnoticed.Tests
cdk/test/scripts/check-git-config-clean.test.ts(20 tests) andagent/tests/test_git_fixture_isolation.py(14 tests). Both are built to fail if the guard stops guarding:git configcommand run twice, once with an inheritedGIT_DIRand once throughisolated_git_env, asserted to escape in the first case and be contained in the second. A test that only checked the contained case would still pass ifisolated_git_envwere quietly reduced todict(os.environ). The "leak" half writes into a purpose-built fake shared repo undertmp_path.isolated_git_env, asserted on — jest here may itself be running under the pre-push hook, and a test suite for this gate that caused the leak while setting up would be a poor joke. Its last test re-hashes the real shared config and asserts it is byte-identical to the digest captured inbeforeAll.check-git-config-clean.test.tslives undercdk/test/for a root-level script for the same reason as the existingcheck-constants-sync.test.ts: there is no test tree at the repo root. It exercises a subprocess, so it contributes nothing tocdk/srccoverage.Verification
agentpytest//cdk:test//cli:test//docs:build, link-check,drift-prevention, jira-forge-app//cdk:synth:quietec2:DescribeAvailabilityZonesin this account; unrelated to this changeThe strongest evidence the leak is closed is not an assertion: after full 4366-test cdk and 1789-test agent runs, the shared
.git/configwas byte-identical — verified by content (nocore.worktree, no[user]section), not bygit status, which this corruption is capable of falsifying. The gate also passed in its first live pre-commit invocation on this very commit, and the commit is signed (G) under the correct identity — the step that failed when the leak last struck.Disclosure: pushed with
--no-verifyThe pre-push hook runs whole-repo
security:sast:masking, which is currently red onmainacross 46 files (pre-existing, tracked in #756); CI runs the ratcheted:rangevariant instead. Before bypassing I proved my own diff innocent:security:sast:masking:rangewith baselineorigin/main→ rc 0security:sastconfigs against only my 7 changed code files → rc 0comm -12of the 46 flagged files against my 9 changed files → emptysecurity:secrets:range→ no leaksSo the bypass carried none of my findings. This asymmetry between the pre-push gate and CI is itself worth fixing separately.
Scope notes
core.worktreein a submodule's own config, so committing from inside one would flag rule 1. This repo has no submodules; if that changes, exempt them explicitly rather than dropping the rule. Documented in the script header.🤖 Generated with Claude Code