Skip to content

fix: stop tests leaking sensitive values into one another - #507

Merged
josegonzalez merged 1 commit into
mainfrom
preflight-harden-global-test-state
Sep 1, 2026
Merged

fix: stop tests leaking sensitive values into one another#507
josegonzalez merged 1 commit into
mainfrom
preflight-harden-global-test-state

Conversation

@josegonzalez

@josegonzalez josegonzalez commented Aug 31, 2026

Copy link
Copy Markdown
Member

Four property tests register a secret and never clear it, so it stays registered for every test that runs afterwards and any of them reading that literal back would see *** instead. Nothing fails today, because no test happens to assert on token123 or deploy-bot - but the exposure is one unlucky fixture value away, and it becomes a real hazard the moment tests run in parallel and the set of tests that ran first stops being fixed.

Production code is the source and is behaving correctly: planProperty registers a sensitive property's desired and probed values so a drift reason cannot echo a credential, and nothing in tasks clears the registry afterwards because a CLI process exits. The four tests that reach that path now take an empty registry and put the previous set back when they finish, through a shared isolateMaskRegistry helper, and the seven tests that were already managing the registry by hand use the same helper instead of clearing to nil.

Restoring rather than clearing is what makes the leak observable at all. Clearing is why it went unnoticed: the tests that set the registry directly wiped it on the way out, so residue never survived far enough to be seen - an end-of-run check came back clean until those cleanups stopped wiping. With nothing wiping, TestMain can assert the registry is empty when the package finishes and fail the run when it is not, which is what pins this rather than leaving it to be rediscovered.

Integration tests register through the same production path, and there the registration is the behaviour under test rather than a leak - the documented dokku_letsencrypt_property example sets a dns-provider-* credential. They are isolated at skipIfNoDokkuT, the one gate all 139 of them already pass through, so none has to know whether the task it applies happens to be sensitive.

commands needs none of it: every command clears the registry on the way out of Run, so residue cannot accumulate there.

Also saves and restores color.NoColor in the fmt colour test rather than assuming the previous value was true, matching TestFormatterColorOnEmitsAnsi.

Prerequisite for #502.

Four property tests register a secret and never clear it, so it stays registered for every test that runs afterwards and any of them reading that literal back would see `***` instead. Nothing fails today, because no test happens to assert on `token123` or `deploy-bot` - but the exposure is one unlucky fixture value away, and it becomes a real hazard the moment tests run in parallel and the set of tests that ran first stops being fixed.

Production code is the source and is behaving correctly: `planProperty` registers a sensitive property's desired and probed values so a drift reason cannot echo a credential, and nothing in `tasks` clears the registry afterwards because a CLI process exits. The four tests that reach that path now take an empty registry and put the previous set back when they finish, through a shared `isolateMaskRegistry` helper, and the seven tests that were already managing the registry by hand use the same helper instead of clearing to nil.

Restoring rather than clearing is what makes the leak observable at all. Clearing is why it went unnoticed: the tests that set the registry directly wiped it on the way out, so residue never survived far enough to be seen - an end-of-run check came back clean until those cleanups stopped wiping. With nothing wiping, `TestMain` can assert the registry is empty when the package finishes and fail the run when it is not, which is what pins this rather than leaving it to be rediscovered.

Integration tests register through the same production path, and there the registration is the behaviour under test rather than a leak - the documented `dokku_letsencrypt_property` example sets a `dns-provider-*` credential. They are isolated at `skipIfNoDokkuT`, the one gate all 139 of them already pass through, so none has to know whether the task it applies happens to be sensitive.

`commands` needs none of it: every command clears the registry on the way out of `Run`, so residue cannot accumulate there.

Also saves and restores `color.NoColor` in the fmt colour test rather than assuming the previous value was `true`, matching `TestFormatterColorOnEmitsAnsi`.
@josegonzalez
josegonzalez force-pushed the preflight-harden-global-test-state branch from 61e2835 to 05e20a9 Compare August 31, 2026 07:46
@josegonzalez
josegonzalez merged commit 25d8e0e into main Sep 1, 2026
19 checks passed
@josegonzalez
josegonzalez deleted the preflight-harden-global-test-state branch September 1, 2026 23:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant