Current score → target
Test effectiveness: 4/5 → 5/5 (test-quality floor).
Defect class prevented
(a) Low-quality tests counting green — tests with no assertions, tests emitting output, tests leaking global state, PHPUnit warnings — all pass silently today; (b) hidden inter-test order dependencies (the Integration suite shares live MongoDB state), a latent flake class that fixed alphabetical order permanently masks. Infection --min-msi=100 catches neither: mutants don't detect order coupling, warnings, or risky-test status.
Evidence
phpunit.xml.dist:4-9 carries only backupGlobals/colors/bootstrap — no failOnRisky, failOnWarning, beStrictAboutOutputDuringTests, beStrictAboutTestsThatDoNotTestAnything, beStrictAboutChangesToGlobalState, no executionOrder. Same for phpunit.memory.xml.dist.- No CLI flags compensate: all Makefile phpunit invocations (lines 68, 89, 93, 101, 512, 515) and a repo-wide grep for
order-by|executionOrder are clean.
Proposed fix
Staged, on the <phpunit> element of both dist files:
<phpunit failOnRisky="true" failOnWarning="true"
beStrictAboutOutputDuringTests="true"
beStrictAboutTestsThatDoNotTestAnything="true"
beStrictAboutChangesToGlobalState="true"
executionOrder="random" ...>Stage 1: strict flags (fix any newly-red risky tests in the same PR). Stage 2: executionOrder="random" (PHPUnit prints the seed on failure for reproduction). Add both dist files to the validate-configuration.sh LOCKED list so the strictness can't be quietly reverted.
Effort
S
Acceptance criteria
Filed by an automated CI/CD verification-coverage audit (OpenSSF Scorecard baseline: aggregate 6.0/10, local mode — SAST 0, Fuzzing 0, Token-Permissions 0, Pinned-Dependencies 5, Vulnerabilities 4). Audit method: inventory → gap analysis → git-history evidence mining → 3-iteration adversarial critique loop.
Generated by Claude Code
Current score → target
Test effectiveness: 4/5 → 5/5 (test-quality floor).
Defect class prevented
(a) Low-quality tests counting green — tests with no assertions, tests emitting output, tests leaking global state, PHPUnit warnings — all pass silently today; (b) hidden inter-test order dependencies (the Integration suite shares live MongoDB state), a latent flake class that fixed alphabetical order permanently masks. Infection
--min-msi=100catches neither: mutants don't detect order coupling, warnings, or risky-test status.Evidence
phpunit.xml.dist:4-9carries onlybackupGlobals/colors/bootstrap— nofailOnRisky,failOnWarning,beStrictAboutOutputDuringTests,beStrictAboutTestsThatDoNotTestAnything,beStrictAboutChangesToGlobalState, noexecutionOrder. Same forphpunit.memory.xml.dist.order-by|executionOrderare clean.Proposed fix
Staged, on the
<phpunit>element of both dist files:Stage 1: strict flags (fix any newly-red risky tests in the same PR). Stage 2:
executionOrder="random"(PHPUnit prints the seed on failure for reproduction). Add both dist files to thevalidate-configuration.shLOCKED list so the strictness can't be quietly reverted.Effort
S
Acceptance criteria
Filed by an automated CI/CD verification-coverage audit (OpenSSF Scorecard baseline: aggregate 6.0/10, local mode — SAST 0, Fuzzing 0, Token-Permissions 0, Pinned-Dependencies 5, Vulnerabilities 4). Audit method: inventory → gap analysis → git-history evidence mining → 3-iteration adversarial critique loop.
Generated by Claude Code