Current score → target
Test effectiveness / performance-gate integrity: 4/5 → 5/5.
Defect class prevented
The blocking performance gate going vacuously green: k6 raises no error when thresholds are absent, so a regression in threshold construction silently disables the entire perf gate.
Evidence
- The gate's semantics live in ~1,291 lines of untested JS:
tests/Load/utils/ (thresholdsBuilder.js, scenariosBuilder.js 91 ln, utils.js 248 ln, LoadTestConfigManager.js, shared-data loaders).
ThresholdsBuilder already contains a silent-disable branch: K6_SKIP_DURATION_THRESHOLDS swaps p(99)<X for the always-true max>=0. A bug in that env parsing, in tag naming (test_type: vs scenario:), or a build() returning {} = every k6 run passes with zero enforcement.tests/Load/package.json's test script is a no-op stub asserting "Tests run at workspace root" — while the root package.json is literally {}. Documented intent, no implementation.- Only lint ever touches these files, via Super-Linter with
continue-on-error: true.
Proposed fix
node --test (zero new deps) specs for: ThresholdsBuilder (threshold shape per scenario; skip-flag behavior both ways; non-empty output asserted), ScenariosBuilder, and config-manager parsing. Wire as a job in load-tests.yml gating the shards, path-filtered on tests/Load/** with a skip-fallback:
k6-config-tests:
timeout-minutes: 5
steps:
- uses: actions/checkout@<sha>
- run: cd tests/Load && node --test utils/__tests__/
load-tests-sharded:
needs: [k6-config-tests]Effort
M
Acceptance criteria
Filed by an automated CI/CD verification-coverage audit (OpenSSF Scorecard baseline: aggregate 6.0/10, local mode — SAST 0, Fuzzing 0, Token-Permissions 0, Pinned-Dependencies 5, Vulnerabilities 4). Audit method: inventory → gap analysis → git-history evidence mining → 3-iteration adversarial critique loop.
Generated by Claude Code
Current score → target
Test effectiveness / performance-gate integrity: 4/5 → 5/5.
Defect class prevented
The blocking performance gate going vacuously green: k6 raises no error when thresholds are absent, so a regression in threshold construction silently disables the entire perf gate.
Evidence
tests/Load/utils/(thresholdsBuilder.js,scenariosBuilder.js91 ln,utils.js248 ln,LoadTestConfigManager.js, shared-data loaders).ThresholdsBuilderalready contains a silent-disable branch:K6_SKIP_DURATION_THRESHOLDSswapsp(99)<Xfor the always-truemax>=0. A bug in that env parsing, in tag naming (test_type:vsscenario:), or abuild()returning{}= every k6 run passes with zero enforcement.tests/Load/package.json's test script is a no-op stub asserting "Tests run at workspace root" — while the rootpackage.jsonis literally{}. Documented intent, no implementation.continue-on-error: true.Proposed fix
node --test(zero new deps) specs for: ThresholdsBuilder (threshold shape per scenario; skip-flag behavior both ways; non-empty output asserted), ScenariosBuilder, and config-manager parsing. Wire as a job inload-tests.ymlgating the shards, path-filtered ontests/Load/**with a skip-fallback:Effort
M
Acceptance criteria
ThresholdsBuilder.build()return{}fails the new testsFiled by an automated CI/CD verification-coverage audit (OpenSSF Scorecard baseline: aggregate 6.0/10, local mode — SAST 0, Fuzzing 0, Token-Permissions 0, Pinned-Dependencies 5, Vulnerabilities 4). Audit method: inventory → gap analysis → git-history evidence mining → 3-iteration adversarial critique loop.
Generated by Claude Code