bug(perf, P0): fail closed on validation/test failures and parse real Node test results

[jinon86]

Summary

scripts/perf-harness.js can emit apparently completed and scored reports even when its required validation and test phases failed. It also parses the current Node test output with an incompatible OK/WARN line heuristic, producing zero tests and zero throughput.

In the 12-node diagnostic exhibition fleet-12-node-preview-20260722T214739+0900, all 12 reports were structurally generated, but every report recorded:

• parsed test count: 0
• parsed validation pass/fail counts: 0/0
• continued scored-value generation despite the required phases failing early

The results were therefore quarantined from ranking and publication.

Current behavior

• phaseValidation() runs node scripts/validate.js all-v2 2>&1 || true.
• phaseTests() runs npm test 2>&1 || true.
• runCommand() already returns exitCode, but both phase functions discard the true child status because the shell expression returns zero.
• Test counting only matches lines beginning with OK or WARN, while the current node --test TAP summary uses fields such as # tests, # pass, and # fail.
• raw_test_failed is hard-coded to 0.
• Scored values and report output are still generated with no explicit invalid/quarantined phase state.

Expected behavior

Required phase failure must be visible, machine-readable, and ranking-blocking. A failed or unparseable test phase must never look like a fast successful benchmark.

Implementation direction

1. Remove shell-level || true from required validation/test phases and use runCommand().exitCode directly.
2. Record a phase result contract containing status, exit code, duration, parsed counts, and a bounded redacted diagnostic.
3. Parse a stable, explicitly selected test reporter (for example Node TAP summary fields), or consume a structured reporter if supported by the repository's Node floor.
4. Treat output/exit-code disagreement as invalid, not successful.
5. Mark the iteration/report invalid or quarantined when a required phase fails or cannot be parsed.
6. Prevent invalid iterations from receiving normal efficiency/execution scores or entering harness-to-packet/scoreboard publication as valid packets.

Acceptance criteria

• A successful test fixture records non-zero raw_test_count, exact pass/fail counts, and exit code 0.
• A failing test fixture records its non-zero exit code and non-zero failure count and makes the iteration non-comparable.
• A validation crash cannot produce raw_failed_count: 0 plus a normal valid score.
• Missing or malformed reporter output is an explicit parse failure, not zero successful tests.
• Report-level validity/comparability summarizes all iteration phase statuses.
• harness-to-packet refuses, quarantines, or marks non-publishable any report with failed required phases.
• Existing valid demo reports either migrate cleanly or fail with a documented compatibility message.
• Failure diagnostics remain bounded and secret-redacted.

Required tests

• Successful TAP/structured reporter fixture.
• Non-zero child exit fixture.
• Exit-zero but malformed/missing summary fixture.
• Validation import crash fixture.
• Multi-iteration mixed success/failure aggregation fixture.
• Packet bridge/publication rejection for an invalid performance report.

Verification

npm test
node scripts/perf-harness.js --iterations 3 --json --validate
node scripts/harness-to-packet.js results/ --agent-id perf-integrity-check
node scripts/score.js aggregate

The final two commands must not publish a failed-phase report as valid.

Safety boundary

Source-only local subprocesses and sanitized fixtures. No credentials, live providers, remote-node mutation, service restart, or leaderboard publication.

[jinon86]

Parent tracker: #299. Recommended implementation order: Slice B, after or alongside #295; this issue owns the fail-closed report/packet validity contract.