P3.31: durable pending human-triage surface + CLI approve path (#63)

[franciszver]

Summary

Closes #63

A vuln report routed to pending_human_approval previously had no durable surface: ActionLog.export_jsonl was only ever called from emit_snapshot at the top of a campaign iteration, so a --iterations 1 run's own events (including exploit_recorded/vuln_report_pending_human_approval) never reached the exported log; tools/run_campaign.py always used ExploitDB(":memory:")/DocumentationAgent(reports_dir=None); and there was no approve path reachable from the CLI at all. Approving VULN-0004 required a bespoke reconstruction script (tools/approve_vuln_0004.py) because DocumentationAgent._pending was in-memory only and the filing process had exited (issue #66's sharpening comment on #63).

What changed

• Post-loop ActionLog export (redteam/campaign.py): run_campaign now calls action_log.export_jsonl(action_log_ref) unconditionally after the loop (in a try/finally, hardened further below), so a run's own events are never lost regardless of iteration count, an injected snapshot_fn, or an exception escaping the loop.
• DocumentationAgent persists pending reports too (redteam/agents/documentation.py): a pending report is now written to <reports_dir>/<report_id>.pending-human-approval.json (same suffix convention tools/build_vuln_reports.py already used for VULN-0004), and __init__ loads every persisted filed/pending report from reports_dir back into memory at construction. This is what makes approve() reachable across a process boundary — no bespoke reconstruction script needed. A filed report wins over a stale pending leftover of the same exploit_id (self-healing after a crash between persisting-filed and unlinking-pending). A reports_dir this module can't parse raises DocumentationAgentError loudly rather than silently losing a pending report.
• Durable pending-triage count: redteam/observability/findings.py gains pending_human_triage_count(); emit_snapshot wires it into a new optional, additive v1 observability_snapshot.schema.json field pending_human_triage_count (see contracts/README.md's versioning rule — new optional property, not added to required, stays v1).
• CLI approve/list-pending path (tools/run_campaign.py): --list-pending --reports-dir PATH and --approve EXPLOIT_ID --reports-dir PATH --approved-by NAME (--db-path PATH | --unverified-i-vouch-without-db-check). Neither touches a live model/target. When --db-path names a persisted exploit DB, --approve re-derives the report from the original exploit record via build_vuln_report and refuses to approve on any field-value drift from the persisted pending artifact — the same verify-then-approve discipline tools/approve_vuln_0004.py established, generalized to any exploit_id.
• Default run behaviour is unchanged: ExploitDB(":memory:")/DocumentationAgent(reports_dir=None) stay the default (a quick demo/smoke run per docs/DEMO_SCRIPT.md should still leave nothing behind). --reports-dir/--db-path are opt-in.
• Deep-review self-fix: the auto-load behavior above regressed tools/approve_vuln_0004.py's own designed re-run flow (a genuinely-pending report would crash the script via the duplicate-rejection guard, plus a double-unlink on the pending file). Fixed and covered by a new regression test.
• Doc test-count claims updated in docs/ATO_EVIDENCE_PACKET.md/docs/DEMO_SCRIPT.md per tests/test_doc_test_counts.py.

Cold-review remediation (this revision)

A cold review of the original state above returned DO-NOT-MERGE with 4 blockers and several majors/minors, all fixed in follow-up commits on this same branch, each red-first and gated (declutter → sec-audit → deep-review):

1. tools/approve_vuln_0004.py self-comparison (BLOCKER): the already_loaded branch compared the auto-loaded pending report against pending_on_disk — the same file — which always passed regardless of tampering. Proven: a hand-tampered VULN-0004 pending artifact (severity: "low", doctored clinical_impact) filed cleanly (rc=0) before the fix. Fixed: the comparison now uses an independent re-derivation via build_vuln_report(record, ...); already_loaded is used only to satisfy approve()'s in-memory precondition.
2. --approve failed open (BLOCKER): no --db-path meant no provenance check at all (a hand-written pending report approved cleanly); a --db-path naming a non-existent file silently created an empty sqlite DB and downgraded to a warning; --approved-by defaulted to "owner". Fixed: the cross-check is now required by default (explicit --unverified-i-vouch-without-db-check escape hatch, loud WARNING), a missing --db-path file is a hard refusal, --approved-by has no default, and the report body is printed before stamping.
3. --reports-dir without --db-path crashed the run (BLOCKER): the documented "merely collides" flag combo actually raised an uncaught DocumentationAgentError from file_report mid-loop, and — because the post-loop export had no try/finally — silently discarded that run's own action log. Fixed: file_report is now wrapped like its sibling component calls, the whole loop is in try/finally, and run mode refuses to start with this flag combo instead of emitting a NOTE.
4. Filename/report_id mismatch could overwrite approved evidence (MAJOR): _load_persisted keyed reports by exploit_id from file content, never checking the filename or report_id uniqueness. Proven: a weird-name.pending-human-approval.json claiming report_id: VULN-0001, exploit_id: EXP-0002 caused --approve EXP-0002 to overwrite the already-filed, already-approved VULN-0001.json. Fixed: a persisted file whose name doesn't match its own claimed report_id is now rejected at load, report_id collisions across exploit_ids are rejected, and each pending report's real source path is tracked and unlinked on approval (not reconstructed from report_id).
5. Everything else (MINOR/doc): --list-pending/--approve no longer leak a raw traceback for a malformed reports_dir; contracts/README.md and the schema's field description no longer overclaim "consumers stay valid" (true for producers, not for a consumer pinned to a pre-P3.31: pending human-triage reports have no durable surface #63 schema copy); pending_human_triage_count's two different scopes (snapshot-field vs. --list-pending's directory-wide count) are now documented in the schema, the Python docstring, and docs/ARCHITECTURE.md; the --db-path rebuild now derives force_human_gate from the trusted exploit record's category (not the untrusted field under verification) and carries fix_validation_status through; two stray Mermaid colour mutations in docs/ATO_EVIDENCE_PACKET.md reverted; the new CLI approval path is now documented as Beat 5 of docs/DEMO_SCRIPT.md; doc test-count claims refreshed to match the live suite.

Verification

• Red-first, corrected claim: the original 3-commit red-first sequence claimed all 3 were red-first; in fact only 2 were (documentation-agent persistence, campaign post-loop export) — the pending_human_triage_count field shipped bundled with its own already-passing tests in the same commit (see that commit's own message). Every fix in the cold-review remediation above is genuinely red-first, with failing output quoted in each commit message.
• Gates run inline for every fix: declutter (folded a duplicated CLI reports_dir-load try/except into one helper; removed a dead variable assignment pyflakes flagged after the FIX 1 change), sec-audit (re-ran all four of the cold reviewer's proven attacks against the fixed code — each now refused, fail-closed, with a clean CLI message rather than a traceback), deep-review (semantic diff of redteam/campaign.py isolated to exactly the intended try/finally wrap + one new try/except, no accidental logic change from the reindentation).
• docs/vuln_reports/*.json (VULN-0001..0004) sha256-verified byte-identical before and after every commit in this remediation — never touched, never run against with any builder/approve tool.
• Full suite: 375 passed locally (sibling checkout present) — grew from 359 with this remediation's new tests.

Test plan

• python -m pytest tests/ -q → 375 passed
• docs/vuln_reports/*.json sha256 unchanged (verified before this remediation and after every commit)
• Manual end-to-end: file a pending report with one DocumentationAgent instance, del it, approve with a fresh instance via tools/run_campaign.py --approve ... --reports-dir ... --db-path ... --approved-by ... — works with no bespoke script
• All four cold-reviewer attacks reproduced against pre-fix code (confirmed exploitable) and re-run against post-fix code (confirmed refused, fail-closed)
• CI green (polling below)