v1.0.0 — Shadow Score Spec Level 4 (Adversarial Independence)

[DUBSOpenHub]

Summary

v0.1.0 assigned the same model — claude-sonnet-4.6 — to both qa_sealed and lead_eng. That is the same model family on both sides of a test designed to measure independence.

Hiding the tests answers "can the builder see the tests?" It does not answer "does the builder think like the test author?" Two instances of one family share training data, reasoning priors, and — critically — failure modes. If the family doesn't think to test Unicode homographs, it also doesn't think to normalise them. The sealed test is never written, the code is never hardened, the score reads 0%, and the bug ships with a green light.

The bias is not random. It always points toward 0% — the system overclaims quality exactly when it is least entitled to.

This PR makes cross-family separation a machine-checked invariant, then builds out the capabilities that separation makes possible. Target: Shadow Score Spec v2.0 — Level 4 (Adversarial Independence).

The three bugs this also fixes

Bug	Impact
agents/qa-validator.md wrote GAP-REPORT.md; SKILL.md read SHADOW-REPORT.md	Runtime-breaking. Phase 5 extraction found nothing, the Phase 6 archive cp errored, Phase 7 had no input. Fallout from a half-finished gap-score-spec → shadow-score-spec rename.
README claimed Level 3	Unearned. Spec §6 requires tracked hardening velocity — which was uncomputable, since only the final score was ever stored.
Reports omitted shadow_score_spec_version and level	Spec §5.2 required fields.

What's new

Area	Change
Independence	invariants.cross_family_required, enforced at Phase 0 (abort) and in CI. config.families + config.model_capabilities — unsupported reasoning_effort / context_tier are omitted, never defaulted.
Seal plurality	N sealed suites from N families. One suite grades the code; two independent suites grade the specification.
Phase 2.5a — ambiguity gate	AMBIGUITY.md. Where two families read the same PRD and assert opposite behaviour, the spec is ambiguous. Distinguishes contradiction (the signal) from divergence (the coverage dividend).
Phase 2.5b — Arch Critic	rubber-duck agent, cross-family from the architect, reviews ARCH.md before any code exists.
Phase 4.5 — Red Team	security-review agent attacks what the spec forgot. Findings classed spec gap vs implementation bug — the class decides whether the PM or the engineer learns from it.
Tournament / Pareto	N cross-family competitors, one envelope written before any of them starts. The failure-overlap table is the real product: when every competitor fails the same test, that's a spec defect no model swap can fix.
Autonomy gates	dark / supervised / manual. Gates print the measured value, the threshold, and the evidence. A breach that can't explain itself isn't actionable.
Learning loops	Blind-spot memory, empirical routing, and Shadow-vs-Outcome calibration via session_store_sql.
Reporting	SHADOW-REPORT.json with §5.2 fields + provenance (independence, seal_families, implementer_family). hardening_velocity now computable.

Notable design decisions

Bias direction matters more than "everything must differ." A pairing that biases the score upward is conservative and tolerable; one that biases it downward hides defects. PM == seal author is allowed with disclosure; PM == implementer is forbidden. Blanket difference rules miss this.

Deliberate divergence from Spec v1.0.0 §4.3. The spec says to copy sealed tests into the implementation workspace at validation time. That's safe only when validation is terminal — Dark Factory hardens in a loop, so after cycle 1 the tests sit in a directory the builder can grep, and the Lead Engineer has unscoped bash/glob/grep. v1.0 runs validation in a disposable worktree built from the engineer's commit, and moved the vault outside the repo. Isolation has to be filesystem topology, not prompt wording. invariants.workspace_isolation: legacy restores the old behaviour.

Summarisation is now harmful. safety.summarize_artifacts defaults to false and the line caps became emergency backstops (2000/5000). With long-context tiers, routine summarisation lossy-compresses the contract that both the sealed tests and the implementation derive from — requirements drop out of the middle and the Shadow Score ends up measuring the summariser.

premium is per-role now. A single premium_model applied to every role collapses the pipeline into one family and silently voids the score. Premium raises capability within each role's family.

Verification

$ python3 .github/scripts/validate_conformance.py
✅ Shadow Score Spec v2.0 Level 4 conformance checks passed
   implementer family : anthropic
   seal families      : ['google', 'openai']
   independence       : strong

$ npx markdownlint-cli2        # CI config
Summary: 0 issues in 0 files

✅ YAML valid | catalog ok | 8 agents catalogued | 10 SKILL refs resolve | frontmatter ok
✅ All agent prompts ≤ 200 lines (max 140)

New CI steps: Level 4 conformance, every agent catalogued, every SKILL.md reference resolves.

Follow-up

A companion PR to shadow-score-spec should promote Level 4 into the spec itself (model independence, seal plurality, workspace isolation, calibration, and the new report fields). This PR ships the reference implementation; the spec should follow.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com