[Pentexa]

Independent Transfer Evaluation & Scientific Integrity (v0.7)

Pre-registered, implementation-neutral protocol for determining whether METARGON truly transfers learned causal knowledge rather than merely obtaining higher benchmark scores.

Protocol (transfer-eval-protocol-v01)

• generalization_audit/configs/transfer_eval_protocol_v01.json (frozen 2026-11-10; distinct from historical frozen criteria which remain untouched).
• Distinguishes 6 quantities: retained-memory benefit, within-episode learning, goal-directed exploration, repeated-task memorization, true transfer to unseen structures, negative transfer.
• Mandatory control arms: fresh, trained-with-memory, trained-memory-wiped, shuffled-memory, planning-only, matched budgets. Hidden rules only for independent labels; agent decisions use observable info only.

Evidence thresholds (Mission C)

• Level 1 (preliminary): improvement vs fresh AND planning-only; measured memory-to-action attribution; reduced benefit after wiping/shuffling; unseen configurations; per-seed vectors + paired bootstrap CIs; no hidden access.
• Level 2 (confirmatory): all level 1 at confirmatory n; primary-delta CI excluding 0; independent audit recomputation.
• Failures/inconclusive outcomes reported as such; no fabricated metadata.

Audit-owned checks (Mission B)

• generalization_audit/verification/transfer_checks.py + --transfer CLI flag + �erify_experiment.py.
• Checks: protocol registration, control-role coverage/resolution, train/eval separation, frozen immutability, memory-ablation pairing, attribution-trace consistency (sum(source_*) == plans_used), budget fairness.
• Missing metadata => INCONCLUSIVE unless reliably obtainable from artifacts.

Frozen integrity (Mission D)

• Seeds sampled once at registration via SHA-256(protocol_id|date|counter), frozen in protocol file; no tuning to EVREN results; no retroactive edits.

Automation (Mission E)

• 15 regression tests (PASS/FAIL/INCONCLUSIVE semantics), ~2s, included in the audit CI job; no multi-hour benchmarks; backward compatible (no default behavior change).

Results

• Audit suite: 164/164 OK (incl. new tests). Repo hygiene OK.
• --transfer on historical memory_v06 manifest: protocol mismatch FAIL (correct), undeclared roles/separation/immutability INCONCLUSIVE, attribution sums PASS.

No merge performed; please review.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.