Skip to content

Post-Merge Independent Audit v0.6 - #40

Merged
Pentexa merged 1 commit into
mainfrom
audit/post-merge-v06
Nov 10, 2026
Merged

Pentexa merged 1 commit into
mainfrom
audit/post-merge-v06

Conversation

@Pentexa

@Pentexa Pentexa commented Nov 10, 2026

Copy link
Copy Markdown
Owner

Post-Merge Independent Audit v0.6

Independent post-merge scientific audit of the integrated repository (Research v0.13 + Academy Memory v0.6 + Quality v0.5 + Artifact Gate).

Headline results

  • Suites: Python 210/210 · Academy 234/234 · Audit 149/149 (incl. new verifier regression tests) · Rust 32+9 · hygiene OK.
  • Manifests (13): 2927 PASS / 0 FAIL / 19 INCONCLUSIVE (memory-limit metadata + generic-format structural items only).
  • Artifact gate: PASS for both results/v13 and results/academy_v01/memory_v06.

Research v0.13 — independent recomputation (33/33 PASS)

  • Stable exact D1: 16.88% → 39.38% (per-seed [0,0,0.675,0] → [0.775,0,0.8,0]); novel: 23.12% → 33.13% ([0,0.25,0.675,0] → [0.275,0.25,0.8,0]).
  • Δ CIs recomputed: stable +39.38 [0.00, 78.75] (includes 0 → directional pilot evidence, correctly worded); novel +33.13 [6.88, 66.25] (excludes 0).
  • -gen ablation equals v0.12.1 rows exactly → rolling generator is the sole driver.
  • F/H/R preservation: all cells == v0.12.1; -nodc 71.88 reproduced; metargon12 in final_l reproduces the v0.12 study exactly.

Academy Memory v0.6 — independent recomputation (34/34 PASS)

  • Fresh 0.3125; original/memory_wiped 0.28125 (bit-identical vectors); controls random_walk 0.0547 / novocab 0.4609; ctx_full 0.5078; atomtx 0.25.
  • All paired deltas + bootstrap CIs (seed 20261108, 2000 rounds) reproduce exactly.
  • EVREN pool: 24/24 solvable & unique; original frozen 0.40625, Δ vs fresh +0.0944 CI [0.0, 0.1875] (directionally positive, not significant).
  • Storage failure confirmed: verified_then_contradicted 56.5–198 on classic variants; mean contradicted ≈67 vs verified ≈3 → zero-shot transfer unproven (absent in measured cells).

Verifier fix (Audit-owned)

  • v12_temporal_minus_notp_exact_D1 / temporal_minus_notp_exact_D1 key ambiguity resolved data-driven (unique delta-reproducing pair, family hint, or value-identical candidates); duplicate-roster entries deduped. 5 regression tests added.
  • No Research/Academy implementation files touched; no results changed.

Deliverables

  • generalization_audit/docs/POST_MERGE_AUDIT_V06.md
  • results/verification_v13/post_merge_audit_v06.json (machine-readable)
  • results/verification_v13/post_merge_b_recompute.json, post_merge_c_recompute.json
  • generalization_audit/tests/test_comparison_key_resolution.py
  • Recompute scripts under generalization_audit/tools/post_merge_*.py

No merge performed; please review.

Copilot AI balanced review requested due to automatic review settings November 10, 2026 08:50

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@Pentexa
Pentexa merged commit a617365 into main Nov 10, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants