Scope
Independent reproduction program for three September frontier findings with separate research, implementation, adversarial review, security, testing, reproducibility, and release roles. No role may both define a promotion criterion and certify that it passed.
Track A: EAL-Bench authorization laundering
Primary source: arXiv:2609.01836, submitted 2026-09-01.
Compare model-managed memory, generic Core Memory state plus operational constraints, Core Memory PR 46 event-sourced authorization projection, and exact-state repair where reproducible. Pin EAL-Bench source commit and all model/provider versions.
Report false authority formation, downstream unauthorized effect rate, legitimate completion, false rejection, latency, context cost, event storage, per-domain results, confidence intervals, and all failures.
Adversarial cases must include stale summaries, omitted revocations, incremental memory writes, multi-hop handoffs, forged source events, sequence gaps, reused grant IDs, expiry, malformed input, and source misclassification.
Track B: READY deployment qualification
Primary source: arXiv:2609.02095, submitted 2026-09-02.
Build a workflow qualification experiment that compares autonomous model ranking with minimum-cost oversight policy selection at a frozen reliability target. Use at least one real RuV engineering workflow, not only a synthetic classifier.
Report autonomous accuracy, target reliability, required review rate, reviewer accuracy assumption or measurement, review latency, cost per completed task, false accept, false escalation, version scope, held-out qualification confidence, and requalification triggers.
Falsification: if autonomous ranking and oversight-adjusted ranking are stable across the RuV workload, record the null result and do not add another deployment layer.
Track C: CHIME plus MASkills attribution-before-mutation
Sources: arXiv:2609.02074 and arXiv:2609.02094, both submitted 2026-09-02. MASkills has an MIT-licensed public implementation.
Compare current Dream Machine mutation, stage attribution only, skill-level credit only, and hierarchical stage plus skill attribution. Hold model, tasks, seeds, tools, evaluator, and total experiment budget fixed.
Report held-out success, number of memories/skills, mutation count, attribution agreement, false attribution, rollback count, token and model cost, wall time, protected regressions, and reproducibility.
Falsification: if a deterministic error taxonomy performs within variance of model-based attribution, prefer the simpler mechanism.
Independence and governance
Research role pins primary evidence and licenses. Implementation role cannot edit the frozen evaluator. Security role tests authority expansion and poisoning. Testing role owns malformed and partial-failure cases. Reproducibility role reruns from pinned artifacts. Release role may recommend review but cannot merge.
All originating-team performance claims remain unverified until this issue records matched RuV results.
Scope
Independent reproduction program for three September frontier findings with separate research, implementation, adversarial review, security, testing, reproducibility, and release roles. No role may both define a promotion criterion and certify that it passed.
Track A: EAL-Bench authorization laundering
Primary source: arXiv:2609.01836, submitted 2026-09-01.
Compare model-managed memory, generic Core Memory state plus operational constraints, Core Memory PR 46 event-sourced authorization projection, and exact-state repair where reproducible. Pin EAL-Bench source commit and all model/provider versions.
Report false authority formation, downstream unauthorized effect rate, legitimate completion, false rejection, latency, context cost, event storage, per-domain results, confidence intervals, and all failures.
Adversarial cases must include stale summaries, omitted revocations, incremental memory writes, multi-hop handoffs, forged source events, sequence gaps, reused grant IDs, expiry, malformed input, and source misclassification.
Track B: READY deployment qualification
Primary source: arXiv:2609.02095, submitted 2026-09-02.
Build a workflow qualification experiment that compares autonomous model ranking with minimum-cost oversight policy selection at a frozen reliability target. Use at least one real RuV engineering workflow, not only a synthetic classifier.
Report autonomous accuracy, target reliability, required review rate, reviewer accuracy assumption or measurement, review latency, cost per completed task, false accept, false escalation, version scope, held-out qualification confidence, and requalification triggers.
Falsification: if autonomous ranking and oversight-adjusted ranking are stable across the RuV workload, record the null result and do not add another deployment layer.
Track C: CHIME plus MASkills attribution-before-mutation
Sources: arXiv:2609.02074 and arXiv:2609.02094, both submitted 2026-09-02. MASkills has an MIT-licensed public implementation.
Compare current Dream Machine mutation, stage attribution only, skill-level credit only, and hierarchical stage plus skill attribution. Hold model, tasks, seeds, tools, evaluator, and total experiment budget fixed.
Report held-out success, number of memories/skills, mutation count, attribution agreement, false attribution, rollback count, token and model cost, wall time, protected regressions, and reproducibility.
Falsification: if a deterministic error taxonomy performs within variance of model-based attribution, prefer the simpler mechanism.
Independence and governance
Research role pins primary evidence and licenses. Implementation role cannot edit the frozen evaluator. Security role tests authority expansion and poisoning. Testing role owns malformed and partial-failure cases. Reproducibility role reruns from pinned artifacts. Release role may recommend review but cannot merge.
All originating-team performance claims remain unverified until this issue records matched RuV results.