diff --git a/docs/AISLE-technical-report.md b/docs/AISLE-technical-report.md index 5465d266..0906a6ae 100644 --- a/docs/AISLE-technical-report.md +++ b/docs/AISLE-technical-report.md @@ -8,11 +8,12 @@ Measured results cite the [README status table](../README.md#status), which is canonical; the phase record is [`analysis/reports/phase2_phase3_report.md`](../analysis/reports/phase2_phase3_report.md). Forward-looking sections are labelled as such and describe committed design -direction, not shipped capability. The measured record in §9 is unchanged -since this snapshot; what changed afterwards is the pre-registration program -(SPEC 400–540), the pilot-first execution order (ADR-66), and the still-pending -#347 pilot — summarized in the README's "Since 2026-08-28" block and in -Appendix D below, which was refreshed on 2026-09-14.* +direction, not shipped capability. Section 9 remains the historical 2026-08-16 +measured snapshot; later measurements, including H6, Phase 5, and M3 v2, live +in the canonical README. The post-2026-08-28 pre-registration program (SPEC +400–540), pilot-first execution order (ADR-66), and still-pending #347 pilot +are summarized in the README's "Since 2026-08-28" block and in Appendix D +below, which was refreshed on 2026-09-14.* **Publication purpose.** This broad technical report preserves the complete diff --git a/docs/glossary.md b/docs/glossary.md index 7311538f..e176973a 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -42,8 +42,8 @@ actuation threat model), SFE (470 safety exposure), SEM (480 semantic authorization), BND (490 non-oracle task band), CSE (500 causal study), FEL (510 fault-evidence localization), HWP (520 SO-101 hardware gate), RPR (530 reproduction archive), BMK (540 public benchmark). All fifteen are PROPOSED and -human-review gated; `tools/trace_check.py` still requires a citing test for -every MUST. +human-review gated; `tools/trace_check.py` requires every MUST to have a citing +test or an explicit, reviewable waiver, while strict gates ignore waivers. ### "Learning", defined (issue #269) diff --git a/docs/lab-briefing-2026-09.md b/docs/lab-briefing-2026-09.md index 10483680..0ddb3baf 100644 --- a/docs/lab-briefing-2026-09.md +++ b/docs/lab-briefing-2026-09.md @@ -50,12 +50,14 @@ attestation label, and none is a confirmatory treatment effect. | VLM judge | five configurations refused | zero false promotions by the fidelity gate | | M3 environment ladder, v2 | Spearman 0.746 over 16 graphs × 8 seeds | unattested, self-authored population | -Three observations a lab will care about. Zero wrong-medicine deliveries have -been observed across every campaign, and the repository refuses to call that -a prevention claim. The verifier that would port to hardware is measured to -be conservative rather than accurate, and the project publishes that number -instead of tuning it away. And the project's own review of agent-authored -robot code found three defects in the harness and none in the agents' work. +Three observations a lab will care about. Retained development summaries +report zero wrong-medicine deliveries, but no independently audited table spans +the project-wide session denominator, and the repository refuses to call the +observation a prevention claim. The verifier that would port to hardware is +measured to be conservative rather than accurate, and the project publishes +that number instead of tuning it away. And the project's own review of +agent-authored robot code found three defects in the harness and none in the +agents' work. ## 3. What has not been measured, and why @@ -85,8 +87,10 @@ This is the part to hear before the demo. ## 4. How the project is run, and why it matters for reproducibility -- Specs carry numbered MUSTs; every MUST needs a test that cites its ID, and - CI fails otherwise (`tools/trace_check.py`). +- Specs carry numbered MUSTs; every implemented MUST needs a test that cites + its ID. Unimplemented requirements in PROPOSED specs may carry an explicit, + reviewable waiver; CI fails on any MUST with neither coverage nor a waiver + (`tools/trace_check.py`). - The scene, verifier, reset, and expert graphs are a hash-frozen set; a rollout refuses to start on drift; agents can read the judge and never edit it. - Every run is identified by `(git_sha, env_hash, env_fingerprint, platform, diff --git a/docs/research-program.md b/docs/research-program.md index ebf9f8a0..862c6e62 100644 --- a/docs/research-program.md +++ b/docs/research-program.md @@ -1,7 +1,7 @@ # AISLE research program: agents, evidence, and model orchestration -**Technical framing, August 2026; snapshot refreshed 2026-09-14, commit `fa339ee`** (no new measured result since 2026-08-28; the pre-registration program and ADR-66's pilot-first order are the delta, summarized in the README). Status snapshot: 2026-08-28, commit `93de5e0`; design-doc Phases 2 and 3 closed, the model-tier agenda in §here EXECUTED as Phase 5 (closed 2026-08-27 at measured end-states — VLA competence/latency separated by the lockstep-eval condition, five VLM-judge configurations refused, the env-ladder swap proven mechanically; ledger in [next-phases.md](next-phases.md)), and the hardware entry prepared (ADR-phase6-prep). Any verdict restated here is a dated summary for context; the [README status table](../README.md#status) is canonical and wins on conflict (issue #142). This report states the research object, +**Technical framing, August 2026; snapshot refreshed 2026-09-14, commit `fa339ee`** (no new H-series hypothesis or A-series ablation result since 2026-08-28; the unattested M3 v2 measurement, the pre-registration program, and ADR-66's pilot-first order are the delta, summarized in the README). Status snapshot: 2026-08-28, commit `93de5e0`; design-doc Phases 2 and 3 closed, the model-tier agenda in §here EXECUTED as Phase 5 (closed 2026-08-27 at measured end-states — VLA competence/latency separated by the lockstep-eval condition, five VLM-judge configurations refused, the env-ladder swap proven mechanically; ledger in [next-phases.md](next-phases.md)), and the hardware entry prepared (ADR-phase6-prep). Any verdict restated here is a dated summary for context; the [README status table](../README.md#status) is canonical and wins on conflict (issue #142). This report states the research object, experimental logic, evidence standard, and model-integration agenda for AISLE. It is descriptive rather than normative: the numbered specifications define what the implementation must do, the [README](../README.md#status) owns current