Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ and [getting started](docs/getting-started.md) for manual rollouts and NVIDIA
setup. Never install with bare pip/conda.

The supported Linux clone and archive quickstarts passed in
[run 34082153283](https://github.com/teleworksai/aisle/actions/runs/34082153283).
[run 34082153283](https://github.com/moxin-org/aisle/actions/runs/34082153283).
These are development runs; the benchmark's independent-user, release and
physical-evidence gates remain outstanding.

Expand Down
26 changes: 13 additions & 13 deletions docs/AISLE-technical-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -2840,25 +2840,25 @@ understood.

| # | Gap | Why it matters | Section |
|---|---|---|---|
| [#264](https://github.com/teleworksai/aisle/issues/264) | Skill manifests record evidence but not **applicability** | As the library grows, undifferentiated same-capability entries make selection harder — the accumulation benefit can invert | §5.5 |
| [#265](https://github.com/teleworksai/aisle/issues/265) | No **sandbox trust tier** | ADR-37's floor leaves an agent no legitimate way to declare an unproven node so its graph validates | §5.5 |
| [#266](https://github.com/teleworksai/aisle/issues/266) | **Idea trees are not retained** | The pre-registrations that make findings falsifiable are gone; "3/4 expectations met" is currently unverifiable prose | §8.5 |
| [#267](https://github.com/teleworksai/aisle/issues/267) | **SFT label undecided**: `joint_cmd` vs `joint_cmd_safe` | Structural safety and clean credit assignment pull against each other; must be settled before any fine-tune | §10.11 |
| [#268](https://github.com/teleworksai/aisle/issues/268) | **No determinism layer for inference** | The staleness floor couples sim time to wall time, so a loaded host can change the trajectory with every seed identical | §10.11 |
| [#269](https://github.com/teleworksai/aisle/issues/269) | **"Safe Learning" is undefined** in the project's own naming gloss | Students reasonably expect a training loop and find none | §3.5 |
| [#264](https://github.com/moxin-org/aisle/issues/264) | Skill manifests record evidence but not **applicability** | As the library grows, undifferentiated same-capability entries make selection harder — the accumulation benefit can invert | §5.5 |
| [#265](https://github.com/moxin-org/aisle/issues/265) | No **sandbox trust tier** | ADR-37's floor leaves an agent no legitimate way to declare an unproven node so its graph validates | §5.5 |
| [#266](https://github.com/moxin-org/aisle/issues/266) | **Idea trees are not retained** | The pre-registrations that make findings falsifiable are gone; "3/4 expectations met" is currently unverifiable prose | §8.5 |
| [#267](https://github.com/moxin-org/aisle/issues/267) | **SFT label undecided**: `joint_cmd` vs `joint_cmd_safe` | Structural safety and clean credit assignment pull against each other; must be settled before any fine-tune | §10.11 |
| [#268](https://github.com/moxin-org/aisle/issues/268) | **No determinism layer for inference** | The staleness floor couples sim time to wall time, so a loaded host can change the trajectory with every seed identical | §10.11 |
| [#269](https://github.com/moxin-org/aisle/issues/269) | **"Safe Learning" is undefined** in the project's own naming gloss | Students reasonably expect a training loop and find none | §3.5 |

Refreshed 2026-09-14. All six gaps above have since closed (#264 and #265
through ADR-41 and ADR-40; #267 and #268 on 2026-08-17). The current register:

| # | Gap | Why it matters | Where |
|---|---|---|---|
| [#347](https://github.com/teleworksai/aisle/issues/347) | The **typed-vs-monolithic causal study is unrun**; pilot v8 is registered with collection pending on the session-bound confinement attestation (#589–#591) | It is the headline claim; every 4xx spec exists to make its result defensible | README status, ADR-66 |
| [#346](https://github.com/teleworksai/aisle/issues/346) | **No perception-eligible task stratum**: the task-band calibration (v13) records BND-7 failed for the only candidate, 22 of 22 strata | Without an eligible non-oracle task, neither the pilot nor the confirmatory study has an instrument | SPEC 490 |
| [#519](https://github.com/teleworksai/aisle/issues/519) | Matched-arm session admission and common evidence (MON-8/MON-12) not complete | Gate on every scored session in both arms | SPEC 420/440 |
| [#483](https://github.com/teleworksai/aisle/issues/483)–[#486](https://github.com/teleworksai/aisle/issues/486) | **External reviews not arranged**: statistical (STA-12), terminology (CLM-12), independent reproduction (RPR-10/11), external benchmark user (BMK-21) | Confirmatory and release gates that no in-house work can satisfy | SPEC 400/410/530/540 |
| [#562](https://github.com/teleworksai/aisle/issues/562) | No declared **halt contract** for motion nodes (bound, semantics, receipt); ADR-64 already makes a halt terminal | Required by SPEC 460/480/520 as an outcome, defined nowhere; parked until SPEC 460 ratifies or Phase 6 opens | SPEC 520 supplement |
| [#566](https://github.com/teleworksai/aisle/issues/566) | Closed idea-tree entries persist but nothing **retrieves a refuted idea** at the moment of choosing | The failure-side half of "does prior work compound"; parked until an H3 rerun | ADR-41, HAR-7 |
| [#356](https://github.com/teleworksai/aisle/issues/356) | **Hardware** entry gates (M1, judge fidelity) are GPU-gated; SO-101 driver is loopback-tested only | Every hardware claim stays `hardware_pending` | SPEC 520 |
| [#347](https://github.com/moxin-org/aisle/issues/347) | The **typed-vs-monolithic causal study is unrun**; pilot v8 is registered with collection pending on the session-bound confinement attestation (#589–#591) | It is the headline claim; every 4xx spec exists to make its result defensible | README status, ADR-66 |
| [#346](https://github.com/moxin-org/aisle/issues/346) | **No perception-eligible task stratum**: the task-band calibration (v13) records BND-7 failed for the only candidate, 22 of 22 strata | Without an eligible non-oracle task, neither the pilot nor the confirmatory study has an instrument | SPEC 490 |
| [#519](https://github.com/moxin-org/aisle/issues/519) | Matched-arm session admission and common evidence (MON-8/MON-12) not complete | Gate on every scored session in both arms | SPEC 420/440 |
| [#483](https://github.com/moxin-org/aisle/issues/483)–[#486](https://github.com/moxin-org/aisle/issues/486) | **External reviews not arranged**: statistical (STA-12), terminology (CLM-12), independent reproduction (RPR-10/11), external benchmark user (BMK-21) | Confirmatory and release gates that no in-house work can satisfy | SPEC 400/410/530/540 |
| [#562](https://github.com/moxin-org/aisle/issues/562) | No declared **halt contract** for motion nodes (bound, semantics, receipt); ADR-64 already makes a halt terminal | Required by SPEC 460/480/520 as an outcome, defined nowhere; parked until SPEC 460 ratifies or Phase 6 opens | SPEC 520 supplement |
| [#566](https://github.com/moxin-org/aisle/issues/566) | Closed idea-tree entries persist but nothing **retrieves a refuted idea** at the moment of choosing | The failure-side half of "does prior work compound"; parked until an H3 rerun | ADR-41, HAR-7 |
| [#356](https://github.com/moxin-org/aisle/issues/356) | **Hardware** entry gates (M1, judge fidelity) are GPU-gated; SO-101 driver is loopback-tested only | Every hardware claim stays `hardware_pending` | SPEC 520 |

Two threads remain owner-facing rather than contributor-facing: the eval's
**seed set and episode count remain candidate-chosen** (the same self-grading
Expand Down
2 changes: 1 addition & 1 deletion docs/dora-1.0.1-validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ lockstep conformance claim. The legacy fixture runs in free-run mode, while
BRG-1 requires lockstep for acceptance; it also asserts TC-4's hardware
wall-clock band instead of the specified simulation-rate/liveness rules.
Per CON-13, those test changes are paused in
[spec-conflict #497](https://github.com/teleworksai/aisle/issues/497).
[spec-conflict #497](https://github.com/moxin-org/aisle/issues/497).

The remaining throughput work is to resolve the fixture migration, then
measure cold and warm starts separately with a capture boundary that includes
Expand Down
4 changes: 2 additions & 2 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Install Git and rustup as well as Python >= 3.11, managed exclusively through
## 1. Install

```bash
git clone https://github.com/teleworksai/aisle && cd aisle
git clone https://github.com/moxin-org/aisle && cd aisle
uv sync --extra sim --locked
```

Expand Down Expand Up @@ -71,7 +71,7 @@ uv run --extra sim --locked python tools/quickstart.py --runtime-prefix "$AISLE_
This executes graph validation, a public task, bundle validation and reporting.
The quickstart uses the `sim` extra; CUDA experiments use the manual rollout
path below. The Linux clone and archive paths passed with verified source
receipts in [run 34082153283](https://github.com/teleworksai/aisle/actions/runs/34082153283).
receipts in [run 34082153283](https://github.com/moxin-org/aisle/actions/runs/34082153283).
That evidence does not establish independent reproduction or release readiness.

For manual harness and graph commands, make the verified binary available to
Expand Down
2 changes: 1 addition & 1 deletion docs/paper/aisle-paper.md
Original file line number Diff line number Diff line change
Expand Up @@ -929,7 +929,7 @@ verification.)*

## Reproducibility

Repository: teleworksai/aisle (for double-blind review, this link and
Repository: moxin-org/aisle (for double-blind review, this link and
the git metadata are replaced by an anonymized artifact archive; the
run dates in committed findings remain there as provenance and are
deliberately absent from this text). Every campaign: protocol ADR + analyzer tool +
Expand Down
54 changes: 27 additions & 27 deletions docs/project-quality-roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Date: 2026-08-31
Status: ACTIVE planning program
Umbrella tracker: [#359 — AISLE 10/10 project-quality
program](https://github.com/teleworksai/aisle/issues/359)
program](https://github.com/moxin-org/aisle/issues/359)

## Objective

Expand All @@ -24,21 +24,21 @@ simulation, statistics, safety, integrity, and release work can proceed now.

| Dimension | Starting review | Objective exit gate | Issues |
|---|---:|---|---|
| Novelty and potential | 6/10 | Public versioned benchmark, blind evaluation, external user, and a defensible boundary against adjacent systems | [#357](https://github.com/teleworksai/aisle/issues/357), [#358](https://github.com/teleworksai/aisle/issues/358) |
| Technical system quality | 7/10 | Equal-capability control, sealed evaluator, scoped actuation boundary, and adversarial instrument/safety tests | [#344](https://github.com/teleworksai/aisle/issues/344), [#348](https://github.com/teleworksai/aisle/issues/348), [#350](https://github.com/teleworksai/aisle/issues/350), [#353](https://github.com/teleworksai/aisle/issues/353), [#354](https://github.com/teleworksai/aisle/issues/354) |
| Experimental rigor | 3/10 | Session-level estimands, power, replication, randomized blocks, uncertainty/equivalence, no-fault controls, and retained exclusions | [#345](https://github.com/teleworksai/aisle/issues/345), [#346](https://github.com/teleworksai/aisle/issues/346), [#347](https://github.com/teleworksai/aisle/issues/347), [#349](https://github.com/teleworksai/aisle/issues/349) |
| Support for central claim | 2/10 | Replicated typed-dataflow versus monolithic causal study, with null/negative results accepted without reframing | [#344](https://github.com/teleworksai/aisle/issues/344), [#347](https://github.com/teleworksai/aisle/issues/347) |
| Physical-AI relevance | 3/10 | Positive non-oracle result plus SO-101 physical validation and a live-fault operation cell | [#346](https://github.com/teleworksai/aisle/issues/346), [#356](https://github.com/teleworksai/aisle/issues/356) |
| Reproducibility | 8/10 intent | Independent-machine reproduction, complete raw archive, one-command analysis, and DOI | [#353](https://github.com/teleworksai/aisle/issues/353), [#355](https://github.com/teleworksai/aisle/issues/355) |
| Clarity and focus | 4/10 | Mechanically checked claim/evidence matrix and one canonical external architecture/status narrative | [#358](https://github.com/teleworksai/aisle/issues/358) |
| Novelty and potential | 6/10 | Public versioned benchmark, blind evaluation, external user, and a defensible boundary against adjacent systems | [#357](https://github.com/moxin-org/aisle/issues/357), [#358](https://github.com/moxin-org/aisle/issues/358) |
| Technical system quality | 7/10 | Equal-capability control, sealed evaluator, scoped actuation boundary, and adversarial instrument/safety tests | [#344](https://github.com/moxin-org/aisle/issues/344), [#348](https://github.com/moxin-org/aisle/issues/348), [#350](https://github.com/moxin-org/aisle/issues/350), [#353](https://github.com/moxin-org/aisle/issues/353), [#354](https://github.com/moxin-org/aisle/issues/354) |
| Experimental rigor | 3/10 | Session-level estimands, power, replication, randomized blocks, uncertainty/equivalence, no-fault controls, and retained exclusions | [#345](https://github.com/moxin-org/aisle/issues/345), [#346](https://github.com/moxin-org/aisle/issues/346), [#347](https://github.com/moxin-org/aisle/issues/347), [#349](https://github.com/moxin-org/aisle/issues/349) |
| Support for central claim | 2/10 | Replicated typed-dataflow versus monolithic causal study, with null/negative results accepted without reframing | [#344](https://github.com/moxin-org/aisle/issues/344), [#347](https://github.com/moxin-org/aisle/issues/347) |
| Physical-AI relevance | 3/10 | Positive non-oracle result plus SO-101 physical validation and a live-fault operation cell | [#346](https://github.com/moxin-org/aisle/issues/346), [#356](https://github.com/moxin-org/aisle/issues/356) |
| Reproducibility | 8/10 intent | Independent-machine reproduction, complete raw archive, one-command analysis, and DOI | [#353](https://github.com/moxin-org/aisle/issues/353), [#355](https://github.com/moxin-org/aisle/issues/355) |
| Clarity and focus | 4/10 | Mechanically checked claim/evidence matrix and one canonical external architecture/status narrative | [#358](https://github.com/moxin-org/aisle/issues/358) |

Safety is deliberately cross-cutting rather than hidden inside one score:

- [#350](https://github.com/teleworksai/aisle/issues/350) defines and attacks the
- [#350](https://github.com/moxin-org/aisle/issues/350) defines and attacks the
agent-to-actuation threat boundary.
- [#351](https://github.com/teleworksai/aisle/issues/351) separates topology,
- [#351](https://github.com/moxin-org/aisle/issues/351) separates topology,
kinematic enforcement, and empirical semantic outcomes.
- [#352](https://github.com/teleworksai/aisle/issues/352) decides whether AISLE
- [#352](https://github.com/moxin-org/aisle/issues/352) decides whether AISLE
can support an identity-aware semantic authorization claim; otherwise H5 is
permanently narrowed.

Expand All @@ -47,41 +47,41 @@ Safety is deliberately cross-cutting rather than hidden inside one score:
### Wave 0 — rules before results

- [#345 — session-level statistics and power
analysis](https://github.com/teleworksai/aisle/issues/345)
analysis](https://github.com/moxin-org/aisle/issues/345)
- [#353 — treatment integrity
v3](https://github.com/teleworksai/aisle/issues/353)
v3](https://github.com/moxin-org/aisle/issues/353)
- [#354 — independent instrument audit and mutation
benchmark](https://github.com/teleworksai/aisle/issues/354)
benchmark](https://github.com/moxin-org/aisle/issues/354)
- [#358 — claim-to-evidence matrix and architecture
narrative](https://github.com/teleworksai/aisle/issues/358)
narrative](https://github.com/moxin-org/aisle/issues/358)

Exit: experimental units, claim scopes, treatment identity, audit expectations,
and analysis decisions are mechanically explicit.

### Wave 1 — build fair instruments

- [#344 — equal-capability monolithic
control](https://github.com/teleworksai/aisle/issues/344)
control](https://github.com/moxin-org/aisle/issues/344)
- [#346 — non-oracle reachable task
band](https://github.com/teleworksai/aisle/issues/346)
band](https://github.com/moxin-org/aisle/issues/346)
- [#348 — sealed hidden fault bank and
injector](https://github.com/teleworksai/aisle/issues/348)
injector](https://github.com/moxin-org/aisle/issues/348)
- [#350 — actuation threat model and bypass
validation](https://github.com/teleworksai/aisle/issues/350)
validation](https://github.com/moxin-org/aisle/issues/350)
- [#351 — safety exposure and held-command
ablation](https://github.com/teleworksai/aisle/issues/351)
ablation](https://github.com/moxin-org/aisle/issues/351)
- [#352 — semantic authorization
boundary](https://github.com/teleworksai/aisle/issues/352)
boundary](https://github.com/moxin-org/aisle/issues/352)

Exit: expert parity, task reachability, blinding, confinement, and safety
evidence gates pass before confirmatory data collection.

### Wave 2 — confirmatory campaigns

- [#347 — replicated typed versus monolithic causal
study](https://github.com/teleworksai/aisle/issues/347)
study](https://github.com/moxin-org/aisle/issues/347)
- [#349 — replicated typed-evidence versus logs-only fault
study](https://github.com/teleworksai/aisle/issues/349)
study](https://github.com/moxin-org/aisle/issues/349)

Exit: analyzer-derived session-level treatment estimates, uncertainty, no-fault
false-alarm measurements, and all exclusions are retained. A null or negative
Expand All @@ -90,11 +90,11 @@ result is a valid exit.
### Wave 3 — independent and physical closure

- [#355 — independent-machine reproduction and DOI
artifact](https://github.com/teleworksai/aisle/issues/355)
artifact](https://github.com/moxin-org/aisle/issues/355)
- [#356 — SO-101 physical benchmark and live-fault
operation](https://github.com/teleworksai/aisle/issues/356)
operation](https://github.com/moxin-org/aisle/issues/356)
- [#357 — versioned public benchmark and blind evaluation
path](https://github.com/teleworksai/aisle/issues/357)
path](https://github.com/moxin-org/aisle/issues/357)

Exit: an external operator can run and verify the benchmark without the
original campaign machine, and physical claims rest on retained hardware
Expand All @@ -117,7 +117,7 @@ evidence.

## Completion

Close [#359](https://github.com/teleworksai/aisle/issues/359) only after every
Close [#359](https://github.com/moxin-org/aisle/issues/359) only after every
child issue is complete or explicitly dispositioned with a narrower claim, the
claim/evidence matrix has no unsupported headline claim, and an independent
reviewer audits the completed program.
Loading