diff --git a/.gitignore b/.gitignore index a08a4292..68e35c41 100644 --- a/.gitignore +++ b/.gitignore @@ -17,3 +17,11 @@ out/ # input to one: it is derived from graphs/, so committing it would be a # second copy of the topology that silently goes stale. *-graph.html + +# ADR-67: receipt of the locally built Nexus wheel (tools/nexus_runtime.py) +.nexus-runtime-receipt.json +# ADR-68: receipt of the locally built rapier wheel (tools/rapier_runtime.py) +.rapier-runtime-receipt.json + +# ADR-67/ADR-68: pinned engine checkouts fetched by tools/engine_sources.py +.engine-sources/ diff --git a/CLAUDE.md b/CLAUDE.md index 3af0f126..94e4f486 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -28,6 +28,10 @@ benchmark acceptance. Run things: `uv run pytest -m unit`, `uv run harness validate graphs/expert_t0.yaml`, `dora run graphs/expert_t0.yaml --uv`. Never install with bare pip/conda. CUDA-only deps are forbidden in default extras (CON-1). +Optional second engine (ADR-67): `harness rollout --sim-engine nexus` runs +the same scene on Nexus; install its wheel with `tools/nexus_runtime.py` +(docs/getting-started.md §3b). Genesis stays the default and the only engine +behind the measured record. ## Quality gates before EVERY commit (inherited from dora-rs/dora practice) Class B/C (nodes, harness, frozen set, contracts): /review on the diff → diff --git a/README.md b/README.md index 36c60d1a..ae62defa 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,12 @@ execution is the intended path.* Agentic auto-research for robot manipulation on open infrastructure: coding agents (Claude Code / Codex) compose and evolve **typed dora-rs dataflows** against a **Genesis** physics scene, with frozen -verification/reset and scoped safety structure. +verification/reset and scoped safety structure. The scene contract is +engine neutral; two optional engines run the same graphs for development, +[Nexus](docs/decisions/ADR-67.md) on the GPU and +[rapier](docs/decisions/ADR-68.md) on the CPU (`--sim-engine nexus|rapier`), +while Genesis stays the default and the only engine behind the measured +record. The **UNRUN confirmatory claim under test** is whether a typed dataflow substrate makes agentic robotics faster, safer, more diff --git a/analysis/freeze/README.md b/analysis/freeze/README.md index faa20a10..7c9cb54b 100644 --- a/analysis/freeze/README.md +++ b/analysis/freeze/README.md @@ -231,3 +231,10 @@ macOS sandbox denial log with start and end controls. A log with missing reports or unprobed executable names remains incomplete. The inherited seed commitments and all pending gates remain unchanged; these registrations do not authorize collection. + +BND v15, FLT v7, CSE v27 and pilot v13 supersede v14, v6, v26 and v12 after the +ADR-67/ADR-68 engine seam moved three matched sources: the shared harness CLI, +the rollout runner and the simulator bridge now select the physics engine +behind `AISLE_SIM_ENGINE`. Calibration rules, seed lineages, decision rules and +every pending gate are preserved, and Genesis remains the default and the only +engine behind these campaigns. diff --git a/analysis/freeze/bnd-task-band-calibration-v15/declaration.json b/analysis/freeze/bnd-task-band-calibration-v15/declaration.json new file mode 100644 index 00000000..19e8f896 --- /dev/null +++ b/analysis/freeze/bnd-task-band-calibration-v15/declaration.json @@ -0,0 +1,183 @@ +{ + "analysis": { + "scripts": [ + "src/aisle/harness/benchmark_statistics.py", + "src/aisle/harness/non_oracle.py", + "src/aisle/harness/perception_audit.py", + "src/aisle/harness/perception_eligibility.py" + ], + "seed": 490001 + }, + "artifacts": { + "candidate_engineering": "graphs/expert_t2.yaml", + "candidate_short_composition": "graphs/expert_t1_l2.yaml", + "lock": "uv.lock", + "perception_audit_report": "analysis/perception-audit/records/bnd-perception-corpus-03/report.json", + "perception_auditor": "src/aisle/harness/perception_audit.py", + "perception_cli": "src/aisle/harness/cli.py", + "perception_eligibility": "analysis/perception-audit/records/bnd-perception-corpus-03/eligibility.json", + "perception_eligibility_cli": "tools/perception_eligibility.py", + "perception_eligibility_module": "src/aisle/harness/perception_eligibility.py", + "perception_envelope": "analysis/perception-audit/envelope.json", + "perception_model_lock": "src/aisle/verifier/models.lock", + "perception_models": "src/aisle/verifier/models.py", + "perception_raw_predictions": "analysis/perception-audit/records/bnd-perception-corpus-03/raw-predictions.json.gz", + "perception_record_cli": "tools/perception_record.py", + "perception_run_manifest": "analysis/perception-audit/records/bnd-perception-corpus-03/manifest.json", + "public_observation_source:docs/decisions/ADR-pilot-public-observations.md": "docs/decisions/ADR-pilot-public-observations.md", + "public_observation_source:docs/monolithic/public-observations.md": "docs/monolithic/public-observations.md", + "public_observation_source:registry/manifests/pilot-policy-surface.yaml": "registry/manifests/pilot-policy-surface.yaml", + "public_observation_source:src/aisle/harness/pilot_model_cache.py": "src/aisle/harness/pilot_model_cache.py", + "public_observation_source:src/aisle/harness/pilot_policy_surface.py": "src/aisle/harness/pilot_policy_surface.py", + "registry": "registry/manifests", + "research_prompt": "harness/CLAUDE.research.md", + "seed_commitment_predecessor": "analysis/freeze/bnd-task-band-calibration-v14/freeze-manifest.json", + "spec": "specs/490-non-oracle-task-band.md", + "validators": "src/aisle/harness/non_oracle.py" + }, + "budgets": { + "evidence_kind": "unscored_pilot", + "heldout_seeds_per_artifact": 32, + "session_tokens": 200000, + "session_wall_s": 3600, + "sessions_per_interface_per_candidate": 16 + }, + "campaign_id": "bnd-task-band-calibration-v15", + "commands": [ + "AISLE_FRAME_CAPTURE_PERIOD_S=0.1 uv run harness rollout --graph graphs/expert_t1_l2.yaml --tier T1 --embodiment franka --perception L2 --episodes 32 --seeds 0..31 --no-idea-gate --env-baseline local --run-id bnd-perception-corpus-03", + "uv run harness perception audit --run runs/bnd-perception-corpus-03 --envelope analysis/perception-audit/envelope.json --output /tmp/bnd-perception-corpus-03-report.json", + "uv run python tools/perception_record.py --report /tmp/bnd-perception-corpus-03-report.json --run runs/bnd-perception-corpus-03 --output analysis/perception-audit/records/bnd-perception-corpus-03", + "uv run python tools/perception_eligibility.py --report analysis/perception-audit/records/bnd-perception-corpus-03/report.json --envelope analysis/perception-audit/envelope.json --candidate t1-l2-expert-graph --role short_composition --graph graphs/expert_t1_l2.yaml --output analysis/perception-audit/records/bnd-perception-corpus-03/eligibility.json", + "uv run harness freeze check --manifest analysis/freeze/bnd-task-band-calibration-v14/freeze-manifest.json --allow-withheld-seeds" + ], + "decision_rules": { + "alpha": 0.05, + "equivalence_margin": { + "expert_parity": { + "completion_time_s": 10.0, + "sidedness": "two-sided 0.90", + "success": 0.15 + } + }, + "multiplicity": "per-stratum floors are all-must-pass; no pooling rescue (BND-7)", + "power": 0.8, + "smallest_effect": { + "measure": "per-stratum identity accuracy floor", + "value": 0.9 + }, + "stopping_rule": "16 randomized sessions per interface per candidate (BND-9); no threshold relaxed after results (BND-10)" + }, + "disposition": "short_composition candidate expert_t1_l2 is not perception-eligible on the frozen envelope (BND-7): 22 of 22 strata fail, led by target_class=ibuprofen (68 of 80 evaluation frames refused) with every other target's lower bound below the 0.90 floor; no threshold was changed (BND-10); role open pending a new human-reviewed candidate round; the ADR-66 pilot may still run on this candidate with BND-7 recorded as a machine-gate limitation", + "endpoints": { + "primary": [ + { + "aggregation": "proportion per interface with exact interval; selection by BND-10 rule on pooled rate", + "direction": "nearest 0.50", + "id": "E-BND-1", + "outcome": "pilot_session_success_intent_to_treat", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "per stratum lower bound vs frozen floor", + "direction": "higher", + "id": "E-BND-2", + "outcome": "perception_identity_accuracy_by_stratum", + "status": "inferential", + "unit": "frame_record" + } + ], + "secondary": [ + { + "aggregation": "paired equivalence interval within frozen margins (BND-8)", + "direction": "equivalent", + "id": "E-BND-3", + "outcome": "expert_parity_success_and_completion_time", + "status": "inferential", + "unit": "paired_dev_seed" + }, + { + "aggregation": "per stratum", + "direction": "within envelope", + "id": "E-BND-4", + "outcome": "perception_refusal_and_latency", + "status": "descriptive", + "unit": "frame_record" + } + ] + }, + "exclusions": { + "deviation_policy": "any behaviour-affecting change after registration is a documented deviation record and a new campaign version id; confirmatory data never revises the registered protocol", + "infrastructure": [ + "session launcher failure before first agent turn", + "provider outage recorded by the controller" + ], + "rerun_policy": "a permitted rerun receives a new randomized id; the original row is retained with its exclusion reason and never replaced in place", + "treatment_integrity": [ + "pre-registered treatment-integrity invalidation only; reported, never replaced (BND-9)", + "any oracle-derived signal on a policy path (BND-2)" + ] + }, + "hypotheses": [ + { + "direction": "band membership", + "id": "H-BND-1", + "statement": "At least one candidate per role is perception-eligible, expert-parity-passing, and lands in the 0.20-0.80 intent-to-treat pilot band on both interfaces (BND-7..9)." + } + ], + "instrument_set": { + "items": [ + "short_composition:t1-l2-expert-graph", + "engineering:t2-label-read-stack" + ], + "kind": "task_candidate" + }, + "integrity_checks": [ + { + "gate": "CON-7 frozen set", + "kind": "machine_check", + "owner_role": "env_hash checker", + "record": "tools/env_hash.json", + "status": "passed" + }, + { + "gate": "BND-5 perception audit", + "kind": "machine_check", + "owner_role": "perception auditor (hardened, corpus-03, 32 development seeds, envelope unchanged); 22 strata failed their floor", + "record": "analysis/perception-audit/records/bnd-perception-corpus-03/report.json", + "status": "failed" + }, + { + "gate": "BND-8 monolithic expert parity", + "kind": "machine_check", + "owner_role": "parity harness; monolithic control exists, independent authorship and operator evidence remain pending (MON-9)", + "record": null, + "status": "pending" + }, + { + "gate": "BND-1 human-reviewed candidate amendment", + "kind": "external_review", + "owner_role": "project owner (CON-14)", + "record": null, + "status": "pending" + }, + { + "gate": "seed commitment verification", + "kind": "machine_check", + "owner_role": "holder of the original private seed and salt sources", + "record": null, + "status": "pending" + } + ], + "issue": 346, + "purpose": "calibration", + "schema_version": "aisle.freeze.declaration.v1", + "seed_commitment": { + "inherited_from": "analysis/freeze/bnd-task-band-calibration-v14/freeze-manifest.json", + "rule": "32 distinct integers in [1000,9999) drawn once from the OS CSPRNG, retained only under ~/aisle-private/freeze/bnd-task-band-calibration-v1/, disjoint by value from every committed dev/pilot seed list; revealed after the campaign closes", + "salt_source": "~/aisle-private/freeze/bnd-task-band-calibration-v2/salt.bin", + "seeds_source": "~/aisle-private/freeze/bnd-task-band-calibration-v2/seeds.json" + }, + "spec": "490", + "superseded": "bnd-task-band-calibration-v14 (the perception CLI artifact src/aisle/harness/cli.py gained the ADR-67/ADR-68 `--sim-engine` selection on every sim-launching subcommand; calibration rules, seed lineage, pending gates and every other artifact are preserved, and Genesis remains the default and the only engine behind this campaign)" +} diff --git a/analysis/freeze/bnd-task-band-calibration-v15/freeze-manifest.json b/analysis/freeze/bnd-task-band-calibration-v15/freeze-manifest.json new file mode 100644 index 00000000..8828876d --- /dev/null +++ b/analysis/freeze/bnd-task-band-calibration-v15/freeze-manifest.json @@ -0,0 +1,247 @@ +{ + "analysis_script_hashes": { + "src/aisle/harness/benchmark_statistics.py": "sha256:22cde3e9d7c5cc80874c7eec30aa2898e30476d6c939c94ab3009e8b12af641e", + "src/aisle/harness/non_oracle.py": "sha256:723318241bb9ce7065e97bcf66557c2180341bd0c0264ce94aa7bdd50cd52f7d", + "src/aisle/harness/perception_audit.py": "sha256:9f8515abe6a8a51737ed42f8720dbc202156e2a26f13b7100c2f30713b999613", + "src/aisle/harness/perception_eligibility.py": "sha256:954d8af7b487fd653f80e51c58406d6bd8be086e47b100a88295afbe1db9cde4" + }, + "analysis_seed": 490001, + "artifact_hashes": { + "candidate_engineering": "sha256:ba50a70c43b43f6e5ea04745f00094851119c52bdf9f483b91dd27ecfe1f437b", + "candidate_short_composition": "sha256:023eef3fb51f41e0fbe78f0b430989c4a326c617b39dd01ea5662b445aaa350c", + "lock": "sha256:9b8a82a17ba68c192f045405feed3ae7046af9568bddcb398292c20f0ed6f5a8", + "perception_audit_report": "sha256:27134904b4b739423085a9b3c9b23678658d39f7a48e00cd2fbeaedea8c7a4d8", + "perception_auditor": "sha256:9f8515abe6a8a51737ed42f8720dbc202156e2a26f13b7100c2f30713b999613", + "perception_cli": "sha256:a5488dc0350278b2618be9e25043bfcef5ae69610d2de750dffd01c9afa98b44", + "perception_eligibility": "sha256:d18422c4bad1b2c7ebe1cd894cf213d6a7d5447856739456fdd6b8e7e25de7e2", + "perception_eligibility_cli": "sha256:62c5b8facb273cf7ce50aa21f83aeff04efd7e338d0866adde4973e2f0970ccc", + "perception_eligibility_module": "sha256:954d8af7b487fd653f80e51c58406d6bd8be086e47b100a88295afbe1db9cde4", + "perception_envelope": "sha256:263a1c1fd2311ff5f4611e7164431ccf7a22048a6e5f49aa7485676818fdb30a", + "perception_model_lock": "sha256:3fa7271740f3ecd361213f45de140452f41216b80600e343e2daa9f2a413eab5", + "perception_models": "sha256:1b2905181bc757f8390666c2a4c7303f674a6d8a31232e8feae582b5cc24909f", + "perception_raw_predictions": "sha256:7f0ec77b6e562ed2885bfce94af3fcf698b6b0f19a99f1ef78bee3deadce49f8", + "perception_record_cli": "sha256:9cb60f16d8a3f4c8dc9cd0872721dc1574ab67b506142e43ce6f8f852eb0a5d7", + "perception_run_manifest": "sha256:bbcabd8fe7bae609b0d78be6849bdfab69ae77efd67d6de044b363caf70da834", + "public_observation_source:docs/decisions/ADR-pilot-public-observations.md": "sha256:0bf46aac693fa9b9e78997d6843cb87112f7716d006a0f4a63e987f3eca5ea5e", + "public_observation_source:docs/monolithic/public-observations.md": "sha256:1b395b766ef5345169e2047a3a86f80648f5d265f3d4445315f314214c7068b7", + "public_observation_source:registry/manifests/pilot-policy-surface.yaml": "sha256:5e189856d64aa49c90b573879d5b71abe230a2790afad5b9214a061d521de655", + "public_observation_source:src/aisle/harness/pilot_model_cache.py": "sha256:17a90497a33015388f5426dad4d5c3e29700a8b48ccb47a6e02380c3274339f5", + "public_observation_source:src/aisle/harness/pilot_policy_surface.py": "sha256:bb08dc54cf6a86907d68bab554baab1d562ec754b2736812e68798b96395cf80", + "registry": "sha256:2c3bb1574fddde14007e63c5e7a579c8de0b34cbbf74e874e7170b84ec9cce45", + "research_prompt": "sha256:e736bbd6caa7236dbfafe92ae42c0c7f13088764af1b753f6685d398c111b7c0", + "seed_commitment_predecessor": "sha256:c379134e35d9c90986342424a433adf079572bd583241a91adf27e8f6a8172c3", + "spec": "sha256:e89398651eb537654d9e326f3fb53ba9ba98d77377522100d751debb55e86a82", + "validators": "sha256:723318241bb9ce7065e97bcf66557c2180341bd0c0264ce94aa7bdd50cd52f7d" + }, + "campaign_id": "bnd-task-band-calibration-v15", + "commands": [ + "AISLE_FRAME_CAPTURE_PERIOD_S=0.1 uv run harness rollout --graph graphs/expert_t1_l2.yaml --tier T1 --embodiment franka --perception L2 --episodes 32 --seeds 0..31 --no-idea-gate --env-baseline local --run-id bnd-perception-corpus-03", + "uv run harness perception audit --run runs/bnd-perception-corpus-03 --envelope analysis/perception-audit/envelope.json --output /tmp/bnd-perception-corpus-03-report.json", + "uv run python tools/perception_record.py --report /tmp/bnd-perception-corpus-03-report.json --run runs/bnd-perception-corpus-03 --output analysis/perception-audit/records/bnd-perception-corpus-03", + "uv run python tools/perception_eligibility.py --report analysis/perception-audit/records/bnd-perception-corpus-03/report.json --envelope analysis/perception-audit/envelope.json --candidate t1-l2-expert-graph --role short_composition --graph graphs/expert_t1_l2.yaml --output analysis/perception-audit/records/bnd-perception-corpus-03/eligibility.json", + "uv run harness freeze check --manifest analysis/freeze/bnd-task-band-calibration-v14/freeze-manifest.json --allow-withheld-seeds" + ], + "declaration": { + "analysis": { + "scripts": [ + "src/aisle/harness/benchmark_statistics.py", + "src/aisle/harness/non_oracle.py", + "src/aisle/harness/perception_audit.py", + "src/aisle/harness/perception_eligibility.py" + ], + "seed": 490001 + }, + "artifacts": { + "candidate_engineering": "graphs/expert_t2.yaml", + "candidate_short_composition": "graphs/expert_t1_l2.yaml", + "lock": "uv.lock", + "perception_audit_report": "analysis/perception-audit/records/bnd-perception-corpus-03/report.json", + "perception_auditor": "src/aisle/harness/perception_audit.py", + "perception_cli": "src/aisle/harness/cli.py", + "perception_eligibility": "analysis/perception-audit/records/bnd-perception-corpus-03/eligibility.json", + "perception_eligibility_cli": "tools/perception_eligibility.py", + "perception_eligibility_module": "src/aisle/harness/perception_eligibility.py", + "perception_envelope": "analysis/perception-audit/envelope.json", + "perception_model_lock": "src/aisle/verifier/models.lock", + "perception_models": "src/aisle/verifier/models.py", + "perception_raw_predictions": "analysis/perception-audit/records/bnd-perception-corpus-03/raw-predictions.json.gz", + "perception_record_cli": "tools/perception_record.py", + "perception_run_manifest": "analysis/perception-audit/records/bnd-perception-corpus-03/manifest.json", + "public_observation_source:docs/decisions/ADR-pilot-public-observations.md": "docs/decisions/ADR-pilot-public-observations.md", + "public_observation_source:docs/monolithic/public-observations.md": "docs/monolithic/public-observations.md", + "public_observation_source:registry/manifests/pilot-policy-surface.yaml": "registry/manifests/pilot-policy-surface.yaml", + "public_observation_source:src/aisle/harness/pilot_model_cache.py": "src/aisle/harness/pilot_model_cache.py", + "public_observation_source:src/aisle/harness/pilot_policy_surface.py": "src/aisle/harness/pilot_policy_surface.py", + "registry": "registry/manifests", + "research_prompt": "harness/CLAUDE.research.md", + "seed_commitment_predecessor": "analysis/freeze/bnd-task-band-calibration-v14/freeze-manifest.json", + "spec": "specs/490-non-oracle-task-band.md", + "validators": "src/aisle/harness/non_oracle.py" + }, + "budgets": { + "evidence_kind": "unscored_pilot", + "heldout_seeds_per_artifact": 32, + "session_tokens": 200000, + "session_wall_s": 3600, + "sessions_per_interface_per_candidate": 16 + }, + "campaign_id": "bnd-task-band-calibration-v15", + "commands": [ + "AISLE_FRAME_CAPTURE_PERIOD_S=0.1 uv run harness rollout --graph graphs/expert_t1_l2.yaml --tier T1 --embodiment franka --perception L2 --episodes 32 --seeds 0..31 --no-idea-gate --env-baseline local --run-id bnd-perception-corpus-03", + "uv run harness perception audit --run runs/bnd-perception-corpus-03 --envelope analysis/perception-audit/envelope.json --output /tmp/bnd-perception-corpus-03-report.json", + "uv run python tools/perception_record.py --report /tmp/bnd-perception-corpus-03-report.json --run runs/bnd-perception-corpus-03 --output analysis/perception-audit/records/bnd-perception-corpus-03", + "uv run python tools/perception_eligibility.py --report analysis/perception-audit/records/bnd-perception-corpus-03/report.json --envelope analysis/perception-audit/envelope.json --candidate t1-l2-expert-graph --role short_composition --graph graphs/expert_t1_l2.yaml --output analysis/perception-audit/records/bnd-perception-corpus-03/eligibility.json", + "uv run harness freeze check --manifest analysis/freeze/bnd-task-band-calibration-v14/freeze-manifest.json --allow-withheld-seeds" + ], + "decision_rules": { + "alpha": 0.05, + "equivalence_margin": { + "expert_parity": { + "completion_time_s": 10.0, + "sidedness": "two-sided 0.90", + "success": 0.15 + } + }, + "multiplicity": "per-stratum floors are all-must-pass; no pooling rescue (BND-7)", + "power": 0.8, + "smallest_effect": { + "measure": "per-stratum identity accuracy floor", + "value": 0.9 + }, + "stopping_rule": "16 randomized sessions per interface per candidate (BND-9); no threshold relaxed after results (BND-10)" + }, + "disposition": "short_composition candidate expert_t1_l2 is not perception-eligible on the frozen envelope (BND-7): 22 of 22 strata fail, led by target_class=ibuprofen (68 of 80 evaluation frames refused) with every other target's lower bound below the 0.90 floor; no threshold was changed (BND-10); role open pending a new human-reviewed candidate round; the ADR-66 pilot may still run on this candidate with BND-7 recorded as a machine-gate limitation", + "endpoints": { + "primary": [ + { + "aggregation": "proportion per interface with exact interval; selection by BND-10 rule on pooled rate", + "direction": "nearest 0.50", + "id": "E-BND-1", + "outcome": "pilot_session_success_intent_to_treat", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "per stratum lower bound vs frozen floor", + "direction": "higher", + "id": "E-BND-2", + "outcome": "perception_identity_accuracy_by_stratum", + "status": "inferential", + "unit": "frame_record" + } + ], + "secondary": [ + { + "aggregation": "paired equivalence interval within frozen margins (BND-8)", + "direction": "equivalent", + "id": "E-BND-3", + "outcome": "expert_parity_success_and_completion_time", + "status": "inferential", + "unit": "paired_dev_seed" + }, + { + "aggregation": "per stratum", + "direction": "within envelope", + "id": "E-BND-4", + "outcome": "perception_refusal_and_latency", + "status": "descriptive", + "unit": "frame_record" + } + ] + }, + "exclusions": { + "deviation_policy": "any behaviour-affecting change after registration is a documented deviation record and a new campaign version id; confirmatory data never revises the registered protocol", + "infrastructure": [ + "session launcher failure before first agent turn", + "provider outage recorded by the controller" + ], + "rerun_policy": "a permitted rerun receives a new randomized id; the original row is retained with its exclusion reason and never replaced in place", + "treatment_integrity": [ + "pre-registered treatment-integrity invalidation only; reported, never replaced (BND-9)", + "any oracle-derived signal on a policy path (BND-2)" + ] + }, + "hypotheses": [ + { + "direction": "band membership", + "id": "H-BND-1", + "statement": "At least one candidate per role is perception-eligible, expert-parity-passing, and lands in the 0.20-0.80 intent-to-treat pilot band on both interfaces (BND-7..9)." + } + ], + "instrument_set": { + "items": [ + "short_composition:t1-l2-expert-graph", + "engineering:t2-label-read-stack" + ], + "kind": "task_candidate" + }, + "integrity_checks": [ + { + "gate": "CON-7 frozen set", + "kind": "machine_check", + "owner_role": "env_hash checker", + "record": "tools/env_hash.json", + "status": "passed" + }, + { + "gate": "BND-5 perception audit", + "kind": "machine_check", + "owner_role": "perception auditor (hardened, corpus-03, 32 development seeds, envelope unchanged); 22 strata failed their floor", + "record": "analysis/perception-audit/records/bnd-perception-corpus-03/report.json", + "status": "failed" + }, + { + "gate": "BND-8 monolithic expert parity", + "kind": "machine_check", + "owner_role": "parity harness; monolithic control exists, independent authorship and operator evidence remain pending (MON-9)", + "record": null, + "status": "pending" + }, + { + "gate": "BND-1 human-reviewed candidate amendment", + "kind": "external_review", + "owner_role": "project owner (CON-14)", + "record": null, + "status": "pending" + }, + { + "gate": "seed commitment verification", + "kind": "machine_check", + "owner_role": "holder of the original private seed and salt sources", + "record": null, + "status": "pending" + } + ], + "issue": 346, + "purpose": "calibration", + "schema_version": "aisle.freeze.declaration.v1", + "seed_commitment": { + "inherited_from": "analysis/freeze/bnd-task-band-calibration-v14/freeze-manifest.json", + "rule": "32 distinct integers in [1000,9999) drawn once from the OS CSPRNG, retained only under ~/aisle-private/freeze/bnd-task-band-calibration-v1/, disjoint by value from every committed dev/pilot seed list; revealed after the campaign closes", + "salt_source": "~/aisle-private/freeze/bnd-task-band-calibration-v2/salt.bin", + "seeds_source": "~/aisle-private/freeze/bnd-task-band-calibration-v2/seeds.json" + }, + "spec": "490", + "superseded": "bnd-task-band-calibration-v14 (the perception CLI artifact src/aisle/harness/cli.py gained the ADR-67/ADR-68 `--sim-engine` selection on every sim-launching subcommand; calibration rules, seed lineage, pending gates and every other artifact are preserved, and Genesis remains the default and the only engine behind this campaign)" + }, + "declaration_sha256": "sha256:cc588a3a47e35a320a877809969f2a57c84c4dc5c298341876d5073248f8d1dc", + "frozen": false, + "gate_record_hashes": { + "CON-7 frozen set": "sha256:4e84ebe40d37cb26ef774e4fc6b46b54df7813ee051042308d1f409a9d9103ac" + }, + "git_head": "6173684e8129dff85bcb3411a1f011bee4b96aab", + "issue": 346, + "pending_gates": [ + "BND-1 human-reviewed candidate amendment", + "BND-5 perception audit", + "BND-8 monolithic expert parity", + "seed commitment verification" + ], + "purpose": "calibration", + "schema_version": "aisle.freeze.manifest.v1", + "seed_commitment": "sha256:5cddb7ca6bc198421c19c6c202d0d9810ccc925a5aefce2d3fea3037b6c4c132", + "spec": "490", + "status": "registered_pending_review", + "timestamp": null, + "timestamp_source": null +} diff --git a/analysis/freeze/cse-causal-study-pilot-v13/declaration.json b/analysis/freeze/cse-causal-study-pilot-v13/declaration.json new file mode 100644 index 00000000..e80692e0 --- /dev/null +++ b/analysis/freeze/cse-causal-study-pilot-v13/declaration.json @@ -0,0 +1,394 @@ +{ + "analysis": { + "scripts": [ + "src/aisle/harness/benchmark_statistics.py", + "src/aisle/harness/causal_study.py", + "src/aisle/harness/pilot_records.py" + ], + "seed": 500001 + }, + "artifacts": { + "agent_adapters": "tools/agent_adapters.py", + "analyzer": "src/aisle/harness/benchmark_statistics.py", + "bnd_calibration_manifest": "analysis/freeze/bnd-task-band-calibration-v11/freeze-manifest.json", + "bnd_perception_audit_report": "analysis/perception-audit/records/bnd-perception-corpus-03/report.json", + "budget_ceiling": "harness/budget.toml", + "dependency_declaration": "pyproject.toml", + "dependency_lock": "uv.lock", + "dora_runtime_selection": "dora-runtime.json", + "frontend_request_authorization_documentation": "docs/benchmark/v1/frontend-request-authorization.md", + "matched_session_decision": "docs/decisions/ADR-62.md", + "matched_session_documentation": "docs/monolithic/matched-session.md", + "matched_source:assets/so101/so101.urdf": "assets/so101/so101.urdf", + "matched_source:docs/monolithic/allowlist-t1-l2.json": "docs/monolithic/allowlist-t1-l2.json", + "matched_source:docs/monolithic/candidates.json": "docs/monolithic/candidates.json", + "matched_source:docs/monolithic/experts-t1-l2.json": "docs/monolithic/experts-t1-l2.json", + "matched_source:docs/monolithic/interface-map-t1-l2.json": "docs/monolithic/interface-map-t1-l2.json", + "matched_source:docs/monolithic/parity-protocol-t1-l2.json": "docs/monolithic/parity-protocol-t1-l2.json", + "matched_source:docs/monolithic/treatment-table-t1-l2.json": "docs/monolithic/treatment-table-t1-l2.json", + "matched_source:env/limits.toml": "env/limits.toml", + "matched_source:experts/monolithic/expert_t1_l2.py": "experts/monolithic/expert_t1_l2.py", + "matched_source:graphs/pilot_t1_l2_monolithic.yaml": "graphs/pilot_t1_l2_monolithic.yaml", + "matched_source:src/aisle/__init__.py": "src/aisle/__init__.py", + "matched_source:src/aisle/embodiment.py": "src/aisle/embodiment.py", + "matched_source:src/aisle/harness/_code_mode_protocol/__init__.py": "src/aisle/harness/_code_mode_protocol/__init__.py", + "matched_source:src/aisle/harness/_code_mode_protocol/descriptor.pb": "src/aisle/harness/_code_mode_protocol/descriptor.pb", + "matched_source:src/aisle/harness/_code_mode_protocol/protocol.proto": "src/aisle/harness/_code_mode_protocol/protocol.proto", + "matched_source:src/aisle/harness/candidate_launch.py": "src/aisle/harness/candidate_launch.py", + "matched_source:src/aisle/harness/candidates.py": "src/aisle/harness/candidates.py", + "matched_source:src/aisle/harness/cli.py": "src/aisle/harness/cli.py", + "matched_source:src/aisle/harness/code_mode_audit.py": "src/aisle/harness/code_mode_audit.py", + "matched_source:src/aisle/harness/code_mode_authority.py": "src/aisle/harness/code_mode_authority.py", + "matched_source:src/aisle/harness/code_mode_proxy.py": "src/aisle/harness/code_mode_proxy.py", + "matched_source:src/aisle/harness/code_mode_runner.py": "src/aisle/harness/code_mode_runner.py", + "matched_source:src/aisle/harness/common.py": "src/aisle/harness/common.py", + "matched_source:src/aisle/harness/frontend_app_server.py": "src/aisle/harness/frontend_app_server.py", + "matched_source:src/aisle/harness/frontend_app_server_audit.py": "src/aisle/harness/frontend_app_server_audit.py", + "matched_source:src/aisle/harness/frontend_dispatch.py": "src/aisle/harness/frontend_dispatch.py", + "matched_source:src/aisle/harness/frontend_dispatch_audit.py": "src/aisle/harness/frontend_dispatch_audit.py", + "matched_source:src/aisle/harness/frontend_request_audit.py": "src/aisle/harness/frontend_request_audit.py", + "matched_source:src/aisle/harness/frontend_request_authority.py": "src/aisle/harness/frontend_request_authority.py", + "matched_source:src/aisle/harness/guard_divergence.py": "src/aisle/harness/guard_divergence.py", + "matched_source:src/aisle/harness/heldout_eval.py": "src/aisle/harness/heldout_eval.py", + "matched_source:src/aisle/harness/hidden_access_log.py": "src/aisle/harness/hidden_access_log.py", + "matched_source:src/aisle/harness/matched_app_server.py": "src/aisle/harness/matched_app_server.py", + "matched_source:src/aisle/harness/matched_collection.py": "src/aisle/harness/matched_collection.py", + "matched_source:src/aisle/harness/matched_dynamic_run.py": "src/aisle/harness/matched_dynamic_run.py", + "matched_source:src/aisle/harness/matched_evidence.py": "src/aisle/harness/matched_evidence.py", + "matched_source:src/aisle/harness/matched_frontend.py": "src/aisle/harness/matched_frontend.py", + "matched_source:src/aisle/harness/matched_run.py": "src/aisle/harness/matched_run.py", + "matched_source:src/aisle/harness/matched_run_launch.py": "src/aisle/harness/matched_run_launch.py", + "matched_source:src/aisle/harness/matched_runtime.py": "src/aisle/harness/matched_runtime.py", + "matched_source:src/aisle/harness/matched_session.py": "src/aisle/harness/matched_session.py", + "matched_source:src/aisle/harness/matched_tool_service.py": "src/aisle/harness/matched_tool_service.py", + "matched_source:src/aisle/harness/matched_tools.py": "src/aisle/harness/matched_tools.py", + "matched_source:src/aisle/harness/monolith.py": "src/aisle/harness/monolith.py", + "matched_source:src/aisle/harness/monolithic.py": "src/aisle/harness/monolithic.py", + "matched_source:src/aisle/harness/monolithic_run_evidence.py": "src/aisle/harness/monolithic_run_evidence.py", + "matched_source:src/aisle/harness/monolithic_run_prepare.py": "src/aisle/harness/monolithic_run_prepare.py", + "matched_source:src/aisle/harness/pilot_records.py": "src/aisle/harness/pilot_records.py", + "matched_source:src/aisle/harness/provider_relay.py": "src/aisle/harness/provider_relay.py", + "matched_source:src/aisle/harness/provider_response_authority.py": "src/aisle/harness/provider_response_authority.py", + "matched_source:src/aisle/harness/provider_runner.py": "src/aisle/harness/provider_runner.py", + "matched_source:src/aisle/harness/provider_source_audit.py": "src/aisle/harness/provider_source_audit.py", + "matched_source:src/aisle/harness/registry.py": "src/aisle/harness/registry.py", + "matched_source:src/aisle/harness/rollout.py": "src/aisle/harness/rollout.py", + "matched_source:src/aisle/harness/rollout_client.py": "src/aisle/harness/rollout_client.py", + "matched_source:src/aisle/harness/simulator_work.py": "src/aisle/harness/simulator_work.py", + "matched_source:src/aisle/harness/traces.py": "src/aisle/harness/traces.py", + "matched_source:src/aisle/harness/treatment_ambient.py": "src/aisle/harness/treatment_ambient.py", + "matched_source:src/aisle/harness/treatment_confinement.py": "src/aisle/harness/treatment_confinement.py", + "matched_source:src/aisle/harness/treatment_integrity.py": "src/aisle/harness/treatment_integrity.py", + "matched_source:src/aisle/harness/treatment_postflight.py": "src/aisle/harness/treatment_postflight.py", + "matched_source:src/aisle/harness/typed_execution_bundle.py": "src/aisle/harness/typed_execution_bundle.py", + "matched_source:src/aisle/harness/typed_graph_audit.py": "src/aisle/harness/typed_graph_audit.py", + "matched_source:src/aisle/harness/typed_graph_hosts.py": "src/aisle/harness/typed_graph_hosts.py", + "matched_source:src/aisle/harness/typed_graph_stage.py": "src/aisle/harness/typed_graph_stage.py", + "matched_source:src/aisle/harness/typed_node_host.py": "src/aisle/harness/typed_node_host.py", + "matched_source:src/aisle/harness/typed_node_requests.py": "src/aisle/harness/typed_node_requests.py", + "matched_source:src/aisle/harness/typed_node_supervisor.py": "src/aisle/harness/typed_node_supervisor.py", + "matched_source:src/aisle/harness/typed_node_worker.py": "src/aisle/harness/typed_node_worker.py", + "matched_source:src/aisle/harness/typed_run_prepare.py": "src/aisle/harness/typed_run_prepare.py", + "matched_source:src/aisle/harness/typed_snapshot.py": "src/aisle/harness/typed_snapshot.py", + "matched_source:src/aisle/harness/typed_stage_provider.py": "src/aisle/harness/typed_stage_provider.py", + "matched_source:src/aisle/harness/typed_validation.py": "src/aisle/harness/typed_validation.py", + "matched_source:src/aisle/harness/typed_worker_launch.py": "src/aisle/harness/typed_worker_launch.py", + "matched_source:src/aisle/harness/typed_worker_provisioning.py": "src/aisle/harness/typed_worker_provisioning.py", + "matched_source:src/aisle/harness/validate.py": "src/aisle/harness/validate.py", + "matched_source:src/aisle/harness/worker_authority_probe.py": "src/aisle/harness/worker_authority_probe.py", + "matched_source:src/aisle/harness/worker_capability.py": "src/aisle/harness/worker_capability.py", + "matched_source:src/aisle/harness/worker_declaration.py": "src/aisle/harness/worker_declaration.py", + "matched_source:src/aisle/harness/worker_network_probe.py": "src/aisle/harness/worker_network_probe.py", + "matched_source:src/aisle/kinematics.py": "src/aisle/kinematics.py", + "matched_source:src/aisle/mobility/guard.py": "src/aisle/mobility/guard.py", + "matched_source:src/aisle/monolith/confinement.py": "src/aisle/monolith/confinement.py", + "matched_source:src/aisle/monolith/primitive_api.py": "src/aisle/monolith/primitive_api.py", + "matched_source:src/aisle/monolith/primitives.py": "src/aisle/monolith/primitives.py", + "matched_source:src/aisle/monolith/proxy.py": "src/aisle/monolith/proxy.py", + "matched_source:src/aisle/monolith/requests.py": "src/aisle/monolith/requests.py", + "matched_source:src/aisle/monolith/supervisor.py": "src/aisle/monolith/supervisor.py", + "matched_source:src/aisle/monolith/wire.py": "src/aisle/monolith/wire.py", + "matched_source:src/aisle/monolith/worker.py": "src/aisle/monolith/worker.py", + "matched_source:src/aisle/monolith/worker_config.py": "src/aisle/monolith/worker_config.py", + "matched_source:src/aisle/monolith/worker_launch.py": "src/aisle/monolith/worker_launch.py", + "matched_source:src/aisle/nodes/budget_guard.py": "src/aisle/nodes/budget_guard.py", + "matched_source:src/aisle/nodes/h6_fault.py": "src/aisle/nodes/h6_fault.py", + "matched_source:src/aisle/nodes/l2_pose.py": "src/aisle/nodes/l2_pose.py", + "matched_source:src/aisle/nodes/label_reader.py": "src/aisle/nodes/label_reader.py", + "matched_source:src/aisle/nodes/monolith_broker.py": "src/aisle/nodes/monolith_broker.py", + "matched_source:src/aisle/nodes/perception_session.py": "src/aisle/nodes/perception_session.py", + "matched_source:src/aisle/scenes/locations.toml": "src/aisle/scenes/locations.toml", + "matched_source:src/aisle/scenes/meds.toml": "src/aisle/scenes/meds.toml", + "matched_source:src/aisle/scenes/pharmacy.py": "src/aisle/scenes/pharmacy.py", + "matched_source:src/aisle/scenes/physics.toml": "src/aisle/scenes/physics.toml", + "matched_source:src/aisle/scenes/planogram.toml": "src/aisle/scenes/planogram.toml", + "matched_source:src/aisle/scenes/store.py": "src/aisle/scenes/store.py", + "matched_source:src/aisle/topics.py": "src/aisle/topics.py", + "matched_source:src/aisle/turn_node.py": "src/aisle/turn_node.py", + "matched_source:src/aisle/turns.py": "src/aisle/turns.py", + "matched_source:src/aisle/verifier/calibration.py": "src/aisle/verifier/calibration.py", + "matched_source:src/aisle/verifier/models.lock": "src/aisle/verifier/models.lock", + "matched_source:src/aisle/verifier/models.py": "src/aisle/verifier/models.py", + "matched_source:src/aisle/verifier/placement.toml": "src/aisle/verifier/placement.toml", + "matched_source:src/aisle/verifier/realistic.py": "src/aisle/verifier/realistic.py", + "matched_source:src/aisle/verifier/stages.py": "src/aisle/verifier/stages.py", + "matched_source:src/aisle/verifier/thresholds.toml": "src/aisle/verifier/thresholds.toml", + "matched_source:tools/campaign.py": "tools/campaign.py", + "matched_source:tools/matched_campaign.py": "tools/matched_campaign.py", + "monolithic_validators": "src/aisle/harness/monolithic.py", + "nested_host_documentation": "docs/monolithic/code-mode-admission.md", + "pilot_assignments_cli": "tools/pilot_assignments.py", + "pilot_decision": "docs/decisions/ADR-66.md", + "pilot_records_cli": "tools/pilot_records.py", + "protocol": "analysis/freeze/cse-causal-study-pilot-v12/protocol.json", + "protocol_template": "analysis/statistics/protocol-template.json", + "provider_admission_documentation": "docs/monolithic/provider-admission.md", + "public_observation_source:analysis/monolithic-control/experts-t1-l2-v1.json": "analysis/monolithic-control/experts-t1-l2-v1.json", + "public_observation_source:analysis/monolithic-control/treatment-table-t1-l2-v1.json": "analysis/monolithic-control/treatment-table-t1-l2-v1.json", + "public_observation_source:docs/decisions/ADR-pilot-public-observations.md": "docs/decisions/ADR-pilot-public-observations.md", + "public_observation_source:docs/monolithic/public-observations.md": "docs/monolithic/public-observations.md", + "public_observation_source:graphs/pilot_t1_l2_monolithic.yaml": "graphs/pilot_t1_l2_monolithic.yaml", + "public_observation_source:graphs/pilot_t1_l2_typed.yaml": "graphs/pilot_t1_l2_typed.yaml", + "public_observation_source:graphs/turn_plans/pilot_t1_l2_monolithic.json": "graphs/turn_plans/pilot_t1_l2_monolithic.json", + "public_observation_source:graphs/turn_plans/pilot_t1_l2_typed.json": "graphs/turn_plans/pilot_t1_l2_typed.json", + "public_observation_source:registry/manifests/pilot-policy-surface.yaml": "registry/manifests/pilot-policy-surface.yaml", + "public_observation_source:src/aisle/harness/pilot_model_cache.py": "src/aisle/harness/pilot_model_cache.py", + "public_observation_source:src/aisle/harness/pilot_policy_surface.py": "src/aisle/harness/pilot_policy_surface.py", + "research_prompt": "harness/CLAUDE.research.md", + "seed_commitment_predecessor": "analysis/freeze/cse-causal-study-pilot-v12/freeze-manifest.json", + "session_runner": "tools/campaign.py", + "session_validators": "src/aisle/harness/causal_study.py", + "simulator_bridge": "src/aisle/nodes/dora_genesis.py", + "spec": "specs/500-typed-monolithic-causal-study.md", + "treatment_integrity": "src/aisle/harness/treatment_integrity.py", + "treatment_postflight": "src/aisle/harness/treatment_postflight.py", + "treatment_table": "analysis/monolithic-control/treatment-table-v1.json" + }, + "budgets": { + "campaign_ceiling": "harness/budget.toml", + "rollouts_per_session": 20, + "session_tokens": 200000, + "session_wall_s": 3600, + "sim_episodes_per_session": 160 + }, + "campaign_id": "cse-causal-study-pilot-v13", + "commands": [ + "uv run harness stats validate --protocol analysis/freeze/cse-causal-study-pilot-v12/protocol.json --purpose power", + "uv run harness stats power --protocol analysis/freeze/cse-causal-study-pilot-v12/protocol.json --output analysis/freeze/cse-causal-study-pilot-v8/power.json", + "uv run harness freeze check --manifest analysis/freeze/cse-causal-study-pilot-v12/freeze-manifest.json --allow-withheld-seeds" + ], + "decision_rules": { + "alpha": 0.05, + "equivalence_margin": { + "cost_at_equal_quality": { + "acceptance_probability": 0.1, + "heldout_quality": 0.1, + "rule": "CSE-5: both must pass or cost claims stay unconditional", + "sidedness": "two-sided 0.90" + } + }, + "multiplicity": "single primary endpoint", + "power": 0.8, + "primary_rule": "one-sided superiority at alpha 0.05 on the pooled Newcombe interval lower bound > 0 with no reversed stratum; otherwise weakened/rejected/undecidable by the #358 rule (CSE-3)", + "smallest_effect": { + "measure": "risk difference in session success", + "value": 0.25 + }, + "stopping_rule": "fixed n from the frozen power analysis using pilot-only control success; absolute floor 10 randomized assignments per arm per agent-system x task stratum (CSE-8); no optional stopping or extension" + }, + "disposition": "registered 2026-09-13; collection pending. Supersedes the zero-session pilot-v1 lineage with the instrument set the pilot can actually run on (T1-L2 realistic pair, #581 to #586). Named limitations: BND-7 not eligible (recorded failed), external-review and parity gates pending, and the session-bound capability audit not merged (an unattested environment stops every session).", + "endpoints": { + "primary": [ + { + "aggregation": "risk difference typed_dataflow - monolithic; exact arm intervals; Newcombe interval; per agent-system x task stratum before pooling (CSE-1, CSE-17)", + "direction": "higher", + "id": "E-CSE-1", + "outcome": "session_success", + "status": "inferential", + "unit": "agent_session" + } + ], + "secondary": [ + { + "aggregation": "counts by arm", + "direction": "descriptive", + "id": "E-CSE-2", + "outcome": "validation_and_launch_failure_class", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "Kaplan-Meier with censoring", + "direction": "lower", + "id": "E-CSE-3", + "outcome": "time_to_first_accepted_system", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "unconditional session estimands; no at-equal-quality wording unless CSE-5 equivalence passes", + "direction": "lower", + "id": "E-CSE-4", + "outcome": "tokens_tool_calls_wall_active_time_sim_work_rollouts", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "counts", + "direction": "descriptive", + "id": "E-CSE-5", + "outcome": "safety_interventions", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "dropped unless a blinded rubric is ratified before freeze (CSE-18)", + "direction": "n/a", + "id": "E-CSE-6", + "outcome": "artifact_legibility", + "status": "exploratory", + "unit": "agent_session" + } + ] + }, + "exclusions": { + "deviation_policy": "any behaviour-affecting change after registration is a documented deviation record and a new campaign version id; confirmatory data never revises the registered protocol", + "infrastructure": [ + "independent-controller-determined, arm-neutral facts only (CSE-14): host outage, provider interruption before first tool call, launcher crash before the session started" + ], + "rerun_policy": "a permitted rerun receives a new randomized id; the original row is retained with its exclusion reason and never replaced in place", + "treatment_integrity": [ + "contamination sentinel tripped (CSE-12)", + "model alias or CLI drift outside the frozen identity (CSE-6)", + "integrity verdict authored session-side (CSE-11)" + ] + }, + "hypotheses": [ + { + "direction": "typed_dataflow - monolithic > 0", + "id": "H-CSE-1", + "statement": "The bundled typed-dataflow surface raises the probability that a fresh coding-agent session produces a launching system passing the hidden held-out acceptance within budget, versus the equal-capability monolithic surface." + } + ], + "instrument_set": { + "agent_systems": [ + { + "binary": "codex", + "binary_sha256": "b973d440acac501fd2594a43e7ca9ce41e0a65b9dfb28d0d7a7837c99e1261e3", + "frozen": true, + "id": "codex-cli", + "install": "private prefix ~/aisle-private/codex-0.153.4 (the qualified darwin-arm64 binary, not the JS shim); ChatGPT-login credentials copied byte-identically into both arms' private CODEX_HOME, hash recorded only", + "version_observed": "0.153.4" + } + ], + "candidate": { + "id": "t1-l2-realistic", + "monolithic_module": "experts/monolithic/expert_t1_l2.py", + "monolithic_template": "graphs/monolithic_t1_l2.yaml", + "perception": "L2", + "reset": "teleport", + "typed_graph": "graphs/expert_t1_l2.yaml", + "verifier": "realistic (oracle demoted to the held-out scoring tap, BND-3)" + }, + "candidate_table_sha256": "08a1dc56502abc4673cb0b54f01664382214bb473c3ef293b1df7b6c74f64778", + "items": [ + "short_composition:t1-l2-realistic" + ], + "kind": "selected_task" + }, + "integrity_checks": [ + { + "gate": "CON-14 spec approval", + "kind": "external_review", + "owner_role": "project owner", + "pilot_record_2026_09_13": "pending; external review, not a pilot prerequisite (ADR-66 section 2); named limitation", + "record": null, + "status": "pending" + }, + { + "gate": "STA-12 independent statistical review", + "kind": "external_review", + "owner_role": "independent statistician", + "pilot_record_2026_09_13": "pending; external review, not a pilot prerequisite (ADR-66 section 2); named limitation", + "record": null, + "status": "pending" + }, + { + "gate": "MON-1 treatment table and expert parity (#344)", + "kind": "machine_check", + "owner_role": "parity harness; control surface exists, independent authorship and operator evidence remain pending", + "pilot_record_2026_09_13": "pending; the T1-L2 pair (#583) passes the interface, table and module checks but has no operator parity record at an immutable hash; named limitation", + "record": null, + "status": "pending" + }, + { + "gate": "BND-12 task-band freeze (#346)", + "kind": "machine_check", + "owner_role": "task-band auditor", + "pilot_record_2026_09_13": "recorded failed: bnd-task-band-calibration-v11 regenerated the BND-5 perception audit on a 32-seed T1-L2 corpus; 22 of 22 strata below the 0.90 floor (BND-7), led by target_class=ibuprofen refusals; role open (BND-10). ADR-66 section 2: a failing machine gate is a named limitation of this pilot, not a blocker", + "record": "analysis/perception-audit/records/bnd-perception-corpus-03/report.json", + "status": "failed" + }, + { + "gate": "TRT confinement/mutation/agent-path (#353)", + "kind": "machine_check", + "owner_role": "integrity controller", + "pilot_record_2026_09_13": "pending; the session-bound capability audit (loopback relay policy) is not merged; an unattested environment is a stop condition for every session", + "record": null, + "status": "pending" + }, + { + "gate": "seed commitment verification", + "kind": "machine_check", + "owner_role": "holder of the original private seed and salt sources", + "pilot_record_2026_09_13": "not applicable to this registration: it commits a fresh private seed source directly (no inheritance); kept for shape parity and left pending", + "public_observation_successor_note": "Inherited from pilot-v8 without access to private seed sources; verification remains pending. This byte-binding successor does not authorize a pilot run.", + "record": null, + "status": "pending" + }, + { + "gate": "MON-8/MON-12 matched-session acceptance (#519)", + "kind": "machine_check", + "owner_role": "matched-session controller; complete acceptance evidence still required", + "pilot_record_2026_09_13": "pending; frontend conformance qualified on the T1 oracle development surface only (#568); the T1-L2 realistic surface runs under the same executor with the same admission checks; named limitation", + "record": null, + "status": "pending" + } + ], + "issue": 347, + "pilot_design": { + "agent_system": "codex-cli 0.153.4 only (the owned provider relay, the ChatGPT-login account header and the pinned conformance profile are Codex-bound)", + "arm_order": "sealed balanced plan (revealed one assignment at a time); no longer a fixed alternation", + "arms": [ + "typed", + "monolithic" + ], + "assignment": "tools/pilot_assignments.py create --protocol analysis/freeze/cse-causal-study-pilot-v8/protocol.json (sealed balanced plan, one typed and one monolithic assignment per temporal block, TRT-8/CSE-7); reveal exactly one assignment before each session; the ledger retains every assignment, started or not", + "below_cse8_floor": "6 per arm is below the CSE-8 floor of 10 by design; this registration can never become a confirmatory estimate (STA-3)", + "decision": "docs/decisions/ADR-66.md (pilot first; agent-drafted under the ADR-65 label, renumbered by #568)", + "executor": "tools/matched_campaign.py admit / run --purpose engineering / collect-run, then the held-out evaluation; retained and unscored, output root ~/aisle-private/raw/cse-causal-study-pilot-v8/", + "records": "tools/pilot_records.py derives one aisle.stats.records.v1 row per ledger assignment; harness stats validate --purpose analysis; harness stats analyze; harness stats power from observed control success (CSE-8)", + "reporting": "analysis/reports/cse_pilot_v8.md; results enter docs/claim-evidence.yaml only as CSE-8 power inputs under the unrun confirmatory claim", + "rerun_policy": "no reruns; a failed attempt is retained under a new session id, never overwritten", + "session_success": "controller-run held-out evaluation of the retained final snapshot on the 32 private seeds after the agent process ends, oracle as the held-out scorer (BND-3); success iff at least 28 of 32 oracle successes; a launch failure counts as a failed seed (CSE-2, CSE-9)", + "sessions_per_arm": 6, + "stop_conditions": [ + "wrong_object outcome", + "guard bypass", + "unattested environment", + "relay bypass (provider traffic outside the owned relay)", + "any oracle field on a policy input" + ], + "task_stratum": "short_composition on candidate t1-l2-realistic (T1 franka, L2 perception, realistic verifier); bnd-task-band-calibration-v11 records it as not perception-eligible (BND-7), carried as a machine-gate limitation per ADR-66" + }, + "purpose": "pilot", + "schema_version": "aisle.freeze.declaration.v1", + "seed_commitment": { + "inherited_from": "analysis/freeze/cse-causal-study-pilot-v12/freeze-manifest.json", + "rule": "32 distinct integers in [1000,9999) drawn once from the OS CSPRNG on 2026-09-13, retained only under ~/aisle-private/freeze/cse-causal-study-pilot-v8/, disjoint by value from every private seed list committed so far (including cse-causal-study-pilot-v1 and the cse-causal-study-v1 confirmatory commitment) and from the committed development seed lists; never inherited by a confirmatory declaration (ADR-66); revealed after the pilot closes. The same directory holds the controller-private 256-bit randomization seed for the sealed assignment plan (TRT-8); its commitment is recorded in the assignment ledger, never here.", + "salt_source": "~/aisle-private/freeze/cse-causal-study-pilot-v8/salt.bin", + "seeds_source": "~/aisle-private/freeze/cse-causal-study-pilot-v8/seeds.json" + }, + "spec": "500", + "superseded": "cse-causal-study-pilot-v12 (the matched source set gained the ADR-67/ADR-68 engine seam: src/aisle/harness/cli.py, rollout.py and src/aisle/nodes/dora_genesis.py now select the physics engine behind AISLE_SIM_ENGINE. The protocol, seed lineage, decision rules and pending gates are unchanged, and Genesis remains the default and the only engine behind this campaign)" +} diff --git a/analysis/freeze/cse-causal-study-pilot-v13/freeze-manifest.json b/analysis/freeze/cse-causal-study-pilot-v13/freeze-manifest.json new file mode 100644 index 00000000..77c50c47 --- /dev/null +++ b/analysis/freeze/cse-causal-study-pilot-v13/freeze-manifest.json @@ -0,0 +1,586 @@ +{ + "analysis_script_hashes": { + "src/aisle/harness/benchmark_statistics.py": "sha256:22cde3e9d7c5cc80874c7eec30aa2898e30476d6c939c94ab3009e8b12af641e", + "src/aisle/harness/causal_study.py": "sha256:1b1c53d6389881c146726d865a741a681f3d5039c9bf99558a53ee368250971e", + "src/aisle/harness/pilot_records.py": "sha256:68ac7ef9afd449d8527e2ffe99a9ef926e06253c466f3421bc1848e9aa094b70" + }, + "analysis_seed": 500001, + "artifact_hashes": { + "agent_adapters": "sha256:848f8f7da5ad978e53712991a688eb74f56c7a77deabee8afae717ad712666ca", + "analyzer": "sha256:22cde3e9d7c5cc80874c7eec30aa2898e30476d6c939c94ab3009e8b12af641e", + "bnd_calibration_manifest": "sha256:8615de2078337065bf6045c3e4bb0f92924987606768fdb699a8ce58ce8f8b57", + "bnd_perception_audit_report": "sha256:27134904b4b739423085a9b3c9b23678658d39f7a48e00cd2fbeaedea8c7a4d8", + "budget_ceiling": "sha256:bcbb79aa146afed1d877da18093175ae144b4607db50119022ef4f8173122cda", + "dependency_declaration": "sha256:add4e718001da7678b404bdf37ea8d9889f563324ffba9b03fe65c46b172ee4c", + "dependency_lock": "sha256:9b8a82a17ba68c192f045405feed3ae7046af9568bddcb398292c20f0ed6f5a8", + "dora_runtime_selection": "sha256:f9eb7ffe285a0f0ed26d50303e31cf1e94d2f93ed62b599071dff5a183c15f53", + "frontend_request_authorization_documentation": "sha256:a1d4e65c0e467af4bd77922b95b12f26ee6478932eb5dd460de34b81249ca86b", + "matched_session_decision": "sha256:d88e0ef18c92cedff7b74dd3ffafed14d92affaec65beb9ddfec2801b365dc61", + "matched_session_documentation": "sha256:ca006514d55b2c1dc1d046a17e59ebcad52c57b006460e9aa05960e02d527d05", + "matched_source:assets/so101/so101.urdf": "sha256:3a65d2d35e68a8d2f0c2cc176d19b884506543c93ba72980145b80abe276022c", + "matched_source:docs/monolithic/allowlist-t1-l2.json": "sha256:6aac85db63a7061ab09950741eba8ac0e77410755d85c427731c45864adc1dc5", + "matched_source:docs/monolithic/candidates.json": "sha256:2fc51e6ff0e4b73a9b203d6ed1debf1cfa40ece9b21a35245d50b9a403f80abb", + "matched_source:docs/monolithic/experts-t1-l2.json": "sha256:68f01e1081afb36964f591fe0e58a630c8e734c8809833279ac608b1c554fe1a", + "matched_source:docs/monolithic/interface-map-t1-l2.json": "sha256:31fede313d1573e29a83c42acc92883ae8ffeb36beee0a62a8d4481ad68cd591", + "matched_source:docs/monolithic/parity-protocol-t1-l2.json": "sha256:2bfc9854fb5cffbe70be341799f208813eee8c02c9d54993b83ce86f6b80f054", + "matched_source:docs/monolithic/treatment-table-t1-l2.json": "sha256:544bd956688b8d85b54ff496dc9d98bdc42676c1c0c5fe5583e276614d4ed0d5", + "matched_source:env/limits.toml": "sha256:1ab1f4fd5d52e7949b81d21933daaa2c66e80ef07fdedf8217280468e0142edc", + "matched_source:experts/monolithic/expert_t1_l2.py": "sha256:93ade82db523fe808312f9d83c88ab3464eee303be1bfc3f824880bf1de7924c", + "matched_source:graphs/pilot_t1_l2_monolithic.yaml": "sha256:ebf12538d0b9def23ce4ea70a9feb375c98535f774bbf16d4d839aaa4e99d27c", + "matched_source:src/aisle/__init__.py": "sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "matched_source:src/aisle/embodiment.py": "sha256:d17c204492d680175cbaf365aaf164e2a6049da23ce125a9b9f21606b79f4cca", + "matched_source:src/aisle/harness/_code_mode_protocol/__init__.py": "sha256:7e7fb09527b647c2bb13787011de8f9a4412e8b6f304e89efa107162d8150e82", + "matched_source:src/aisle/harness/_code_mode_protocol/descriptor.pb": "sha256:459465e666c6a76a70e902c10c58905dfd8417b6c2ca3e038be99624e3542540", + "matched_source:src/aisle/harness/_code_mode_protocol/protocol.proto": "sha256:406d8f47d8d0b95a9d4ae104890e5e52764c9597e9271f75ff434a4ce24748d0", + "matched_source:src/aisle/harness/candidate_launch.py": "sha256:654eb90264703e3ce3ab599bb8e143047eaee288c8852c5b09c74e3147039270", + "matched_source:src/aisle/harness/candidates.py": "sha256:d1b441a30288a6c0500aaa765a24d4aeecca4f8d52e96f633befc50c84ecf5f5", + "matched_source:src/aisle/harness/cli.py": "sha256:a5488dc0350278b2618be9e25043bfcef5ae69610d2de750dffd01c9afa98b44", + "matched_source:src/aisle/harness/code_mode_audit.py": "sha256:24638163f205408da1c6cf20a1b77c80a0d740a3f8ed44696eb90baf7de61f6a", + "matched_source:src/aisle/harness/code_mode_authority.py": "sha256:217ea025f10a4dc02b4b0ff2ff733adf4a695d1098b3c8ce0d2b27b8a140853d", + "matched_source:src/aisle/harness/code_mode_proxy.py": "sha256:333eee561d610489d2f48f9359ad94a3d9d8ff729fbae0fd3dec4acf64be2cb2", + "matched_source:src/aisle/harness/code_mode_runner.py": "sha256:4f91577dcfc6c239bf4111bd6735a64e97d6e22b7ca6287389abae09023cc25d", + "matched_source:src/aisle/harness/common.py": "sha256:ef62bf518a99c8c4154908bc9cf99e68402e6ae6c8e3fb5ec30c3f5fce9b5fac", + "matched_source:src/aisle/harness/frontend_app_server.py": "sha256:d21b9bf218efc3066399b8b22bbe3892e772433da315a9ecd40e2773369e9283", + "matched_source:src/aisle/harness/frontend_app_server_audit.py": "sha256:fb3cf5b8a115eb93e3f292dae694a4318cbe43e9552347fbc25ea8e43102ddbb", + "matched_source:src/aisle/harness/frontend_dispatch.py": "sha256:6bff2abbacca6c332abdb5c5cb4cb3d688d8b00535e3f428633619bf91cd30c8", + "matched_source:src/aisle/harness/frontend_dispatch_audit.py": "sha256:dc75024bd52c21c89a6112da847875a1d0ebf44f53a3704705e0bb8c81b9bc4a", + "matched_source:src/aisle/harness/frontend_request_audit.py": "sha256:a4bc60b2ce1968ada1776324b72ad87025b93d366046029445fdf9b2e92f8830", + "matched_source:src/aisle/harness/frontend_request_authority.py": "sha256:0a061f1528b8bb2071dd1ba7e1d0bbee178b2c982bd6b688b916fb612522e163", + "matched_source:src/aisle/harness/guard_divergence.py": "sha256:af2c4755cabe46fabba913aee4819e05c10caaae79105615d50f8cf4567dad14", + "matched_source:src/aisle/harness/heldout_eval.py": "sha256:56c83164b04ae1b4623e8fd7a0579dd0a9caa7d5dbb891448643464a07893962", + "matched_source:src/aisle/harness/hidden_access_log.py": "sha256:27eabce8642b296afeae87a86b7ef44fa47ff755d479e9148af60a167ba5735c", + "matched_source:src/aisle/harness/matched_app_server.py": "sha256:b7177caca8f1e4d3664b6cef76539b572c2ea6604fe0799c629bc50a52951b52", + "matched_source:src/aisle/harness/matched_collection.py": "sha256:b347127344a4c15fbb73fa9f3ac7c711a3d55b0fa368a11f76683bb1b5bbea14", + "matched_source:src/aisle/harness/matched_dynamic_run.py": "sha256:b7f2b81397bcd232710408b9c6c9f463b3928e37f117d7310d519728f9b4b2c9", + "matched_source:src/aisle/harness/matched_evidence.py": "sha256:6d65476a9a923fd445ea9dad0334e521378353454b873ef59e2b0ff848c650f7", + "matched_source:src/aisle/harness/matched_frontend.py": "sha256:a4b9827af7dbf36f4af5228ca8c87ea36d2d32f8cffc7fcb8fea1923bcc09401", + "matched_source:src/aisle/harness/matched_run.py": "sha256:9ea045eb0abf99661b5bc4e1726f428c83b99ba06f1f2b5441e0daa3cc8ceb56", + "matched_source:src/aisle/harness/matched_run_launch.py": "sha256:8701f034d0bd9873aa7981e3db47849d50bd4465f085e617e6e3193aac070539", + "matched_source:src/aisle/harness/matched_runtime.py": "sha256:c4d81e8309fcf5212315013dbf8e83347a65080e1221f37a0491757834ac2797", + "matched_source:src/aisle/harness/matched_session.py": "sha256:ac784258cff693adee829fef4eaf9a8970d785d347738cafc96edbf06b853754", + "matched_source:src/aisle/harness/matched_tool_service.py": "sha256:1bf6685addbeee541961e1053b2eea68cba66ce110667d8e7720398d49191a17", + "matched_source:src/aisle/harness/matched_tools.py": "sha256:414592a1e1a346716a25b06012d82739253f6440aae4c85979ace3312efa9ccd", + "matched_source:src/aisle/harness/monolith.py": "sha256:d898a3734adef6a72101bcec463fb82265107d1efa13f2121e3fd5f6a1494d43", + "matched_source:src/aisle/harness/monolithic.py": "sha256:1983ed8c26b2371c993c571f33f5c7e4ee81ea2597f9468b43d53c9970465ccf", + "matched_source:src/aisle/harness/monolithic_run_evidence.py": "sha256:4c0cb658f0148eb5d06ad69dceafd39148a2d4787e085cb624047c365c87bb24", + "matched_source:src/aisle/harness/monolithic_run_prepare.py": "sha256:14d80d436e04e3a1a07fe78716bec41c187002a9bac821ff6f11ede67443ac5a", + "matched_source:src/aisle/harness/pilot_records.py": "sha256:68ac7ef9afd449d8527e2ffe99a9ef926e06253c466f3421bc1848e9aa094b70", + "matched_source:src/aisle/harness/provider_relay.py": "sha256:0b1698ac82da51458da5d4cdee6ca9f79334703d8c4a9dbc297af173ec0ff9a2", + "matched_source:src/aisle/harness/provider_response_authority.py": "sha256:9562a8de28967e2eb83910ea5062dc491c8588a6bb4a2a04407f44d59f2251fe", + "matched_source:src/aisle/harness/provider_runner.py": "sha256:2c39d11d67cc4da2f9a2c732e86ba4e9a4b8da88570bdb1ecd1364d4668831cb", + "matched_source:src/aisle/harness/provider_source_audit.py": "sha256:e3be1f9d70288b4f596427a39ab4274399e9fdc74e87043e3c1249596fe1872b", + "matched_source:src/aisle/harness/registry.py": "sha256:214da7bb8db26c80c79262fa79c40a624d8a9981463d38a9c4aa30bc989bc1fb", + "matched_source:src/aisle/harness/rollout.py": "sha256:568f98bb6593f8f007408185956e7a97e7d575359c6c9f8b683bca37ca90c5f2", + "matched_source:src/aisle/harness/rollout_client.py": "sha256:18808db0aa2df356d1a7cc9a538585aa9dcc653abd120e761649f1dff2c27c31", + "matched_source:src/aisle/harness/simulator_work.py": "sha256:2ade87416ab8c5d8902e9734af850b7613ee9ee42f9ca672c9482625e15d3d91", + "matched_source:src/aisle/harness/traces.py": "sha256:c2e6a717a56ce47e296bdc0115b15de9361b72084638b4f3bc66f7e7d032546b", + "matched_source:src/aisle/harness/treatment_ambient.py": "sha256:13dce400a649bc0d01749661517138f8ad1f8cde106caceaedb9f33af6f6d856", + "matched_source:src/aisle/harness/treatment_confinement.py": "sha256:851339ca608e6649317f42a918eecd846e3aa339cba6aee472da03193aca334e", + "matched_source:src/aisle/harness/treatment_integrity.py": "sha256:b37162c6f295c5686da9b3089eb3bd96b69f68f01f41da0a784c4017658b10a6", + "matched_source:src/aisle/harness/treatment_postflight.py": "sha256:27b591c291491ff1e943a62ec29ecd054b6caa29547b7872844cdc67dc403e9c", + "matched_source:src/aisle/harness/typed_execution_bundle.py": "sha256:607945d5a51e3a139ecd35a8dceaf77a2027b0f842a577587616e0490b35c663", + "matched_source:src/aisle/harness/typed_graph_audit.py": "sha256:2266249a3a96e8acdfd8a40e2085eef31c519b351832e9b7bd7c05edb029a186", + "matched_source:src/aisle/harness/typed_graph_hosts.py": "sha256:a5cfd9b3b603f901370579f65320f995d4bc0c71852cf2ed64784d429a942abd", + "matched_source:src/aisle/harness/typed_graph_stage.py": "sha256:0ea37ad396fb3321978c5f8f63d869a8ba0a652f7377cad60f13060b5a5ccbfb", + "matched_source:src/aisle/harness/typed_node_host.py": "sha256:ab3a222a2599f46415af929023d519efd84a427794dd154ef1e2113a7e9f0d4a", + "matched_source:src/aisle/harness/typed_node_requests.py": "sha256:c3712381bc7383beef1e3304ce05f6a4fe27fb16d9b7e97e7f51f8d7cb6de78b", + "matched_source:src/aisle/harness/typed_node_supervisor.py": "sha256:d964517b767bdd087771bf9233c05de61604cf5f705124d6b83be05feb0cb577", + "matched_source:src/aisle/harness/typed_node_worker.py": "sha256:890cb125e12b05afa3609186ceae6d1d45d1d765f653e2c0b7345b00b2c24a89", + "matched_source:src/aisle/harness/typed_run_prepare.py": "sha256:787ba2edc04efa6df752fdba2112ff50e34d2061acb74471c52c78e813382063", + "matched_source:src/aisle/harness/typed_snapshot.py": "sha256:96003b648f09d1348375ca72709537dd80aa4ddecd60e87465011d221d3200ec", + "matched_source:src/aisle/harness/typed_stage_provider.py": "sha256:afe7e37ef08020f5bf19630ef9117af2bdf9d8fe68ceba7f4a352f7c61c8c9ff", + "matched_source:src/aisle/harness/typed_validation.py": "sha256:fa46d29d23d32ee4e65cbc9738ff5af6573607865162363f9d7dcd983b3a0ec9", + "matched_source:src/aisle/harness/typed_worker_launch.py": "sha256:bceac67581c7f143ef778d417217f9a823547c5747f9a2e9a2e6d39bd26c2c4a", + "matched_source:src/aisle/harness/typed_worker_provisioning.py": "sha256:0e594beec5a07829044900d4a93fcde8e2e0e9d50e1e0f7e19809f9d2845b3ba", + "matched_source:src/aisle/harness/validate.py": "sha256:6324c0a684e4d56a8711ff7ed5851a23c1c6131f1040f3f6e80c2f896a2ac955", + "matched_source:src/aisle/harness/worker_authority_probe.py": "sha256:9ee0638062e062c500c7103664eecd541e5d68056bb83368223c09f145dad6ee", + "matched_source:src/aisle/harness/worker_capability.py": "sha256:272c82452cd6f81912593b4584729cb7bb0e8f528d662cb3021e8dfc6ce70e83", + "matched_source:src/aisle/harness/worker_declaration.py": "sha256:15f42f83cae62a8e917f010b3e1571dda81a3cc7e7e3d630ed2b697fb0e8c0c2", + "matched_source:src/aisle/harness/worker_network_probe.py": "sha256:eb0d75fcbd4e8242eb61701f07e02e83ee6e7d9053247df5572f699969bcceb2", + "matched_source:src/aisle/kinematics.py": "sha256:0656cb4c9c4769365c42fc175deefb69d7a7a3dda77991329b1cedf33f7d5aaa", + "matched_source:src/aisle/mobility/guard.py": "sha256:3d0d4ddd757a8ae1840807511f8816f85feff0f2103fc490a8d29db6fa3663d0", + "matched_source:src/aisle/monolith/confinement.py": "sha256:f207d08bac411ae0cb9a5986b3f5f4d55e55d1407e0c0e6889ec5d489b232969", + "matched_source:src/aisle/monolith/primitive_api.py": "sha256:d5a344125ca3f434e489325b6bc7e20422082d18abee8ba7cd4deccda901327a", + "matched_source:src/aisle/monolith/primitives.py": "sha256:e4b3ef279708ef5fbd6e114f524a6c727f3c9d39a41d56dbfbe73d26e3ac515b", + "matched_source:src/aisle/monolith/proxy.py": "sha256:191b4ce164ea431ccbe728ad9ca5b482a09894093ee389643e099c3774df4ca4", + "matched_source:src/aisle/monolith/requests.py": "sha256:04151bf8838eda75be2a9d0c91240607283fb826712b90a8f99eab5d83d30185", + "matched_source:src/aisle/monolith/supervisor.py": "sha256:7dbe215dc5bddd12d414d1b88e896000fc64a9c71746230518eb2f47f1ef1051", + "matched_source:src/aisle/monolith/wire.py": "sha256:3418587b67e3ce2481e68c2809aaafc8e17f244246d33b055ee05f06ac6177a6", + "matched_source:src/aisle/monolith/worker.py": "sha256:931e42442b9f69157eab17cb73f1fa235eb053da7d843915ef889df5f84fb2d7", + "matched_source:src/aisle/monolith/worker_config.py": "sha256:55a26fdfa8b701470e50d54ffa9384d0ed89a0dc157867ac7e0005671e623312", + "matched_source:src/aisle/monolith/worker_launch.py": "sha256:3b823405305819ba4f1bd7cce0ef18a3ecc1276d0a221e78f3e0524ecfb2a8ef", + "matched_source:src/aisle/nodes/budget_guard.py": "sha256:e15bc5f70b295e7dfd32f72e4a5cf72dffca9da50e9e8f6bb18360fb38069abf", + "matched_source:src/aisle/nodes/h6_fault.py": "sha256:c93b8e1607b7627adef9d971753727b9c602be44ae539def9188344332a63485", + "matched_source:src/aisle/nodes/l2_pose.py": "sha256:d2889dac35af94cdf89a280cf734f672098ce95a4a347e1b9797545c2269fda2", + "matched_source:src/aisle/nodes/label_reader.py": "sha256:aeb51080744779f68fb56d6ff408993e92e3e0b929b7d353af7521537f252517", + "matched_source:src/aisle/nodes/monolith_broker.py": "sha256:6fb86d2fdcae8d9ac0314e4cab3100cd925c35a96f74249006392e042847c048", + "matched_source:src/aisle/nodes/perception_session.py": "sha256:cf0fd80a43a338711456d7db546167d082fb6ecc461a828f3cb0e2d42701fc9e", + "matched_source:src/aisle/scenes/locations.toml": "sha256:baea316611ce2f05751ab3058b4b46acbdd9801d8734faca0272c06b2565bcca", + "matched_source:src/aisle/scenes/meds.toml": "sha256:3b25c5eb6586b311fbe8aa3ffc9fdab76cb1b60c9781c3a7a31b12ab6b61dc6b", + "matched_source:src/aisle/scenes/pharmacy.py": "sha256:642e4a57c9225804b44d4e6909fad6514475598cf002dff300b9ce4455225981", + "matched_source:src/aisle/scenes/physics.toml": "sha256:7a400f5cb01a4a93e2a1daa07fa3b2fafd6598643e15965d779c924480f7c36c", + "matched_source:src/aisle/scenes/planogram.toml": "sha256:c579e207de210ed102ff81e15bf0877a91ce0d4ac31ebce2c31e484322f0e576", + "matched_source:src/aisle/scenes/store.py": "sha256:ea79ce1c91d2085ac920fa1ffe87c0c662c875e0e36da5866c1bc60ae5093a66", + "matched_source:src/aisle/topics.py": "sha256:431a0e0bee4c4c3d83f09f9452eb481f238ec0b8cab11b7370bebc117beeb26f", + "matched_source:src/aisle/turn_node.py": "sha256:0c6cd11cd802ebf8934a671caeb4b8a590e5f43b23f03c52515f51eac4288705", + "matched_source:src/aisle/turns.py": "sha256:f0014e3d8ea6b07d4effd10c5bf0b7de08b89ecec1e164580509ad25694b36a4", + "matched_source:src/aisle/verifier/calibration.py": "sha256:a00b91406d27a3b6123701a7a7270258633ad9906c28dbb6f1110fbb07b59568", + "matched_source:src/aisle/verifier/models.lock": "sha256:3fa7271740f3ecd361213f45de140452f41216b80600e343e2daa9f2a413eab5", + "matched_source:src/aisle/verifier/models.py": "sha256:1b2905181bc757f8390666c2a4c7303f674a6d8a31232e8feae582b5cc24909f", + "matched_source:src/aisle/verifier/placement.toml": "sha256:8d8f37165cf4bb4214486597689d40d5df8fb36e2bb8f15e65f6be8801d3717f", + "matched_source:src/aisle/verifier/realistic.py": "sha256:769f6295d3c15a5be02da8ec815a401728059608f2ae92c904d5bec0dd18d156", + "matched_source:src/aisle/verifier/stages.py": "sha256:63848f9e4c0e59020461e3350bf8ca2a8d0ae93cc45cd43c5786c7b6f95a443c", + "matched_source:src/aisle/verifier/thresholds.toml": "sha256:567ac0544a4c382d3ce6547d8df868e6ca37c056b146cf0d1ee11934b1be16d3", + "matched_source:tools/campaign.py": "sha256:cb33ab83f9125ea9708a73f639ca38ad539fbc77e6c5b03e0b49dc2e66abd8f0", + "matched_source:tools/matched_campaign.py": "sha256:882cbdd39c6a9413c8926c392c106eb2d014b3ac5615271bb6e54b38d1f6758b", + "monolithic_validators": "sha256:1983ed8c26b2371c993c571f33f5c7e4ee81ea2597f9468b43d53c9970465ccf", + "nested_host_documentation": "sha256:1c4777e4d8a29882ae5332f96df6b175423e7e5d1f279357fdb1286bace12959", + "pilot_assignments_cli": "sha256:bafcfdf4f79d5d0abd7442355454ea7ca476928afd57ed7c83f38da0c3726cc8", + "pilot_decision": "sha256:8a2f932b7ff6e87c833baa0fd85f2e5d892bed4bd7287e05488eaa8652a85966", + "pilot_records_cli": "sha256:51438c6e14fca6368e1b91ebce77fd13c6bcc821edb61666553750a91d3ab75f", + "protocol": "sha256:99191c6abddd71b19c5e0511ddcd48c67440591f4fe857c4648e5c541e80b115", + "protocol_template": "sha256:9d636f3dc462a2e051ee0b3ded565aff1c5be86331e5b1b6692988509ccf088e", + "provider_admission_documentation": "sha256:856851935d15d8cb0c5653548ae9312441263cce9a3300e36d0863f5bdb96cf4", + "public_observation_source:analysis/monolithic-control/experts-t1-l2-v1.json": "sha256:eca1034c63620f5a901a0b2da4c157391643bc4e5aaf48457ac172ff8a656288", + "public_observation_source:analysis/monolithic-control/treatment-table-t1-l2-v1.json": "sha256:58f7164a95500f514a815b44ef1a681462c13127c3fbff9f887ae3873f21eac2", + "public_observation_source:docs/decisions/ADR-pilot-public-observations.md": "sha256:0bf46aac693fa9b9e78997d6843cb87112f7716d006a0f4a63e987f3eca5ea5e", + "public_observation_source:docs/monolithic/public-observations.md": "sha256:1b395b766ef5345169e2047a3a86f80648f5d265f3d4445315f314214c7068b7", + "public_observation_source:graphs/pilot_t1_l2_monolithic.yaml": "sha256:ebf12538d0b9def23ce4ea70a9feb375c98535f774bbf16d4d839aaa4e99d27c", + "public_observation_source:graphs/pilot_t1_l2_typed.yaml": "sha256:9dd2a3f4a71ed954987861e80a163c644e3a0aec4f3556b5fa353fa8f83aa8cb", + "public_observation_source:graphs/turn_plans/pilot_t1_l2_monolithic.json": "sha256:089558615e62539fb8121c6521d1bcc2af1403d4e896be9ef731b353e9b55d8c", + "public_observation_source:graphs/turn_plans/pilot_t1_l2_typed.json": "sha256:605b00af15cc3e8ba0331468532277c2e8ca4d3bcc196e84978d957fde59f23e", + "public_observation_source:registry/manifests/pilot-policy-surface.yaml": "sha256:5e189856d64aa49c90b573879d5b71abe230a2790afad5b9214a061d521de655", + "public_observation_source:src/aisle/harness/pilot_model_cache.py": "sha256:17a90497a33015388f5426dad4d5c3e29700a8b48ccb47a6e02380c3274339f5", + "public_observation_source:src/aisle/harness/pilot_policy_surface.py": "sha256:bb08dc54cf6a86907d68bab554baab1d562ec754b2736812e68798b96395cf80", + "research_prompt": "sha256:e736bbd6caa7236dbfafe92ae42c0c7f13088764af1b753f6685d398c111b7c0", + "seed_commitment_predecessor": "sha256:b3f9d09b0757c72e7d370b583dbebfabc9afa99b2270ba8571fd88f470f4d2df", + "session_runner": "sha256:cb33ab83f9125ea9708a73f639ca38ad539fbc77e6c5b03e0b49dc2e66abd8f0", + "session_validators": "sha256:1b1c53d6389881c146726d865a741a681f3d5039c9bf99558a53ee368250971e", + "simulator_bridge": "sha256:b4d5e6eab8e60b65f2f0eb29c94f0634a58b921aae0bd643587d400ff689ba7c", + "spec": "sha256:f5096c587cd9687859f2dc80ec807d1bbf959cb4e05c152154e71e47bdc1f593", + "treatment_integrity": "sha256:b37162c6f295c5686da9b3089eb3bd96b69f68f01f41da0a784c4017658b10a6", + "treatment_postflight": "sha256:27b591c291491ff1e943a62ec29ecd054b6caa29547b7872844cdc67dc403e9c", + "treatment_table": "sha256:1b4b37f216ae222a673b7fc83741ec666a994c1e42320f821479455453aef3a0" + }, + "campaign_id": "cse-causal-study-pilot-v13", + "commands": [ + "uv run harness stats validate --protocol analysis/freeze/cse-causal-study-pilot-v12/protocol.json --purpose power", + "uv run harness stats power --protocol analysis/freeze/cse-causal-study-pilot-v12/protocol.json --output analysis/freeze/cse-causal-study-pilot-v8/power.json", + "uv run harness freeze check --manifest analysis/freeze/cse-causal-study-pilot-v12/freeze-manifest.json --allow-withheld-seeds" + ], + "declaration": { + "analysis": { + "scripts": [ + "src/aisle/harness/benchmark_statistics.py", + "src/aisle/harness/causal_study.py", + "src/aisle/harness/pilot_records.py" + ], + "seed": 500001 + }, + "artifacts": { + "agent_adapters": "tools/agent_adapters.py", + "analyzer": "src/aisle/harness/benchmark_statistics.py", + "bnd_calibration_manifest": "analysis/freeze/bnd-task-band-calibration-v11/freeze-manifest.json", + "bnd_perception_audit_report": "analysis/perception-audit/records/bnd-perception-corpus-03/report.json", + "budget_ceiling": "harness/budget.toml", + "dependency_declaration": "pyproject.toml", + "dependency_lock": "uv.lock", + "dora_runtime_selection": "dora-runtime.json", + "frontend_request_authorization_documentation": "docs/benchmark/v1/frontend-request-authorization.md", + "matched_session_decision": "docs/decisions/ADR-62.md", + "matched_session_documentation": "docs/monolithic/matched-session.md", + "matched_source:assets/so101/so101.urdf": "assets/so101/so101.urdf", + "matched_source:docs/monolithic/allowlist-t1-l2.json": "docs/monolithic/allowlist-t1-l2.json", + "matched_source:docs/monolithic/candidates.json": "docs/monolithic/candidates.json", + "matched_source:docs/monolithic/experts-t1-l2.json": "docs/monolithic/experts-t1-l2.json", + "matched_source:docs/monolithic/interface-map-t1-l2.json": "docs/monolithic/interface-map-t1-l2.json", + "matched_source:docs/monolithic/parity-protocol-t1-l2.json": "docs/monolithic/parity-protocol-t1-l2.json", + "matched_source:docs/monolithic/treatment-table-t1-l2.json": "docs/monolithic/treatment-table-t1-l2.json", + "matched_source:env/limits.toml": "env/limits.toml", + "matched_source:experts/monolithic/expert_t1_l2.py": "experts/monolithic/expert_t1_l2.py", + "matched_source:graphs/pilot_t1_l2_monolithic.yaml": "graphs/pilot_t1_l2_monolithic.yaml", + "matched_source:src/aisle/__init__.py": "src/aisle/__init__.py", + "matched_source:src/aisle/embodiment.py": "src/aisle/embodiment.py", + "matched_source:src/aisle/harness/_code_mode_protocol/__init__.py": "src/aisle/harness/_code_mode_protocol/__init__.py", + "matched_source:src/aisle/harness/_code_mode_protocol/descriptor.pb": "src/aisle/harness/_code_mode_protocol/descriptor.pb", + "matched_source:src/aisle/harness/_code_mode_protocol/protocol.proto": "src/aisle/harness/_code_mode_protocol/protocol.proto", + "matched_source:src/aisle/harness/candidate_launch.py": "src/aisle/harness/candidate_launch.py", + "matched_source:src/aisle/harness/candidates.py": "src/aisle/harness/candidates.py", + "matched_source:src/aisle/harness/cli.py": "src/aisle/harness/cli.py", + "matched_source:src/aisle/harness/code_mode_audit.py": "src/aisle/harness/code_mode_audit.py", + "matched_source:src/aisle/harness/code_mode_authority.py": "src/aisle/harness/code_mode_authority.py", + "matched_source:src/aisle/harness/code_mode_proxy.py": "src/aisle/harness/code_mode_proxy.py", + "matched_source:src/aisle/harness/code_mode_runner.py": "src/aisle/harness/code_mode_runner.py", + "matched_source:src/aisle/harness/common.py": "src/aisle/harness/common.py", + "matched_source:src/aisle/harness/frontend_app_server.py": "src/aisle/harness/frontend_app_server.py", + "matched_source:src/aisle/harness/frontend_app_server_audit.py": "src/aisle/harness/frontend_app_server_audit.py", + "matched_source:src/aisle/harness/frontend_dispatch.py": "src/aisle/harness/frontend_dispatch.py", + "matched_source:src/aisle/harness/frontend_dispatch_audit.py": "src/aisle/harness/frontend_dispatch_audit.py", + "matched_source:src/aisle/harness/frontend_request_audit.py": "src/aisle/harness/frontend_request_audit.py", + "matched_source:src/aisle/harness/frontend_request_authority.py": "src/aisle/harness/frontend_request_authority.py", + "matched_source:src/aisle/harness/guard_divergence.py": "src/aisle/harness/guard_divergence.py", + "matched_source:src/aisle/harness/heldout_eval.py": "src/aisle/harness/heldout_eval.py", + "matched_source:src/aisle/harness/hidden_access_log.py": "src/aisle/harness/hidden_access_log.py", + "matched_source:src/aisle/harness/matched_app_server.py": "src/aisle/harness/matched_app_server.py", + "matched_source:src/aisle/harness/matched_collection.py": "src/aisle/harness/matched_collection.py", + "matched_source:src/aisle/harness/matched_dynamic_run.py": "src/aisle/harness/matched_dynamic_run.py", + "matched_source:src/aisle/harness/matched_evidence.py": "src/aisle/harness/matched_evidence.py", + "matched_source:src/aisle/harness/matched_frontend.py": "src/aisle/harness/matched_frontend.py", + "matched_source:src/aisle/harness/matched_run.py": "src/aisle/harness/matched_run.py", + "matched_source:src/aisle/harness/matched_run_launch.py": "src/aisle/harness/matched_run_launch.py", + "matched_source:src/aisle/harness/matched_runtime.py": "src/aisle/harness/matched_runtime.py", + "matched_source:src/aisle/harness/matched_session.py": "src/aisle/harness/matched_session.py", + "matched_source:src/aisle/harness/matched_tool_service.py": "src/aisle/harness/matched_tool_service.py", + "matched_source:src/aisle/harness/matched_tools.py": "src/aisle/harness/matched_tools.py", + "matched_source:src/aisle/harness/monolith.py": "src/aisle/harness/monolith.py", + "matched_source:src/aisle/harness/monolithic.py": "src/aisle/harness/monolithic.py", + "matched_source:src/aisle/harness/monolithic_run_evidence.py": "src/aisle/harness/monolithic_run_evidence.py", + "matched_source:src/aisle/harness/monolithic_run_prepare.py": "src/aisle/harness/monolithic_run_prepare.py", + "matched_source:src/aisle/harness/pilot_records.py": "src/aisle/harness/pilot_records.py", + "matched_source:src/aisle/harness/provider_relay.py": "src/aisle/harness/provider_relay.py", + "matched_source:src/aisle/harness/provider_response_authority.py": "src/aisle/harness/provider_response_authority.py", + "matched_source:src/aisle/harness/provider_runner.py": "src/aisle/harness/provider_runner.py", + "matched_source:src/aisle/harness/provider_source_audit.py": "src/aisle/harness/provider_source_audit.py", + "matched_source:src/aisle/harness/registry.py": "src/aisle/harness/registry.py", + "matched_source:src/aisle/harness/rollout.py": "src/aisle/harness/rollout.py", + "matched_source:src/aisle/harness/rollout_client.py": "src/aisle/harness/rollout_client.py", + "matched_source:src/aisle/harness/simulator_work.py": "src/aisle/harness/simulator_work.py", + "matched_source:src/aisle/harness/traces.py": "src/aisle/harness/traces.py", + "matched_source:src/aisle/harness/treatment_ambient.py": "src/aisle/harness/treatment_ambient.py", + "matched_source:src/aisle/harness/treatment_confinement.py": "src/aisle/harness/treatment_confinement.py", + "matched_source:src/aisle/harness/treatment_integrity.py": "src/aisle/harness/treatment_integrity.py", + "matched_source:src/aisle/harness/treatment_postflight.py": "src/aisle/harness/treatment_postflight.py", + "matched_source:src/aisle/harness/typed_execution_bundle.py": "src/aisle/harness/typed_execution_bundle.py", + "matched_source:src/aisle/harness/typed_graph_audit.py": "src/aisle/harness/typed_graph_audit.py", + "matched_source:src/aisle/harness/typed_graph_hosts.py": "src/aisle/harness/typed_graph_hosts.py", + "matched_source:src/aisle/harness/typed_graph_stage.py": "src/aisle/harness/typed_graph_stage.py", + "matched_source:src/aisle/harness/typed_node_host.py": "src/aisle/harness/typed_node_host.py", + "matched_source:src/aisle/harness/typed_node_requests.py": "src/aisle/harness/typed_node_requests.py", + "matched_source:src/aisle/harness/typed_node_supervisor.py": "src/aisle/harness/typed_node_supervisor.py", + "matched_source:src/aisle/harness/typed_node_worker.py": "src/aisle/harness/typed_node_worker.py", + "matched_source:src/aisle/harness/typed_run_prepare.py": "src/aisle/harness/typed_run_prepare.py", + "matched_source:src/aisle/harness/typed_snapshot.py": "src/aisle/harness/typed_snapshot.py", + "matched_source:src/aisle/harness/typed_stage_provider.py": "src/aisle/harness/typed_stage_provider.py", + "matched_source:src/aisle/harness/typed_validation.py": "src/aisle/harness/typed_validation.py", + "matched_source:src/aisle/harness/typed_worker_launch.py": "src/aisle/harness/typed_worker_launch.py", + "matched_source:src/aisle/harness/typed_worker_provisioning.py": "src/aisle/harness/typed_worker_provisioning.py", + "matched_source:src/aisle/harness/validate.py": "src/aisle/harness/validate.py", + "matched_source:src/aisle/harness/worker_authority_probe.py": "src/aisle/harness/worker_authority_probe.py", + "matched_source:src/aisle/harness/worker_capability.py": "src/aisle/harness/worker_capability.py", + "matched_source:src/aisle/harness/worker_declaration.py": "src/aisle/harness/worker_declaration.py", + "matched_source:src/aisle/harness/worker_network_probe.py": "src/aisle/harness/worker_network_probe.py", + "matched_source:src/aisle/kinematics.py": "src/aisle/kinematics.py", + "matched_source:src/aisle/mobility/guard.py": "src/aisle/mobility/guard.py", + "matched_source:src/aisle/monolith/confinement.py": "src/aisle/monolith/confinement.py", + "matched_source:src/aisle/monolith/primitive_api.py": "src/aisle/monolith/primitive_api.py", + "matched_source:src/aisle/monolith/primitives.py": "src/aisle/monolith/primitives.py", + "matched_source:src/aisle/monolith/proxy.py": "src/aisle/monolith/proxy.py", + "matched_source:src/aisle/monolith/requests.py": "src/aisle/monolith/requests.py", + "matched_source:src/aisle/monolith/supervisor.py": "src/aisle/monolith/supervisor.py", + "matched_source:src/aisle/monolith/wire.py": "src/aisle/monolith/wire.py", + "matched_source:src/aisle/monolith/worker.py": "src/aisle/monolith/worker.py", + "matched_source:src/aisle/monolith/worker_config.py": "src/aisle/monolith/worker_config.py", + "matched_source:src/aisle/monolith/worker_launch.py": "src/aisle/monolith/worker_launch.py", + "matched_source:src/aisle/nodes/budget_guard.py": "src/aisle/nodes/budget_guard.py", + "matched_source:src/aisle/nodes/h6_fault.py": "src/aisle/nodes/h6_fault.py", + "matched_source:src/aisle/nodes/l2_pose.py": "src/aisle/nodes/l2_pose.py", + "matched_source:src/aisle/nodes/label_reader.py": "src/aisle/nodes/label_reader.py", + "matched_source:src/aisle/nodes/monolith_broker.py": "src/aisle/nodes/monolith_broker.py", + "matched_source:src/aisle/nodes/perception_session.py": "src/aisle/nodes/perception_session.py", + "matched_source:src/aisle/scenes/locations.toml": "src/aisle/scenes/locations.toml", + "matched_source:src/aisle/scenes/meds.toml": "src/aisle/scenes/meds.toml", + "matched_source:src/aisle/scenes/pharmacy.py": "src/aisle/scenes/pharmacy.py", + "matched_source:src/aisle/scenes/physics.toml": "src/aisle/scenes/physics.toml", + "matched_source:src/aisle/scenes/planogram.toml": "src/aisle/scenes/planogram.toml", + "matched_source:src/aisle/scenes/store.py": "src/aisle/scenes/store.py", + "matched_source:src/aisle/topics.py": "src/aisle/topics.py", + "matched_source:src/aisle/turn_node.py": "src/aisle/turn_node.py", + "matched_source:src/aisle/turns.py": "src/aisle/turns.py", + "matched_source:src/aisle/verifier/calibration.py": "src/aisle/verifier/calibration.py", + "matched_source:src/aisle/verifier/models.lock": "src/aisle/verifier/models.lock", + "matched_source:src/aisle/verifier/models.py": "src/aisle/verifier/models.py", + "matched_source:src/aisle/verifier/placement.toml": "src/aisle/verifier/placement.toml", + "matched_source:src/aisle/verifier/realistic.py": "src/aisle/verifier/realistic.py", + "matched_source:src/aisle/verifier/stages.py": "src/aisle/verifier/stages.py", + "matched_source:src/aisle/verifier/thresholds.toml": "src/aisle/verifier/thresholds.toml", + "matched_source:tools/campaign.py": "tools/campaign.py", + "matched_source:tools/matched_campaign.py": "tools/matched_campaign.py", + "monolithic_validators": "src/aisle/harness/monolithic.py", + "nested_host_documentation": "docs/monolithic/code-mode-admission.md", + "pilot_assignments_cli": "tools/pilot_assignments.py", + "pilot_decision": "docs/decisions/ADR-66.md", + "pilot_records_cli": "tools/pilot_records.py", + "protocol": "analysis/freeze/cse-causal-study-pilot-v12/protocol.json", + "protocol_template": "analysis/statistics/protocol-template.json", + "provider_admission_documentation": "docs/monolithic/provider-admission.md", + "public_observation_source:analysis/monolithic-control/experts-t1-l2-v1.json": "analysis/monolithic-control/experts-t1-l2-v1.json", + "public_observation_source:analysis/monolithic-control/treatment-table-t1-l2-v1.json": "analysis/monolithic-control/treatment-table-t1-l2-v1.json", + "public_observation_source:docs/decisions/ADR-pilot-public-observations.md": "docs/decisions/ADR-pilot-public-observations.md", + "public_observation_source:docs/monolithic/public-observations.md": "docs/monolithic/public-observations.md", + "public_observation_source:graphs/pilot_t1_l2_monolithic.yaml": "graphs/pilot_t1_l2_monolithic.yaml", + "public_observation_source:graphs/pilot_t1_l2_typed.yaml": "graphs/pilot_t1_l2_typed.yaml", + "public_observation_source:graphs/turn_plans/pilot_t1_l2_monolithic.json": "graphs/turn_plans/pilot_t1_l2_monolithic.json", + "public_observation_source:graphs/turn_plans/pilot_t1_l2_typed.json": "graphs/turn_plans/pilot_t1_l2_typed.json", + "public_observation_source:registry/manifests/pilot-policy-surface.yaml": "registry/manifests/pilot-policy-surface.yaml", + "public_observation_source:src/aisle/harness/pilot_model_cache.py": "src/aisle/harness/pilot_model_cache.py", + "public_observation_source:src/aisle/harness/pilot_policy_surface.py": "src/aisle/harness/pilot_policy_surface.py", + "research_prompt": "harness/CLAUDE.research.md", + "seed_commitment_predecessor": "analysis/freeze/cse-causal-study-pilot-v12/freeze-manifest.json", + "session_runner": "tools/campaign.py", + "session_validators": "src/aisle/harness/causal_study.py", + "simulator_bridge": "src/aisle/nodes/dora_genesis.py", + "spec": "specs/500-typed-monolithic-causal-study.md", + "treatment_integrity": "src/aisle/harness/treatment_integrity.py", + "treatment_postflight": "src/aisle/harness/treatment_postflight.py", + "treatment_table": "analysis/monolithic-control/treatment-table-v1.json" + }, + "budgets": { + "campaign_ceiling": "harness/budget.toml", + "rollouts_per_session": 20, + "session_tokens": 200000, + "session_wall_s": 3600, + "sim_episodes_per_session": 160 + }, + "campaign_id": "cse-causal-study-pilot-v13", + "commands": [ + "uv run harness stats validate --protocol analysis/freeze/cse-causal-study-pilot-v12/protocol.json --purpose power", + "uv run harness stats power --protocol analysis/freeze/cse-causal-study-pilot-v12/protocol.json --output analysis/freeze/cse-causal-study-pilot-v8/power.json", + "uv run harness freeze check --manifest analysis/freeze/cse-causal-study-pilot-v12/freeze-manifest.json --allow-withheld-seeds" + ], + "decision_rules": { + "alpha": 0.05, + "equivalence_margin": { + "cost_at_equal_quality": { + "acceptance_probability": 0.1, + "heldout_quality": 0.1, + "rule": "CSE-5: both must pass or cost claims stay unconditional", + "sidedness": "two-sided 0.90" + } + }, + "multiplicity": "single primary endpoint", + "power": 0.8, + "primary_rule": "one-sided superiority at alpha 0.05 on the pooled Newcombe interval lower bound > 0 with no reversed stratum; otherwise weakened/rejected/undecidable by the #358 rule (CSE-3)", + "smallest_effect": { + "measure": "risk difference in session success", + "value": 0.25 + }, + "stopping_rule": "fixed n from the frozen power analysis using pilot-only control success; absolute floor 10 randomized assignments per arm per agent-system x task stratum (CSE-8); no optional stopping or extension" + }, + "disposition": "registered 2026-09-13; collection pending. Supersedes the zero-session pilot-v1 lineage with the instrument set the pilot can actually run on (T1-L2 realistic pair, #581 to #586). Named limitations: BND-7 not eligible (recorded failed), external-review and parity gates pending, and the session-bound capability audit not merged (an unattested environment stops every session).", + "endpoints": { + "primary": [ + { + "aggregation": "risk difference typed_dataflow - monolithic; exact arm intervals; Newcombe interval; per agent-system x task stratum before pooling (CSE-1, CSE-17)", + "direction": "higher", + "id": "E-CSE-1", + "outcome": "session_success", + "status": "inferential", + "unit": "agent_session" + } + ], + "secondary": [ + { + "aggregation": "counts by arm", + "direction": "descriptive", + "id": "E-CSE-2", + "outcome": "validation_and_launch_failure_class", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "Kaplan-Meier with censoring", + "direction": "lower", + "id": "E-CSE-3", + "outcome": "time_to_first_accepted_system", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "unconditional session estimands; no at-equal-quality wording unless CSE-5 equivalence passes", + "direction": "lower", + "id": "E-CSE-4", + "outcome": "tokens_tool_calls_wall_active_time_sim_work_rollouts", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "counts", + "direction": "descriptive", + "id": "E-CSE-5", + "outcome": "safety_interventions", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "dropped unless a blinded rubric is ratified before freeze (CSE-18)", + "direction": "n/a", + "id": "E-CSE-6", + "outcome": "artifact_legibility", + "status": "exploratory", + "unit": "agent_session" + } + ] + }, + "exclusions": { + "deviation_policy": "any behaviour-affecting change after registration is a documented deviation record and a new campaign version id; confirmatory data never revises the registered protocol", + "infrastructure": [ + "independent-controller-determined, arm-neutral facts only (CSE-14): host outage, provider interruption before first tool call, launcher crash before the session started" + ], + "rerun_policy": "a permitted rerun receives a new randomized id; the original row is retained with its exclusion reason and never replaced in place", + "treatment_integrity": [ + "contamination sentinel tripped (CSE-12)", + "model alias or CLI drift outside the frozen identity (CSE-6)", + "integrity verdict authored session-side (CSE-11)" + ] + }, + "hypotheses": [ + { + "direction": "typed_dataflow - monolithic > 0", + "id": "H-CSE-1", + "statement": "The bundled typed-dataflow surface raises the probability that a fresh coding-agent session produces a launching system passing the hidden held-out acceptance within budget, versus the equal-capability monolithic surface." + } + ], + "instrument_set": { + "agent_systems": [ + { + "binary": "codex", + "binary_sha256": "b973d440acac501fd2594a43e7ca9ce41e0a65b9dfb28d0d7a7837c99e1261e3", + "frozen": true, + "id": "codex-cli", + "install": "private prefix ~/aisle-private/codex-0.153.4 (the qualified darwin-arm64 binary, not the JS shim); ChatGPT-login credentials copied byte-identically into both arms' private CODEX_HOME, hash recorded only", + "version_observed": "0.153.4" + } + ], + "candidate": { + "id": "t1-l2-realistic", + "monolithic_module": "experts/monolithic/expert_t1_l2.py", + "monolithic_template": "graphs/monolithic_t1_l2.yaml", + "perception": "L2", + "reset": "teleport", + "typed_graph": "graphs/expert_t1_l2.yaml", + "verifier": "realistic (oracle demoted to the held-out scoring tap, BND-3)" + }, + "candidate_table_sha256": "08a1dc56502abc4673cb0b54f01664382214bb473c3ef293b1df7b6c74f64778", + "items": [ + "short_composition:t1-l2-realistic" + ], + "kind": "selected_task" + }, + "integrity_checks": [ + { + "gate": "CON-14 spec approval", + "kind": "external_review", + "owner_role": "project owner", + "pilot_record_2026_09_13": "pending; external review, not a pilot prerequisite (ADR-66 section 2); named limitation", + "record": null, + "status": "pending" + }, + { + "gate": "STA-12 independent statistical review", + "kind": "external_review", + "owner_role": "independent statistician", + "pilot_record_2026_09_13": "pending; external review, not a pilot prerequisite (ADR-66 section 2); named limitation", + "record": null, + "status": "pending" + }, + { + "gate": "MON-1 treatment table and expert parity (#344)", + "kind": "machine_check", + "owner_role": "parity harness; control surface exists, independent authorship and operator evidence remain pending", + "pilot_record_2026_09_13": "pending; the T1-L2 pair (#583) passes the interface, table and module checks but has no operator parity record at an immutable hash; named limitation", + "record": null, + "status": "pending" + }, + { + "gate": "BND-12 task-band freeze (#346)", + "kind": "machine_check", + "owner_role": "task-band auditor", + "pilot_record_2026_09_13": "recorded failed: bnd-task-band-calibration-v11 regenerated the BND-5 perception audit on a 32-seed T1-L2 corpus; 22 of 22 strata below the 0.90 floor (BND-7), led by target_class=ibuprofen refusals; role open (BND-10). ADR-66 section 2: a failing machine gate is a named limitation of this pilot, not a blocker", + "record": "analysis/perception-audit/records/bnd-perception-corpus-03/report.json", + "status": "failed" + }, + { + "gate": "TRT confinement/mutation/agent-path (#353)", + "kind": "machine_check", + "owner_role": "integrity controller", + "pilot_record_2026_09_13": "pending; the session-bound capability audit (loopback relay policy) is not merged; an unattested environment is a stop condition for every session", + "record": null, + "status": "pending" + }, + { + "gate": "seed commitment verification", + "kind": "machine_check", + "owner_role": "holder of the original private seed and salt sources", + "pilot_record_2026_09_13": "not applicable to this registration: it commits a fresh private seed source directly (no inheritance); kept for shape parity and left pending", + "public_observation_successor_note": "Inherited from pilot-v8 without access to private seed sources; verification remains pending. This byte-binding successor does not authorize a pilot run.", + "record": null, + "status": "pending" + }, + { + "gate": "MON-8/MON-12 matched-session acceptance (#519)", + "kind": "machine_check", + "owner_role": "matched-session controller; complete acceptance evidence still required", + "pilot_record_2026_09_13": "pending; frontend conformance qualified on the T1 oracle development surface only (#568); the T1-L2 realistic surface runs under the same executor with the same admission checks; named limitation", + "record": null, + "status": "pending" + } + ], + "issue": 347, + "pilot_design": { + "agent_system": "codex-cli 0.153.4 only (the owned provider relay, the ChatGPT-login account header and the pinned conformance profile are Codex-bound)", + "arm_order": "sealed balanced plan (revealed one assignment at a time); no longer a fixed alternation", + "arms": [ + "typed", + "monolithic" + ], + "assignment": "tools/pilot_assignments.py create --protocol analysis/freeze/cse-causal-study-pilot-v8/protocol.json (sealed balanced plan, one typed and one monolithic assignment per temporal block, TRT-8/CSE-7); reveal exactly one assignment before each session; the ledger retains every assignment, started or not", + "below_cse8_floor": "6 per arm is below the CSE-8 floor of 10 by design; this registration can never become a confirmatory estimate (STA-3)", + "decision": "docs/decisions/ADR-66.md (pilot first; agent-drafted under the ADR-65 label, renumbered by #568)", + "executor": "tools/matched_campaign.py admit / run --purpose engineering / collect-run, then the held-out evaluation; retained and unscored, output root ~/aisle-private/raw/cse-causal-study-pilot-v8/", + "records": "tools/pilot_records.py derives one aisle.stats.records.v1 row per ledger assignment; harness stats validate --purpose analysis; harness stats analyze; harness stats power from observed control success (CSE-8)", + "reporting": "analysis/reports/cse_pilot_v8.md; results enter docs/claim-evidence.yaml only as CSE-8 power inputs under the unrun confirmatory claim", + "rerun_policy": "no reruns; a failed attempt is retained under a new session id, never overwritten", + "session_success": "controller-run held-out evaluation of the retained final snapshot on the 32 private seeds after the agent process ends, oracle as the held-out scorer (BND-3); success iff at least 28 of 32 oracle successes; a launch failure counts as a failed seed (CSE-2, CSE-9)", + "sessions_per_arm": 6, + "stop_conditions": [ + "wrong_object outcome", + "guard bypass", + "unattested environment", + "relay bypass (provider traffic outside the owned relay)", + "any oracle field on a policy input" + ], + "task_stratum": "short_composition on candidate t1-l2-realistic (T1 franka, L2 perception, realistic verifier); bnd-task-band-calibration-v11 records it as not perception-eligible (BND-7), carried as a machine-gate limitation per ADR-66" + }, + "purpose": "pilot", + "schema_version": "aisle.freeze.declaration.v1", + "seed_commitment": { + "inherited_from": "analysis/freeze/cse-causal-study-pilot-v12/freeze-manifest.json", + "rule": "32 distinct integers in [1000,9999) drawn once from the OS CSPRNG on 2026-09-13, retained only under ~/aisle-private/freeze/cse-causal-study-pilot-v8/, disjoint by value from every private seed list committed so far (including cse-causal-study-pilot-v1 and the cse-causal-study-v1 confirmatory commitment) and from the committed development seed lists; never inherited by a confirmatory declaration (ADR-66); revealed after the pilot closes. The same directory holds the controller-private 256-bit randomization seed for the sealed assignment plan (TRT-8); its commitment is recorded in the assignment ledger, never here.", + "salt_source": "~/aisle-private/freeze/cse-causal-study-pilot-v8/salt.bin", + "seeds_source": "~/aisle-private/freeze/cse-causal-study-pilot-v8/seeds.json" + }, + "spec": "500", + "superseded": "cse-causal-study-pilot-v12 (the matched source set gained the ADR-67/ADR-68 engine seam: src/aisle/harness/cli.py, rollout.py and src/aisle/nodes/dora_genesis.py now select the physics engine behind AISLE_SIM_ENGINE. The protocol, seed lineage, decision rules and pending gates are unchanged, and Genesis remains the default and the only engine behind this campaign)" + }, + "declaration_sha256": "sha256:3369f7ad437d1fa7f7015a1516685a839def97898e464c6feebf2c131fe42bf0", + "frozen": false, + "gate_record_hashes": {}, + "git_head": "cabf2eb5d3be7fc960d4737cd64e449e26f5997d", + "issue": 347, + "pending_gates": [ + "BND-12 task-band freeze (#346)", + "CON-14 spec approval", + "MON-1 treatment table and expert parity (#344)", + "MON-8/MON-12 matched-session acceptance (#519)", + "STA-12 independent statistical review", + "TRT confinement/mutation/agent-path (#353)", + "seed commitment verification" + ], + "purpose": "pilot", + "schema_version": "aisle.freeze.manifest.v1", + "seed_commitment": "sha256:bc4a2c141eded05f77bee114330c2f9d1c5112e26618f2feb9694cf60dd56143", + "spec": "500", + "status": "registered_pending_review", + "timestamp": null, + "timestamp_source": null +} diff --git a/analysis/freeze/cse-causal-study-v27/declaration.json b/analysis/freeze/cse-causal-study-v27/declaration.json new file mode 100644 index 00000000..fdfb8a39 --- /dev/null +++ b/analysis/freeze/cse-causal-study-v27/declaration.json @@ -0,0 +1,360 @@ +{ + "analysis": { + "scripts": [ + "src/aisle/harness/benchmark_statistics.py", + "src/aisle/harness/causal_study.py", + "src/aisle/harness/pilot_records.py" + ], + "seed": 500001 + }, + "artifacts": { + "agent_adapters": "tools/agent_adapters.py", + "analyzer": "src/aisle/harness/benchmark_statistics.py", + "budget_ceiling": "harness/budget.toml", + "dependency_declaration": "pyproject.toml", + "dependency_lock": "uv.lock", + "dora_runtime_selection": "dora-runtime.json", + "frontend_conformance_decision": "docs/decisions/ADR-65.md", + "frontend_request_authorization_documentation": "docs/benchmark/v1/frontend-request-authorization.md", + "matched_session_decision": "docs/decisions/ADR-62.md", + "matched_session_documentation": "docs/monolithic/matched-session.md", + "matched_source:assets/so101/so101.urdf": "assets/so101/so101.urdf", + "matched_source:docs/monolithic/allowlist-t1-l2.json": "docs/monolithic/allowlist-t1-l2.json", + "matched_source:docs/monolithic/candidates.json": "docs/monolithic/candidates.json", + "matched_source:docs/monolithic/experts-t1-l2.json": "docs/monolithic/experts-t1-l2.json", + "matched_source:docs/monolithic/interface-map-t1-l2.json": "docs/monolithic/interface-map-t1-l2.json", + "matched_source:docs/monolithic/parity-protocol-t1-l2.json": "docs/monolithic/parity-protocol-t1-l2.json", + "matched_source:docs/monolithic/treatment-table-t1-l2.json": "docs/monolithic/treatment-table-t1-l2.json", + "matched_source:env/limits.toml": "env/limits.toml", + "matched_source:experts/monolithic/expert_t1_l2.py": "experts/monolithic/expert_t1_l2.py", + "matched_source:graphs/pilot_t1_l2_monolithic.yaml": "graphs/pilot_t1_l2_monolithic.yaml", + "matched_source:src/aisle/__init__.py": "src/aisle/__init__.py", + "matched_source:src/aisle/embodiment.py": "src/aisle/embodiment.py", + "matched_source:src/aisle/harness/_code_mode_protocol/__init__.py": "src/aisle/harness/_code_mode_protocol/__init__.py", + "matched_source:src/aisle/harness/_code_mode_protocol/descriptor.pb": "src/aisle/harness/_code_mode_protocol/descriptor.pb", + "matched_source:src/aisle/harness/_code_mode_protocol/protocol.proto": "src/aisle/harness/_code_mode_protocol/protocol.proto", + "matched_source:src/aisle/harness/candidate_launch.py": "src/aisle/harness/candidate_launch.py", + "matched_source:src/aisle/harness/candidates.py": "src/aisle/harness/candidates.py", + "matched_source:src/aisle/harness/cli.py": "src/aisle/harness/cli.py", + "matched_source:src/aisle/harness/code_mode_audit.py": "src/aisle/harness/code_mode_audit.py", + "matched_source:src/aisle/harness/code_mode_authority.py": "src/aisle/harness/code_mode_authority.py", + "matched_source:src/aisle/harness/code_mode_proxy.py": "src/aisle/harness/code_mode_proxy.py", + "matched_source:src/aisle/harness/code_mode_runner.py": "src/aisle/harness/code_mode_runner.py", + "matched_source:src/aisle/harness/common.py": "src/aisle/harness/common.py", + "matched_source:src/aisle/harness/frontend_app_server.py": "src/aisle/harness/frontend_app_server.py", + "matched_source:src/aisle/harness/frontend_app_server_audit.py": "src/aisle/harness/frontend_app_server_audit.py", + "matched_source:src/aisle/harness/frontend_conformance.py": "src/aisle/harness/frontend_conformance.py", + "matched_source:src/aisle/harness/frontend_dispatch.py": "src/aisle/harness/frontend_dispatch.py", + "matched_source:src/aisle/harness/frontend_dispatch_audit.py": "src/aisle/harness/frontend_dispatch_audit.py", + "matched_source:src/aisle/harness/frontend_effects.py": "src/aisle/harness/frontend_effects.py", + "matched_source:src/aisle/harness/frontend_qualification.py": "src/aisle/harness/frontend_qualification.py", + "matched_source:src/aisle/harness/frontend_request_audit.py": "src/aisle/harness/frontend_request_audit.py", + "matched_source:src/aisle/harness/frontend_request_authority.py": "src/aisle/harness/frontend_request_authority.py", + "matched_source:src/aisle/harness/guard_divergence.py": "src/aisle/harness/guard_divergence.py", + "matched_source:src/aisle/harness/heldout_eval.py": "src/aisle/harness/heldout_eval.py", + "matched_source:src/aisle/harness/hidden_access_log.py": "src/aisle/harness/hidden_access_log.py", + "matched_source:src/aisle/harness/matched_app_server.py": "src/aisle/harness/matched_app_server.py", + "matched_source:src/aisle/harness/matched_collection.py": "src/aisle/harness/matched_collection.py", + "matched_source:src/aisle/harness/matched_dynamic_run.py": "src/aisle/harness/matched_dynamic_run.py", + "matched_source:src/aisle/harness/matched_evidence.py": "src/aisle/harness/matched_evidence.py", + "matched_source:src/aisle/harness/matched_frontend.py": "src/aisle/harness/matched_frontend.py", + "matched_source:src/aisle/harness/matched_run.py": "src/aisle/harness/matched_run.py", + "matched_source:src/aisle/harness/matched_run_launch.py": "src/aisle/harness/matched_run_launch.py", + "matched_source:src/aisle/harness/matched_runtime.py": "src/aisle/harness/matched_runtime.py", + "matched_source:src/aisle/harness/matched_session.py": "src/aisle/harness/matched_session.py", + "matched_source:src/aisle/harness/matched_tool_service.py": "src/aisle/harness/matched_tool_service.py", + "matched_source:src/aisle/harness/matched_tools.py": "src/aisle/harness/matched_tools.py", + "matched_source:src/aisle/harness/mcp_harness_authority.py": "src/aisle/harness/mcp_harness_authority.py", + "matched_source:src/aisle/harness/mcp_harness_source.py": "src/aisle/harness/mcp_harness_source.py", + "matched_source:src/aisle/harness/mcp_harness_transport.py": "src/aisle/harness/mcp_harness_transport.py", + "matched_source:src/aisle/harness/monolith.py": "src/aisle/harness/monolith.py", + "matched_source:src/aisle/harness/monolithic.py": "src/aisle/harness/monolithic.py", + "matched_source:src/aisle/harness/monolithic_run_evidence.py": "src/aisle/harness/monolithic_run_evidence.py", + "matched_source:src/aisle/harness/monolithic_run_prepare.py": "src/aisle/harness/monolithic_run_prepare.py", + "matched_source:src/aisle/harness/pilot_records.py": "src/aisle/harness/pilot_records.py", + "matched_source:src/aisle/harness/provider_hosted.py": "src/aisle/harness/provider_hosted.py", + "matched_source:src/aisle/harness/provider_relay.py": "src/aisle/harness/provider_relay.py", + "matched_source:src/aisle/harness/provider_response_authority.py": "src/aisle/harness/provider_response_authority.py", + "matched_source:src/aisle/harness/provider_runner.py": "src/aisle/harness/provider_runner.py", + "matched_source:src/aisle/harness/provider_source_audit.py": "src/aisle/harness/provider_source_audit.py", + "matched_source:src/aisle/harness/registry.py": "src/aisle/harness/registry.py", + "matched_source:src/aisle/harness/rollout.py": "src/aisle/harness/rollout.py", + "matched_source:src/aisle/harness/rollout_client.py": "src/aisle/harness/rollout_client.py", + "matched_source:src/aisle/harness/simulator_work.py": "src/aisle/harness/simulator_work.py", + "matched_source:src/aisle/harness/traces.py": "src/aisle/harness/traces.py", + "matched_source:src/aisle/harness/treatment_ambient.py": "src/aisle/harness/treatment_ambient.py", + "matched_source:src/aisle/harness/treatment_confinement.py": "src/aisle/harness/treatment_confinement.py", + "matched_source:src/aisle/harness/treatment_integrity.py": "src/aisle/harness/treatment_integrity.py", + "matched_source:src/aisle/harness/treatment_postflight.py": "src/aisle/harness/treatment_postflight.py", + "matched_source:src/aisle/harness/typed_execution_bundle.py": "src/aisle/harness/typed_execution_bundle.py", + "matched_source:src/aisle/harness/typed_graph_audit.py": "src/aisle/harness/typed_graph_audit.py", + "matched_source:src/aisle/harness/typed_graph_hosts.py": "src/aisle/harness/typed_graph_hosts.py", + "matched_source:src/aisle/harness/typed_graph_stage.py": "src/aisle/harness/typed_graph_stage.py", + "matched_source:src/aisle/harness/typed_node_host.py": "src/aisle/harness/typed_node_host.py", + "matched_source:src/aisle/harness/typed_node_requests.py": "src/aisle/harness/typed_node_requests.py", + "matched_source:src/aisle/harness/typed_node_supervisor.py": "src/aisle/harness/typed_node_supervisor.py", + "matched_source:src/aisle/harness/typed_node_worker.py": "src/aisle/harness/typed_node_worker.py", + "matched_source:src/aisle/harness/typed_run_prepare.py": "src/aisle/harness/typed_run_prepare.py", + "matched_source:src/aisle/harness/typed_snapshot.py": "src/aisle/harness/typed_snapshot.py", + "matched_source:src/aisle/harness/typed_stage_provider.py": "src/aisle/harness/typed_stage_provider.py", + "matched_source:src/aisle/harness/typed_validation.py": "src/aisle/harness/typed_validation.py", + "matched_source:src/aisle/harness/typed_worker_launch.py": "src/aisle/harness/typed_worker_launch.py", + "matched_source:src/aisle/harness/typed_worker_provisioning.py": "src/aisle/harness/typed_worker_provisioning.py", + "matched_source:src/aisle/harness/validate.py": "src/aisle/harness/validate.py", + "matched_source:src/aisle/harness/worker_authority_probe.py": "src/aisle/harness/worker_authority_probe.py", + "matched_source:src/aisle/harness/worker_capability.py": "src/aisle/harness/worker_capability.py", + "matched_source:src/aisle/harness/worker_declaration.py": "src/aisle/harness/worker_declaration.py", + "matched_source:src/aisle/harness/worker_network_probe.py": "src/aisle/harness/worker_network_probe.py", + "matched_source:src/aisle/kinematics.py": "src/aisle/kinematics.py", + "matched_source:src/aisle/mobility/guard.py": "src/aisle/mobility/guard.py", + "matched_source:src/aisle/monolith/confinement.py": "src/aisle/monolith/confinement.py", + "matched_source:src/aisle/monolith/primitive_api.py": "src/aisle/monolith/primitive_api.py", + "matched_source:src/aisle/monolith/primitives.py": "src/aisle/monolith/primitives.py", + "matched_source:src/aisle/monolith/proxy.py": "src/aisle/monolith/proxy.py", + "matched_source:src/aisle/monolith/requests.py": "src/aisle/monolith/requests.py", + "matched_source:src/aisle/monolith/supervisor.py": "src/aisle/monolith/supervisor.py", + "matched_source:src/aisle/monolith/wire.py": "src/aisle/monolith/wire.py", + "matched_source:src/aisle/monolith/worker.py": "src/aisle/monolith/worker.py", + "matched_source:src/aisle/monolith/worker_config.py": "src/aisle/monolith/worker_config.py", + "matched_source:src/aisle/monolith/worker_launch.py": "src/aisle/monolith/worker_launch.py", + "matched_source:src/aisle/nodes/budget_guard.py": "src/aisle/nodes/budget_guard.py", + "matched_source:src/aisle/nodes/h6_fault.py": "src/aisle/nodes/h6_fault.py", + "matched_source:src/aisle/nodes/l2_pose.py": "src/aisle/nodes/l2_pose.py", + "matched_source:src/aisle/nodes/label_reader.py": "src/aisle/nodes/label_reader.py", + "matched_source:src/aisle/nodes/monolith_broker.py": "src/aisle/nodes/monolith_broker.py", + "matched_source:src/aisle/nodes/perception_session.py": "src/aisle/nodes/perception_session.py", + "matched_source:src/aisle/scenes/locations.toml": "src/aisle/scenes/locations.toml", + "matched_source:src/aisle/scenes/meds.toml": "src/aisle/scenes/meds.toml", + "matched_source:src/aisle/scenes/pharmacy.py": "src/aisle/scenes/pharmacy.py", + "matched_source:src/aisle/scenes/physics.toml": "src/aisle/scenes/physics.toml", + "matched_source:src/aisle/scenes/planogram.toml": "src/aisle/scenes/planogram.toml", + "matched_source:src/aisle/scenes/store.py": "src/aisle/scenes/store.py", + "matched_source:src/aisle/topics.py": "src/aisle/topics.py", + "matched_source:src/aisle/turn_node.py": "src/aisle/turn_node.py", + "matched_source:src/aisle/turns.py": "src/aisle/turns.py", + "matched_source:src/aisle/verifier/calibration.py": "src/aisle/verifier/calibration.py", + "matched_source:src/aisle/verifier/models.lock": "src/aisle/verifier/models.lock", + "matched_source:src/aisle/verifier/models.py": "src/aisle/verifier/models.py", + "matched_source:src/aisle/verifier/placement.toml": "src/aisle/verifier/placement.toml", + "matched_source:src/aisle/verifier/realistic.py": "src/aisle/verifier/realistic.py", + "matched_source:src/aisle/verifier/stages.py": "src/aisle/verifier/stages.py", + "matched_source:src/aisle/verifier/thresholds.toml": "src/aisle/verifier/thresholds.toml", + "matched_source:tools/campaign.py": "tools/campaign.py", + "matched_source:tools/matched_campaign.py": "tools/matched_campaign.py", + "monolithic_validators": "src/aisle/harness/monolithic.py", + "nested_host_documentation": "docs/monolithic/code-mode-admission.md", + "pilot_assignments_cli": "tools/pilot_assignments.py", + "pilot_records_cli": "tools/pilot_records.py", + "protocol": "analysis/freeze/cse-causal-study-v1/protocol.json", + "protocol_template": "analysis/statistics/protocol-template.json", + "provider_admission_documentation": "docs/monolithic/provider-admission.md", + "public_observation_source:analysis/monolithic-control/experts-t1-l2-v1.json": "analysis/monolithic-control/experts-t1-l2-v1.json", + "public_observation_source:analysis/monolithic-control/treatment-table-t1-l2-v1.json": "analysis/monolithic-control/treatment-table-t1-l2-v1.json", + "public_observation_source:docs/decisions/ADR-pilot-public-observations.md": "docs/decisions/ADR-pilot-public-observations.md", + "public_observation_source:docs/monolithic/public-observations.md": "docs/monolithic/public-observations.md", + "public_observation_source:graphs/pilot_t1_l2_monolithic.yaml": "graphs/pilot_t1_l2_monolithic.yaml", + "public_observation_source:graphs/pilot_t1_l2_typed.yaml": "graphs/pilot_t1_l2_typed.yaml", + "public_observation_source:graphs/turn_plans/pilot_t1_l2_monolithic.json": "graphs/turn_plans/pilot_t1_l2_monolithic.json", + "public_observation_source:graphs/turn_plans/pilot_t1_l2_typed.json": "graphs/turn_plans/pilot_t1_l2_typed.json", + "public_observation_source:registry/manifests/pilot-policy-surface.yaml": "registry/manifests/pilot-policy-surface.yaml", + "public_observation_source:src/aisle/harness/pilot_model_cache.py": "src/aisle/harness/pilot_model_cache.py", + "public_observation_source:src/aisle/harness/pilot_policy_surface.py": "src/aisle/harness/pilot_policy_surface.py", + "research_prompt": "harness/CLAUDE.research.md", + "seed_commitment_predecessor": "analysis/freeze/cse-causal-study-v26/freeze-manifest.json", + "session_runner": "tools/campaign.py", + "session_validators": "src/aisle/harness/causal_study.py", + "simulator_bridge": "src/aisle/nodes/dora_genesis.py", + "spec": "specs/500-typed-monolithic-causal-study.md", + "treatment_integrity": "src/aisle/harness/treatment_integrity.py", + "treatment_postflight": "src/aisle/harness/treatment_postflight.py", + "treatment_table": "analysis/monolithic-control/treatment-table-v1.json" + }, + "budgets": { + "campaign_ceiling": "harness/budget.toml", + "rollouts_per_session": 20, + "session_tokens": 200000, + "session_wall_s": 3600, + "sim_episodes_per_session": 160 + }, + "campaign_id": "cse-causal-study-v27", + "commands": [ + "uv run harness stats validate --protocol analysis/freeze/cse-causal-study-v1/protocol.json --purpose power", + "uv run harness stats validate --protocol analysis/freeze/cse-causal-study-v1/protocol.json --purpose freeze --output /tmp/cse-causal-study-v22-freeze-refusal.json", + "uv run harness freeze check --manifest analysis/freeze/cse-causal-study-v26/freeze-manifest.json --allow-withheld-seeds" + ], + "decision_rules": { + "alpha": 0.05, + "equivalence_margin": { + "cost_at_equal_quality": { + "acceptance_probability": 0.1, + "heldout_quality": 0.1, + "rule": "CSE-5: both must pass or cost claims stay unconditional", + "sidedness": "two-sided 0.90" + } + }, + "multiplicity": "single primary endpoint", + "power": 0.8, + "primary_rule": "one-sided superiority at alpha 0.05 on the pooled Newcombe interval lower bound > 0 with no reversed stratum; otherwise weakened/rejected/undecidable by the #358 rule (CSE-3)", + "smallest_effect": { + "measure": "risk difference in session success", + "value": 0.25 + }, + "stopping_rule": "fixed n from the frozen power analysis using pilot-only control success; absolute floor 10 randomized assignments per arm per agent-system x task stratum (CSE-8); no optional stopping or extension" + }, + "endpoints": { + "primary": [ + { + "aggregation": "risk difference typed_dataflow - monolithic; exact arm intervals; Newcombe interval; per agent-system x task stratum before pooling (CSE-1, CSE-17)", + "direction": "higher", + "id": "E-CSE-1", + "outcome": "session_success", + "status": "inferential", + "unit": "agent_session" + } + ], + "secondary": [ + { + "aggregation": "counts by arm", + "direction": "descriptive", + "id": "E-CSE-2", + "outcome": "validation_and_launch_failure_class", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "Kaplan-Meier with censoring", + "direction": "lower", + "id": "E-CSE-3", + "outcome": "time_to_first_accepted_system", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "unconditional session estimands; no at-equal-quality wording unless CSE-5 equivalence passes", + "direction": "lower", + "id": "E-CSE-4", + "outcome": "tokens_tool_calls_wall_active_time_sim_work_rollouts", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "counts", + "direction": "descriptive", + "id": "E-CSE-5", + "outcome": "safety_interventions", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "dropped unless a blinded rubric is ratified before freeze (CSE-18)", + "direction": "n/a", + "id": "E-CSE-6", + "outcome": "artifact_legibility", + "status": "exploratory", + "unit": "agent_session" + } + ] + }, + "exclusions": { + "deviation_policy": "any behaviour-affecting change after registration is a documented deviation record and a new campaign version id; confirmatory data never revises the registered protocol", + "infrastructure": [ + "independent-controller-determined, arm-neutral facts only (CSE-14): host outage, provider interruption before first tool call, launcher crash before the session started" + ], + "rerun_policy": "a permitted rerun receives a new randomized id; the original row is retained with its exclusion reason and never replaced in place", + "treatment_integrity": [ + "contamination sentinel tripped (CSE-12)", + "model alias or CLI drift outside the frozen identity (CSE-6)", + "integrity verdict authored session-side (CSE-11)" + ] + }, + "hypotheses": [ + { + "direction": "typed_dataflow - monolithic > 0", + "id": "H-CSE-1", + "statement": "The bundled typed-dataflow surface raises the probability that a fresh coding-agent session produces a launching system passing the hidden held-out acceptance within budget, versus the equal-capability monolithic surface." + } + ], + "instrument_set": { + "agent_systems": [ + { + "binary": "claude", + "frozen": false, + "id": "claude-code", + "version_observed": "2.1.261" + }, + { + "binary": "codex", + "frozen": false, + "id": "codex-cli", + "version_observed": "0.151.0" + } + ], + "items": [ + "short_composition:TBD-by-bnd-task-band-calibration-v1", + "engineering:TBD-by-bnd-task-band-calibration-v1" + ], + "kind": "selected_task" + }, + "integrity_checks": [ + { + "gate": "CON-14 spec approval", + "kind": "external_review", + "owner_role": "project owner", + "record": null, + "status": "pending" + }, + { + "gate": "STA-12 independent statistical review", + "kind": "external_review", + "owner_role": "independent statistician", + "record": null, + "status": "pending" + }, + { + "gate": "MON-1 treatment table and expert parity (#344)", + "kind": "machine_check", + "owner_role": "parity harness; control surface exists, independent authorship and operator evidence remain pending", + "record": null, + "status": "pending" + }, + { + "gate": "BND-12 task-band freeze (#346)", + "kind": "machine_check", + "owner_role": "task-band auditor", + "record": null, + "status": "pending" + }, + { + "gate": "TRT confinement/mutation/agent-path (#353)", + "kind": "machine_check", + "owner_role": "integrity controller", + "record": null, + "status": "pending" + }, + { + "gate": "seed commitment verification", + "kind": "machine_check", + "owner_role": "holder of the original private seed and salt sources", + "record": null, + "status": "pending" + }, + { + "gate": "MON-8/MON-12 matched-session acceptance (#519)", + "kind": "machine_check", + "owner_role": "matched-session controller; complete acceptance evidence still required", + "record": null, + "status": "pending" + } + ], + "issue": 347, + "purpose": "confirmatory", + "schema_version": "aisle.freeze.declaration.v1", + "seed_commitment": { + "inherited_from": "analysis/freeze/cse-causal-study-v26/freeze-manifest.json", + "rule": "32 distinct integers in [1000,9999) drawn once from the OS CSPRNG, retained only under ~/aisle-private/freeze/cse-causal-study-v1/, disjoint by value from every committed dev/pilot seed list; revealed after the campaign closes", + "salt_source": "~/aisle-private/freeze/cse-causal-study-v1/salt.bin", + "seeds_source": "~/aisle-private/freeze/cse-causal-study-v1/seeds.json" + }, + "spec": "500", + "superseded": "cse-causal-study-v26 (the matched source set gained the ADR-67/ADR-68 engine seam: src/aisle/harness/cli.py, rollout.py and src/aisle/nodes/dora_genesis.py now select the physics engine behind AISLE_SIM_ENGINE. The protocol, seed lineage, decision rules and pending gates are unchanged, and Genesis remains the default and the only engine behind this campaign)" +} diff --git a/analysis/freeze/cse-causal-study-v27/freeze-manifest.json b/analysis/freeze/cse-causal-study-v27/freeze-manifest.json new file mode 100644 index 00000000..c208787b --- /dev/null +++ b/analysis/freeze/cse-causal-study-v27/freeze-manifest.json @@ -0,0 +1,557 @@ +{ + "analysis_script_hashes": { + "src/aisle/harness/benchmark_statistics.py": "sha256:22cde3e9d7c5cc80874c7eec30aa2898e30476d6c939c94ab3009e8b12af641e", + "src/aisle/harness/causal_study.py": "sha256:1b1c53d6389881c146726d865a741a681f3d5039c9bf99558a53ee368250971e", + "src/aisle/harness/pilot_records.py": "sha256:68ac7ef9afd449d8527e2ffe99a9ef926e06253c466f3421bc1848e9aa094b70" + }, + "analysis_seed": 500001, + "artifact_hashes": { + "agent_adapters": "sha256:848f8f7da5ad978e53712991a688eb74f56c7a77deabee8afae717ad712666ca", + "analyzer": "sha256:22cde3e9d7c5cc80874c7eec30aa2898e30476d6c939c94ab3009e8b12af641e", + "budget_ceiling": "sha256:bcbb79aa146afed1d877da18093175ae144b4607db50119022ef4f8173122cda", + "dependency_declaration": "sha256:add4e718001da7678b404bdf37ea8d9889f563324ffba9b03fe65c46b172ee4c", + "dependency_lock": "sha256:9b8a82a17ba68c192f045405feed3ae7046af9568bddcb398292c20f0ed6f5a8", + "dora_runtime_selection": "sha256:f9eb7ffe285a0f0ed26d50303e31cf1e94d2f93ed62b599071dff5a183c15f53", + "frontend_conformance_decision": "sha256:5059582dc735ddb85f329c647070b42bcc689b2836e198d305f381e74f89edc2", + "frontend_request_authorization_documentation": "sha256:a1d4e65c0e467af4bd77922b95b12f26ee6478932eb5dd460de34b81249ca86b", + "matched_session_decision": "sha256:d88e0ef18c92cedff7b74dd3ffafed14d92affaec65beb9ddfec2801b365dc61", + "matched_session_documentation": "sha256:ca006514d55b2c1dc1d046a17e59ebcad52c57b006460e9aa05960e02d527d05", + "matched_source:assets/so101/so101.urdf": "sha256:3a65d2d35e68a8d2f0c2cc176d19b884506543c93ba72980145b80abe276022c", + "matched_source:docs/monolithic/allowlist-t1-l2.json": "sha256:6aac85db63a7061ab09950741eba8ac0e77410755d85c427731c45864adc1dc5", + "matched_source:docs/monolithic/candidates.json": "sha256:2fc51e6ff0e4b73a9b203d6ed1debf1cfa40ece9b21a35245d50b9a403f80abb", + "matched_source:docs/monolithic/experts-t1-l2.json": "sha256:68f01e1081afb36964f591fe0e58a630c8e734c8809833279ac608b1c554fe1a", + "matched_source:docs/monolithic/interface-map-t1-l2.json": "sha256:31fede313d1573e29a83c42acc92883ae8ffeb36beee0a62a8d4481ad68cd591", + "matched_source:docs/monolithic/parity-protocol-t1-l2.json": "sha256:2bfc9854fb5cffbe70be341799f208813eee8c02c9d54993b83ce86f6b80f054", + "matched_source:docs/monolithic/treatment-table-t1-l2.json": "sha256:544bd956688b8d85b54ff496dc9d98bdc42676c1c0c5fe5583e276614d4ed0d5", + "matched_source:env/limits.toml": "sha256:1ab1f4fd5d52e7949b81d21933daaa2c66e80ef07fdedf8217280468e0142edc", + "matched_source:experts/monolithic/expert_t1_l2.py": "sha256:93ade82db523fe808312f9d83c88ab3464eee303be1bfc3f824880bf1de7924c", + "matched_source:graphs/pilot_t1_l2_monolithic.yaml": "sha256:ebf12538d0b9def23ce4ea70a9feb375c98535f774bbf16d4d839aaa4e99d27c", + "matched_source:src/aisle/__init__.py": "sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "matched_source:src/aisle/embodiment.py": "sha256:d17c204492d680175cbaf365aaf164e2a6049da23ce125a9b9f21606b79f4cca", + "matched_source:src/aisle/harness/_code_mode_protocol/__init__.py": "sha256:7e7fb09527b647c2bb13787011de8f9a4412e8b6f304e89efa107162d8150e82", + "matched_source:src/aisle/harness/_code_mode_protocol/descriptor.pb": "sha256:459465e666c6a76a70e902c10c58905dfd8417b6c2ca3e038be99624e3542540", + "matched_source:src/aisle/harness/_code_mode_protocol/protocol.proto": "sha256:406d8f47d8d0b95a9d4ae104890e5e52764c9597e9271f75ff434a4ce24748d0", + "matched_source:src/aisle/harness/candidate_launch.py": "sha256:654eb90264703e3ce3ab599bb8e143047eaee288c8852c5b09c74e3147039270", + "matched_source:src/aisle/harness/candidates.py": "sha256:d1b441a30288a6c0500aaa765a24d4aeecca4f8d52e96f633befc50c84ecf5f5", + "matched_source:src/aisle/harness/cli.py": "sha256:a5488dc0350278b2618be9e25043bfcef5ae69610d2de750dffd01c9afa98b44", + "matched_source:src/aisle/harness/code_mode_audit.py": "sha256:24638163f205408da1c6cf20a1b77c80a0d740a3f8ed44696eb90baf7de61f6a", + "matched_source:src/aisle/harness/code_mode_authority.py": "sha256:217ea025f10a4dc02b4b0ff2ff733adf4a695d1098b3c8ce0d2b27b8a140853d", + "matched_source:src/aisle/harness/code_mode_proxy.py": "sha256:333eee561d610489d2f48f9359ad94a3d9d8ff729fbae0fd3dec4acf64be2cb2", + "matched_source:src/aisle/harness/code_mode_runner.py": "sha256:4f91577dcfc6c239bf4111bd6735a64e97d6e22b7ca6287389abae09023cc25d", + "matched_source:src/aisle/harness/common.py": "sha256:ef62bf518a99c8c4154908bc9cf99e68402e6ae6c8e3fb5ec30c3f5fce9b5fac", + "matched_source:src/aisle/harness/frontend_app_server.py": "sha256:d21b9bf218efc3066399b8b22bbe3892e772433da315a9ecd40e2773369e9283", + "matched_source:src/aisle/harness/frontend_app_server_audit.py": "sha256:fb3cf5b8a115eb93e3f292dae694a4318cbe43e9552347fbc25ea8e43102ddbb", + "matched_source:src/aisle/harness/frontend_conformance.py": "sha256:54f13041eaa2eb8679b33a7c9e441e300d93818901cc8819503425cd025aa02f", + "matched_source:src/aisle/harness/frontend_dispatch.py": "sha256:6bff2abbacca6c332abdb5c5cb4cb3d688d8b00535e3f428633619bf91cd30c8", + "matched_source:src/aisle/harness/frontend_dispatch_audit.py": "sha256:dc75024bd52c21c89a6112da847875a1d0ebf44f53a3704705e0bb8c81b9bc4a", + "matched_source:src/aisle/harness/frontend_effects.py": "sha256:bc539cecad349d27f68b1b2d6e435142c1119310b2129fb5a7c5854aa899d80e", + "matched_source:src/aisle/harness/frontend_qualification.py": "sha256:6c416f992cbc1418fb6fad52c9d0ab6bc3ff2e6fefd218f1bcf62b8500131d45", + "matched_source:src/aisle/harness/frontend_request_audit.py": "sha256:a4bc60b2ce1968ada1776324b72ad87025b93d366046029445fdf9b2e92f8830", + "matched_source:src/aisle/harness/frontend_request_authority.py": "sha256:0a061f1528b8bb2071dd1ba7e1d0bbee178b2c982bd6b688b916fb612522e163", + "matched_source:src/aisle/harness/guard_divergence.py": "sha256:af2c4755cabe46fabba913aee4819e05c10caaae79105615d50f8cf4567dad14", + "matched_source:src/aisle/harness/heldout_eval.py": "sha256:56c83164b04ae1b4623e8fd7a0579dd0a9caa7d5dbb891448643464a07893962", + "matched_source:src/aisle/harness/hidden_access_log.py": "sha256:27eabce8642b296afeae87a86b7ef44fa47ff755d479e9148af60a167ba5735c", + "matched_source:src/aisle/harness/matched_app_server.py": "sha256:b7177caca8f1e4d3664b6cef76539b572c2ea6604fe0799c629bc50a52951b52", + "matched_source:src/aisle/harness/matched_collection.py": "sha256:b347127344a4c15fbb73fa9f3ac7c711a3d55b0fa368a11f76683bb1b5bbea14", + "matched_source:src/aisle/harness/matched_dynamic_run.py": "sha256:b7f2b81397bcd232710408b9c6c9f463b3928e37f117d7310d519728f9b4b2c9", + "matched_source:src/aisle/harness/matched_evidence.py": "sha256:6d65476a9a923fd445ea9dad0334e521378353454b873ef59e2b0ff848c650f7", + "matched_source:src/aisle/harness/matched_frontend.py": "sha256:a4b9827af7dbf36f4af5228ca8c87ea36d2d32f8cffc7fcb8fea1923bcc09401", + "matched_source:src/aisle/harness/matched_run.py": "sha256:9ea045eb0abf99661b5bc4e1726f428c83b99ba06f1f2b5441e0daa3cc8ceb56", + "matched_source:src/aisle/harness/matched_run_launch.py": "sha256:8701f034d0bd9873aa7981e3db47849d50bd4465f085e617e6e3193aac070539", + "matched_source:src/aisle/harness/matched_runtime.py": "sha256:c4d81e8309fcf5212315013dbf8e83347a65080e1221f37a0491757834ac2797", + "matched_source:src/aisle/harness/matched_session.py": "sha256:ac784258cff693adee829fef4eaf9a8970d785d347738cafc96edbf06b853754", + "matched_source:src/aisle/harness/matched_tool_service.py": "sha256:1bf6685addbeee541961e1053b2eea68cba66ce110667d8e7720398d49191a17", + "matched_source:src/aisle/harness/matched_tools.py": "sha256:414592a1e1a346716a25b06012d82739253f6440aae4c85979ace3312efa9ccd", + "matched_source:src/aisle/harness/mcp_harness_authority.py": "sha256:f5760ef7a4e05a1444ee6ae5da3d9c37d0859cf767e2d96862b4d21803d646eb", + "matched_source:src/aisle/harness/mcp_harness_source.py": "sha256:f5a57dd19fc1c2966a8988ff925bf744384a4bf1368760427465340c9167990e", + "matched_source:src/aisle/harness/mcp_harness_transport.py": "sha256:a0c713cd82628109cd7324ff3deeaa33a66d37819a7c517b273a32e5e82cdaf3", + "matched_source:src/aisle/harness/monolith.py": "sha256:d898a3734adef6a72101bcec463fb82265107d1efa13f2121e3fd5f6a1494d43", + "matched_source:src/aisle/harness/monolithic.py": "sha256:1983ed8c26b2371c993c571f33f5c7e4ee81ea2597f9468b43d53c9970465ccf", + "matched_source:src/aisle/harness/monolithic_run_evidence.py": "sha256:4c0cb658f0148eb5d06ad69dceafd39148a2d4787e085cb624047c365c87bb24", + "matched_source:src/aisle/harness/monolithic_run_prepare.py": "sha256:14d80d436e04e3a1a07fe78716bec41c187002a9bac821ff6f11ede67443ac5a", + "matched_source:src/aisle/harness/pilot_records.py": "sha256:68ac7ef9afd449d8527e2ffe99a9ef926e06253c466f3421bc1848e9aa094b70", + "matched_source:src/aisle/harness/provider_hosted.py": "sha256:82a3c8dd10449a07970e39322f649d6680e55fb935ee0dbdc67381b136e81068", + "matched_source:src/aisle/harness/provider_relay.py": "sha256:0b1698ac82da51458da5d4cdee6ca9f79334703d8c4a9dbc297af173ec0ff9a2", + "matched_source:src/aisle/harness/provider_response_authority.py": "sha256:9562a8de28967e2eb83910ea5062dc491c8588a6bb4a2a04407f44d59f2251fe", + "matched_source:src/aisle/harness/provider_runner.py": "sha256:2c39d11d67cc4da2f9a2c732e86ba4e9a4b8da88570bdb1ecd1364d4668831cb", + "matched_source:src/aisle/harness/provider_source_audit.py": "sha256:e3be1f9d70288b4f596427a39ab4274399e9fdc74e87043e3c1249596fe1872b", + "matched_source:src/aisle/harness/registry.py": "sha256:214da7bb8db26c80c79262fa79c40a624d8a9981463d38a9c4aa30bc989bc1fb", + "matched_source:src/aisle/harness/rollout.py": "sha256:568f98bb6593f8f007408185956e7a97e7d575359c6c9f8b683bca37ca90c5f2", + "matched_source:src/aisle/harness/rollout_client.py": "sha256:18808db0aa2df356d1a7cc9a538585aa9dcc653abd120e761649f1dff2c27c31", + "matched_source:src/aisle/harness/simulator_work.py": "sha256:2ade87416ab8c5d8902e9734af850b7613ee9ee42f9ca672c9482625e15d3d91", + "matched_source:src/aisle/harness/traces.py": "sha256:c2e6a717a56ce47e296bdc0115b15de9361b72084638b4f3bc66f7e7d032546b", + "matched_source:src/aisle/harness/treatment_ambient.py": "sha256:13dce400a649bc0d01749661517138f8ad1f8cde106caceaedb9f33af6f6d856", + "matched_source:src/aisle/harness/treatment_confinement.py": "sha256:851339ca608e6649317f42a918eecd846e3aa339cba6aee472da03193aca334e", + "matched_source:src/aisle/harness/treatment_integrity.py": "sha256:b37162c6f295c5686da9b3089eb3bd96b69f68f01f41da0a784c4017658b10a6", + "matched_source:src/aisle/harness/treatment_postflight.py": "sha256:27b591c291491ff1e943a62ec29ecd054b6caa29547b7872844cdc67dc403e9c", + "matched_source:src/aisle/harness/typed_execution_bundle.py": "sha256:607945d5a51e3a139ecd35a8dceaf77a2027b0f842a577587616e0490b35c663", + "matched_source:src/aisle/harness/typed_graph_audit.py": "sha256:2266249a3a96e8acdfd8a40e2085eef31c519b351832e9b7bd7c05edb029a186", + "matched_source:src/aisle/harness/typed_graph_hosts.py": "sha256:a5cfd9b3b603f901370579f65320f995d4bc0c71852cf2ed64784d429a942abd", + "matched_source:src/aisle/harness/typed_graph_stage.py": "sha256:0ea37ad396fb3321978c5f8f63d869a8ba0a652f7377cad60f13060b5a5ccbfb", + "matched_source:src/aisle/harness/typed_node_host.py": "sha256:ab3a222a2599f46415af929023d519efd84a427794dd154ef1e2113a7e9f0d4a", + "matched_source:src/aisle/harness/typed_node_requests.py": "sha256:c3712381bc7383beef1e3304ce05f6a4fe27fb16d9b7e97e7f51f8d7cb6de78b", + "matched_source:src/aisle/harness/typed_node_supervisor.py": "sha256:d964517b767bdd087771bf9233c05de61604cf5f705124d6b83be05feb0cb577", + "matched_source:src/aisle/harness/typed_node_worker.py": "sha256:890cb125e12b05afa3609186ceae6d1d45d1d765f653e2c0b7345b00b2c24a89", + "matched_source:src/aisle/harness/typed_run_prepare.py": "sha256:787ba2edc04efa6df752fdba2112ff50e34d2061acb74471c52c78e813382063", + "matched_source:src/aisle/harness/typed_snapshot.py": "sha256:96003b648f09d1348375ca72709537dd80aa4ddecd60e87465011d221d3200ec", + "matched_source:src/aisle/harness/typed_stage_provider.py": "sha256:afe7e37ef08020f5bf19630ef9117af2bdf9d8fe68ceba7f4a352f7c61c8c9ff", + "matched_source:src/aisle/harness/typed_validation.py": "sha256:fa46d29d23d32ee4e65cbc9738ff5af6573607865162363f9d7dcd983b3a0ec9", + "matched_source:src/aisle/harness/typed_worker_launch.py": "sha256:bceac67581c7f143ef778d417217f9a823547c5747f9a2e9a2e6d39bd26c2c4a", + "matched_source:src/aisle/harness/typed_worker_provisioning.py": "sha256:0e594beec5a07829044900d4a93fcde8e2e0e9d50e1e0f7e19809f9d2845b3ba", + "matched_source:src/aisle/harness/validate.py": "sha256:6324c0a684e4d56a8711ff7ed5851a23c1c6131f1040f3f6e80c2f896a2ac955", + "matched_source:src/aisle/harness/worker_authority_probe.py": "sha256:9ee0638062e062c500c7103664eecd541e5d68056bb83368223c09f145dad6ee", + "matched_source:src/aisle/harness/worker_capability.py": "sha256:272c82452cd6f81912593b4584729cb7bb0e8f528d662cb3021e8dfc6ce70e83", + "matched_source:src/aisle/harness/worker_declaration.py": "sha256:15f42f83cae62a8e917f010b3e1571dda81a3cc7e7e3d630ed2b697fb0e8c0c2", + "matched_source:src/aisle/harness/worker_network_probe.py": "sha256:eb0d75fcbd4e8242eb61701f07e02e83ee6e7d9053247df5572f699969bcceb2", + "matched_source:src/aisle/kinematics.py": "sha256:0656cb4c9c4769365c42fc175deefb69d7a7a3dda77991329b1cedf33f7d5aaa", + "matched_source:src/aisle/mobility/guard.py": "sha256:3d0d4ddd757a8ae1840807511f8816f85feff0f2103fc490a8d29db6fa3663d0", + "matched_source:src/aisle/monolith/confinement.py": "sha256:f207d08bac411ae0cb9a5986b3f5f4d55e55d1407e0c0e6889ec5d489b232969", + "matched_source:src/aisle/monolith/primitive_api.py": "sha256:d5a344125ca3f434e489325b6bc7e20422082d18abee8ba7cd4deccda901327a", + "matched_source:src/aisle/monolith/primitives.py": "sha256:e4b3ef279708ef5fbd6e114f524a6c727f3c9d39a41d56dbfbe73d26e3ac515b", + "matched_source:src/aisle/monolith/proxy.py": "sha256:191b4ce164ea431ccbe728ad9ca5b482a09894093ee389643e099c3774df4ca4", + "matched_source:src/aisle/monolith/requests.py": "sha256:04151bf8838eda75be2a9d0c91240607283fb826712b90a8f99eab5d83d30185", + "matched_source:src/aisle/monolith/supervisor.py": "sha256:7dbe215dc5bddd12d414d1b88e896000fc64a9c71746230518eb2f47f1ef1051", + "matched_source:src/aisle/monolith/wire.py": "sha256:3418587b67e3ce2481e68c2809aaafc8e17f244246d33b055ee05f06ac6177a6", + "matched_source:src/aisle/monolith/worker.py": "sha256:931e42442b9f69157eab17cb73f1fa235eb053da7d843915ef889df5f84fb2d7", + "matched_source:src/aisle/monolith/worker_config.py": "sha256:55a26fdfa8b701470e50d54ffa9384d0ed89a0dc157867ac7e0005671e623312", + "matched_source:src/aisle/monolith/worker_launch.py": "sha256:3b823405305819ba4f1bd7cce0ef18a3ecc1276d0a221e78f3e0524ecfb2a8ef", + "matched_source:src/aisle/nodes/budget_guard.py": "sha256:e15bc5f70b295e7dfd32f72e4a5cf72dffca9da50e9e8f6bb18360fb38069abf", + "matched_source:src/aisle/nodes/h6_fault.py": "sha256:c93b8e1607b7627adef9d971753727b9c602be44ae539def9188344332a63485", + "matched_source:src/aisle/nodes/l2_pose.py": "sha256:d2889dac35af94cdf89a280cf734f672098ce95a4a347e1b9797545c2269fda2", + "matched_source:src/aisle/nodes/label_reader.py": "sha256:aeb51080744779f68fb56d6ff408993e92e3e0b929b7d353af7521537f252517", + "matched_source:src/aisle/nodes/monolith_broker.py": "sha256:6fb86d2fdcae8d9ac0314e4cab3100cd925c35a96f74249006392e042847c048", + "matched_source:src/aisle/nodes/perception_session.py": "sha256:cf0fd80a43a338711456d7db546167d082fb6ecc461a828f3cb0e2d42701fc9e", + "matched_source:src/aisle/scenes/locations.toml": "sha256:baea316611ce2f05751ab3058b4b46acbdd9801d8734faca0272c06b2565bcca", + "matched_source:src/aisle/scenes/meds.toml": "sha256:3b25c5eb6586b311fbe8aa3ffc9fdab76cb1b60c9781c3a7a31b12ab6b61dc6b", + "matched_source:src/aisle/scenes/pharmacy.py": "sha256:642e4a57c9225804b44d4e6909fad6514475598cf002dff300b9ce4455225981", + "matched_source:src/aisle/scenes/physics.toml": "sha256:7a400f5cb01a4a93e2a1daa07fa3b2fafd6598643e15965d779c924480f7c36c", + "matched_source:src/aisle/scenes/planogram.toml": "sha256:c579e207de210ed102ff81e15bf0877a91ce0d4ac31ebce2c31e484322f0e576", + "matched_source:src/aisle/scenes/store.py": "sha256:ea79ce1c91d2085ac920fa1ffe87c0c662c875e0e36da5866c1bc60ae5093a66", + "matched_source:src/aisle/topics.py": "sha256:431a0e0bee4c4c3d83f09f9452eb481f238ec0b8cab11b7370bebc117beeb26f", + "matched_source:src/aisle/turn_node.py": "sha256:0c6cd11cd802ebf8934a671caeb4b8a590e5f43b23f03c52515f51eac4288705", + "matched_source:src/aisle/turns.py": "sha256:f0014e3d8ea6b07d4effd10c5bf0b7de08b89ecec1e164580509ad25694b36a4", + "matched_source:src/aisle/verifier/calibration.py": "sha256:a00b91406d27a3b6123701a7a7270258633ad9906c28dbb6f1110fbb07b59568", + "matched_source:src/aisle/verifier/models.lock": "sha256:3fa7271740f3ecd361213f45de140452f41216b80600e343e2daa9f2a413eab5", + "matched_source:src/aisle/verifier/models.py": "sha256:1b2905181bc757f8390666c2a4c7303f674a6d8a31232e8feae582b5cc24909f", + "matched_source:src/aisle/verifier/placement.toml": "sha256:8d8f37165cf4bb4214486597689d40d5df8fb36e2bb8f15e65f6be8801d3717f", + "matched_source:src/aisle/verifier/realistic.py": "sha256:769f6295d3c15a5be02da8ec815a401728059608f2ae92c904d5bec0dd18d156", + "matched_source:src/aisle/verifier/stages.py": "sha256:63848f9e4c0e59020461e3350bf8ca2a8d0ae93cc45cd43c5786c7b6f95a443c", + "matched_source:src/aisle/verifier/thresholds.toml": "sha256:567ac0544a4c382d3ce6547d8df868e6ca37c056b146cf0d1ee11934b1be16d3", + "matched_source:tools/campaign.py": "sha256:cb33ab83f9125ea9708a73f639ca38ad539fbc77e6c5b03e0b49dc2e66abd8f0", + "matched_source:tools/matched_campaign.py": "sha256:882cbdd39c6a9413c8926c392c106eb2d014b3ac5615271bb6e54b38d1f6758b", + "monolithic_validators": "sha256:1983ed8c26b2371c993c571f33f5c7e4ee81ea2597f9468b43d53c9970465ccf", + "nested_host_documentation": "sha256:1c4777e4d8a29882ae5332f96df6b175423e7e5d1f279357fdb1286bace12959", + "pilot_assignments_cli": "sha256:bafcfdf4f79d5d0abd7442355454ea7ca476928afd57ed7c83f38da0c3726cc8", + "pilot_records_cli": "sha256:51438c6e14fca6368e1b91ebce77fd13c6bcc821edb61666553750a91d3ab75f", + "protocol": "sha256:459ce9961e3f5c7975ca38463e1807ca447f4463b0c9ad3e8eab1831b06b1d58", + "protocol_template": "sha256:9d636f3dc462a2e051ee0b3ded565aff1c5be86331e5b1b6692988509ccf088e", + "provider_admission_documentation": "sha256:856851935d15d8cb0c5653548ae9312441263cce9a3300e36d0863f5bdb96cf4", + "public_observation_source:analysis/monolithic-control/experts-t1-l2-v1.json": "sha256:eca1034c63620f5a901a0b2da4c157391643bc4e5aaf48457ac172ff8a656288", + "public_observation_source:analysis/monolithic-control/treatment-table-t1-l2-v1.json": "sha256:58f7164a95500f514a815b44ef1a681462c13127c3fbff9f887ae3873f21eac2", + "public_observation_source:docs/decisions/ADR-pilot-public-observations.md": "sha256:0bf46aac693fa9b9e78997d6843cb87112f7716d006a0f4a63e987f3eca5ea5e", + "public_observation_source:docs/monolithic/public-observations.md": "sha256:1b395b766ef5345169e2047a3a86f80648f5d265f3d4445315f314214c7068b7", + "public_observation_source:graphs/pilot_t1_l2_monolithic.yaml": "sha256:ebf12538d0b9def23ce4ea70a9feb375c98535f774bbf16d4d839aaa4e99d27c", + "public_observation_source:graphs/pilot_t1_l2_typed.yaml": "sha256:9dd2a3f4a71ed954987861e80a163c644e3a0aec4f3556b5fa353fa8f83aa8cb", + "public_observation_source:graphs/turn_plans/pilot_t1_l2_monolithic.json": "sha256:089558615e62539fb8121c6521d1bcc2af1403d4e896be9ef731b353e9b55d8c", + "public_observation_source:graphs/turn_plans/pilot_t1_l2_typed.json": "sha256:605b00af15cc3e8ba0331468532277c2e8ca4d3bcc196e84978d957fde59f23e", + "public_observation_source:registry/manifests/pilot-policy-surface.yaml": "sha256:5e189856d64aa49c90b573879d5b71abe230a2790afad5b9214a061d521de655", + "public_observation_source:src/aisle/harness/pilot_model_cache.py": "sha256:17a90497a33015388f5426dad4d5c3e29700a8b48ccb47a6e02380c3274339f5", + "public_observation_source:src/aisle/harness/pilot_policy_surface.py": "sha256:bb08dc54cf6a86907d68bab554baab1d562ec754b2736812e68798b96395cf80", + "research_prompt": "sha256:e736bbd6caa7236dbfafe92ae42c0c7f13088764af1b753f6685d398c111b7c0", + "seed_commitment_predecessor": "sha256:a0178d03bbe358505744293f91a499a695dd6e26c767582592d231842b3d24db", + "session_runner": "sha256:cb33ab83f9125ea9708a73f639ca38ad539fbc77e6c5b03e0b49dc2e66abd8f0", + "session_validators": "sha256:1b1c53d6389881c146726d865a741a681f3d5039c9bf99558a53ee368250971e", + "simulator_bridge": "sha256:b4d5e6eab8e60b65f2f0eb29c94f0634a58b921aae0bd643587d400ff689ba7c", + "spec": "sha256:f5096c587cd9687859f2dc80ec807d1bbf959cb4e05c152154e71e47bdc1f593", + "treatment_integrity": "sha256:b37162c6f295c5686da9b3089eb3bd96b69f68f01f41da0a784c4017658b10a6", + "treatment_postflight": "sha256:27b591c291491ff1e943a62ec29ecd054b6caa29547b7872844cdc67dc403e9c", + "treatment_table": "sha256:1b4b37f216ae222a673b7fc83741ec666a994c1e42320f821479455453aef3a0" + }, + "campaign_id": "cse-causal-study-v27", + "commands": [ + "uv run harness stats validate --protocol analysis/freeze/cse-causal-study-v1/protocol.json --purpose power", + "uv run harness stats validate --protocol analysis/freeze/cse-causal-study-v1/protocol.json --purpose freeze --output /tmp/cse-causal-study-v22-freeze-refusal.json", + "uv run harness freeze check --manifest analysis/freeze/cse-causal-study-v26/freeze-manifest.json --allow-withheld-seeds" + ], + "declaration": { + "analysis": { + "scripts": [ + "src/aisle/harness/benchmark_statistics.py", + "src/aisle/harness/causal_study.py", + "src/aisle/harness/pilot_records.py" + ], + "seed": 500001 + }, + "artifacts": { + "agent_adapters": "tools/agent_adapters.py", + "analyzer": "src/aisle/harness/benchmark_statistics.py", + "budget_ceiling": "harness/budget.toml", + "dependency_declaration": "pyproject.toml", + "dependency_lock": "uv.lock", + "dora_runtime_selection": "dora-runtime.json", + "frontend_conformance_decision": "docs/decisions/ADR-65.md", + "frontend_request_authorization_documentation": "docs/benchmark/v1/frontend-request-authorization.md", + "matched_session_decision": "docs/decisions/ADR-62.md", + "matched_session_documentation": "docs/monolithic/matched-session.md", + "matched_source:assets/so101/so101.urdf": "assets/so101/so101.urdf", + "matched_source:docs/monolithic/allowlist-t1-l2.json": "docs/monolithic/allowlist-t1-l2.json", + "matched_source:docs/monolithic/candidates.json": "docs/monolithic/candidates.json", + "matched_source:docs/monolithic/experts-t1-l2.json": "docs/monolithic/experts-t1-l2.json", + "matched_source:docs/monolithic/interface-map-t1-l2.json": "docs/monolithic/interface-map-t1-l2.json", + "matched_source:docs/monolithic/parity-protocol-t1-l2.json": "docs/monolithic/parity-protocol-t1-l2.json", + "matched_source:docs/monolithic/treatment-table-t1-l2.json": "docs/monolithic/treatment-table-t1-l2.json", + "matched_source:env/limits.toml": "env/limits.toml", + "matched_source:experts/monolithic/expert_t1_l2.py": "experts/monolithic/expert_t1_l2.py", + "matched_source:graphs/pilot_t1_l2_monolithic.yaml": "graphs/pilot_t1_l2_monolithic.yaml", + "matched_source:src/aisle/__init__.py": "src/aisle/__init__.py", + "matched_source:src/aisle/embodiment.py": "src/aisle/embodiment.py", + "matched_source:src/aisle/harness/_code_mode_protocol/__init__.py": "src/aisle/harness/_code_mode_protocol/__init__.py", + "matched_source:src/aisle/harness/_code_mode_protocol/descriptor.pb": "src/aisle/harness/_code_mode_protocol/descriptor.pb", + "matched_source:src/aisle/harness/_code_mode_protocol/protocol.proto": "src/aisle/harness/_code_mode_protocol/protocol.proto", + "matched_source:src/aisle/harness/candidate_launch.py": "src/aisle/harness/candidate_launch.py", + "matched_source:src/aisle/harness/candidates.py": "src/aisle/harness/candidates.py", + "matched_source:src/aisle/harness/cli.py": "src/aisle/harness/cli.py", + "matched_source:src/aisle/harness/code_mode_audit.py": "src/aisle/harness/code_mode_audit.py", + "matched_source:src/aisle/harness/code_mode_authority.py": "src/aisle/harness/code_mode_authority.py", + "matched_source:src/aisle/harness/code_mode_proxy.py": "src/aisle/harness/code_mode_proxy.py", + "matched_source:src/aisle/harness/code_mode_runner.py": "src/aisle/harness/code_mode_runner.py", + "matched_source:src/aisle/harness/common.py": "src/aisle/harness/common.py", + "matched_source:src/aisle/harness/frontend_app_server.py": "src/aisle/harness/frontend_app_server.py", + "matched_source:src/aisle/harness/frontend_app_server_audit.py": "src/aisle/harness/frontend_app_server_audit.py", + "matched_source:src/aisle/harness/frontend_conformance.py": "src/aisle/harness/frontend_conformance.py", + "matched_source:src/aisle/harness/frontend_dispatch.py": "src/aisle/harness/frontend_dispatch.py", + "matched_source:src/aisle/harness/frontend_dispatch_audit.py": "src/aisle/harness/frontend_dispatch_audit.py", + "matched_source:src/aisle/harness/frontend_effects.py": "src/aisle/harness/frontend_effects.py", + "matched_source:src/aisle/harness/frontend_qualification.py": "src/aisle/harness/frontend_qualification.py", + "matched_source:src/aisle/harness/frontend_request_audit.py": "src/aisle/harness/frontend_request_audit.py", + "matched_source:src/aisle/harness/frontend_request_authority.py": "src/aisle/harness/frontend_request_authority.py", + "matched_source:src/aisle/harness/guard_divergence.py": "src/aisle/harness/guard_divergence.py", + "matched_source:src/aisle/harness/heldout_eval.py": "src/aisle/harness/heldout_eval.py", + "matched_source:src/aisle/harness/hidden_access_log.py": "src/aisle/harness/hidden_access_log.py", + "matched_source:src/aisle/harness/matched_app_server.py": "src/aisle/harness/matched_app_server.py", + "matched_source:src/aisle/harness/matched_collection.py": "src/aisle/harness/matched_collection.py", + "matched_source:src/aisle/harness/matched_dynamic_run.py": "src/aisle/harness/matched_dynamic_run.py", + "matched_source:src/aisle/harness/matched_evidence.py": "src/aisle/harness/matched_evidence.py", + "matched_source:src/aisle/harness/matched_frontend.py": "src/aisle/harness/matched_frontend.py", + "matched_source:src/aisle/harness/matched_run.py": "src/aisle/harness/matched_run.py", + "matched_source:src/aisle/harness/matched_run_launch.py": "src/aisle/harness/matched_run_launch.py", + "matched_source:src/aisle/harness/matched_runtime.py": "src/aisle/harness/matched_runtime.py", + "matched_source:src/aisle/harness/matched_session.py": "src/aisle/harness/matched_session.py", + "matched_source:src/aisle/harness/matched_tool_service.py": "src/aisle/harness/matched_tool_service.py", + "matched_source:src/aisle/harness/matched_tools.py": "src/aisle/harness/matched_tools.py", + "matched_source:src/aisle/harness/mcp_harness_authority.py": "src/aisle/harness/mcp_harness_authority.py", + "matched_source:src/aisle/harness/mcp_harness_source.py": "src/aisle/harness/mcp_harness_source.py", + "matched_source:src/aisle/harness/mcp_harness_transport.py": "src/aisle/harness/mcp_harness_transport.py", + "matched_source:src/aisle/harness/monolith.py": "src/aisle/harness/monolith.py", + "matched_source:src/aisle/harness/monolithic.py": "src/aisle/harness/monolithic.py", + "matched_source:src/aisle/harness/monolithic_run_evidence.py": "src/aisle/harness/monolithic_run_evidence.py", + "matched_source:src/aisle/harness/monolithic_run_prepare.py": "src/aisle/harness/monolithic_run_prepare.py", + "matched_source:src/aisle/harness/pilot_records.py": "src/aisle/harness/pilot_records.py", + "matched_source:src/aisle/harness/provider_hosted.py": "src/aisle/harness/provider_hosted.py", + "matched_source:src/aisle/harness/provider_relay.py": "src/aisle/harness/provider_relay.py", + "matched_source:src/aisle/harness/provider_response_authority.py": "src/aisle/harness/provider_response_authority.py", + "matched_source:src/aisle/harness/provider_runner.py": "src/aisle/harness/provider_runner.py", + "matched_source:src/aisle/harness/provider_source_audit.py": "src/aisle/harness/provider_source_audit.py", + "matched_source:src/aisle/harness/registry.py": "src/aisle/harness/registry.py", + "matched_source:src/aisle/harness/rollout.py": "src/aisle/harness/rollout.py", + "matched_source:src/aisle/harness/rollout_client.py": "src/aisle/harness/rollout_client.py", + "matched_source:src/aisle/harness/simulator_work.py": "src/aisle/harness/simulator_work.py", + "matched_source:src/aisle/harness/traces.py": "src/aisle/harness/traces.py", + "matched_source:src/aisle/harness/treatment_ambient.py": "src/aisle/harness/treatment_ambient.py", + "matched_source:src/aisle/harness/treatment_confinement.py": "src/aisle/harness/treatment_confinement.py", + "matched_source:src/aisle/harness/treatment_integrity.py": "src/aisle/harness/treatment_integrity.py", + "matched_source:src/aisle/harness/treatment_postflight.py": "src/aisle/harness/treatment_postflight.py", + "matched_source:src/aisle/harness/typed_execution_bundle.py": "src/aisle/harness/typed_execution_bundle.py", + "matched_source:src/aisle/harness/typed_graph_audit.py": "src/aisle/harness/typed_graph_audit.py", + "matched_source:src/aisle/harness/typed_graph_hosts.py": "src/aisle/harness/typed_graph_hosts.py", + "matched_source:src/aisle/harness/typed_graph_stage.py": "src/aisle/harness/typed_graph_stage.py", + "matched_source:src/aisle/harness/typed_node_host.py": "src/aisle/harness/typed_node_host.py", + "matched_source:src/aisle/harness/typed_node_requests.py": "src/aisle/harness/typed_node_requests.py", + "matched_source:src/aisle/harness/typed_node_supervisor.py": "src/aisle/harness/typed_node_supervisor.py", + "matched_source:src/aisle/harness/typed_node_worker.py": "src/aisle/harness/typed_node_worker.py", + "matched_source:src/aisle/harness/typed_run_prepare.py": "src/aisle/harness/typed_run_prepare.py", + "matched_source:src/aisle/harness/typed_snapshot.py": "src/aisle/harness/typed_snapshot.py", + "matched_source:src/aisle/harness/typed_stage_provider.py": "src/aisle/harness/typed_stage_provider.py", + "matched_source:src/aisle/harness/typed_validation.py": "src/aisle/harness/typed_validation.py", + "matched_source:src/aisle/harness/typed_worker_launch.py": "src/aisle/harness/typed_worker_launch.py", + "matched_source:src/aisle/harness/typed_worker_provisioning.py": "src/aisle/harness/typed_worker_provisioning.py", + "matched_source:src/aisle/harness/validate.py": "src/aisle/harness/validate.py", + "matched_source:src/aisle/harness/worker_authority_probe.py": "src/aisle/harness/worker_authority_probe.py", + "matched_source:src/aisle/harness/worker_capability.py": "src/aisle/harness/worker_capability.py", + "matched_source:src/aisle/harness/worker_declaration.py": "src/aisle/harness/worker_declaration.py", + "matched_source:src/aisle/harness/worker_network_probe.py": "src/aisle/harness/worker_network_probe.py", + "matched_source:src/aisle/kinematics.py": "src/aisle/kinematics.py", + "matched_source:src/aisle/mobility/guard.py": "src/aisle/mobility/guard.py", + "matched_source:src/aisle/monolith/confinement.py": "src/aisle/monolith/confinement.py", + "matched_source:src/aisle/monolith/primitive_api.py": "src/aisle/monolith/primitive_api.py", + "matched_source:src/aisle/monolith/primitives.py": "src/aisle/monolith/primitives.py", + "matched_source:src/aisle/monolith/proxy.py": "src/aisle/monolith/proxy.py", + "matched_source:src/aisle/monolith/requests.py": "src/aisle/monolith/requests.py", + "matched_source:src/aisle/monolith/supervisor.py": "src/aisle/monolith/supervisor.py", + "matched_source:src/aisle/monolith/wire.py": "src/aisle/monolith/wire.py", + "matched_source:src/aisle/monolith/worker.py": "src/aisle/monolith/worker.py", + "matched_source:src/aisle/monolith/worker_config.py": "src/aisle/monolith/worker_config.py", + "matched_source:src/aisle/monolith/worker_launch.py": "src/aisle/monolith/worker_launch.py", + "matched_source:src/aisle/nodes/budget_guard.py": "src/aisle/nodes/budget_guard.py", + "matched_source:src/aisle/nodes/h6_fault.py": "src/aisle/nodes/h6_fault.py", + "matched_source:src/aisle/nodes/l2_pose.py": "src/aisle/nodes/l2_pose.py", + "matched_source:src/aisle/nodes/label_reader.py": "src/aisle/nodes/label_reader.py", + "matched_source:src/aisle/nodes/monolith_broker.py": "src/aisle/nodes/monolith_broker.py", + "matched_source:src/aisle/nodes/perception_session.py": "src/aisle/nodes/perception_session.py", + "matched_source:src/aisle/scenes/locations.toml": "src/aisle/scenes/locations.toml", + "matched_source:src/aisle/scenes/meds.toml": "src/aisle/scenes/meds.toml", + "matched_source:src/aisle/scenes/pharmacy.py": "src/aisle/scenes/pharmacy.py", + "matched_source:src/aisle/scenes/physics.toml": "src/aisle/scenes/physics.toml", + "matched_source:src/aisle/scenes/planogram.toml": "src/aisle/scenes/planogram.toml", + "matched_source:src/aisle/scenes/store.py": "src/aisle/scenes/store.py", + "matched_source:src/aisle/topics.py": "src/aisle/topics.py", + "matched_source:src/aisle/turn_node.py": "src/aisle/turn_node.py", + "matched_source:src/aisle/turns.py": "src/aisle/turns.py", + "matched_source:src/aisle/verifier/calibration.py": "src/aisle/verifier/calibration.py", + "matched_source:src/aisle/verifier/models.lock": "src/aisle/verifier/models.lock", + "matched_source:src/aisle/verifier/models.py": "src/aisle/verifier/models.py", + "matched_source:src/aisle/verifier/placement.toml": "src/aisle/verifier/placement.toml", + "matched_source:src/aisle/verifier/realistic.py": "src/aisle/verifier/realistic.py", + "matched_source:src/aisle/verifier/stages.py": "src/aisle/verifier/stages.py", + "matched_source:src/aisle/verifier/thresholds.toml": "src/aisle/verifier/thresholds.toml", + "matched_source:tools/campaign.py": "tools/campaign.py", + "matched_source:tools/matched_campaign.py": "tools/matched_campaign.py", + "monolithic_validators": "src/aisle/harness/monolithic.py", + "nested_host_documentation": "docs/monolithic/code-mode-admission.md", + "pilot_assignments_cli": "tools/pilot_assignments.py", + "pilot_records_cli": "tools/pilot_records.py", + "protocol": "analysis/freeze/cse-causal-study-v1/protocol.json", + "protocol_template": "analysis/statistics/protocol-template.json", + "provider_admission_documentation": "docs/monolithic/provider-admission.md", + "public_observation_source:analysis/monolithic-control/experts-t1-l2-v1.json": "analysis/monolithic-control/experts-t1-l2-v1.json", + "public_observation_source:analysis/monolithic-control/treatment-table-t1-l2-v1.json": "analysis/monolithic-control/treatment-table-t1-l2-v1.json", + "public_observation_source:docs/decisions/ADR-pilot-public-observations.md": "docs/decisions/ADR-pilot-public-observations.md", + "public_observation_source:docs/monolithic/public-observations.md": "docs/monolithic/public-observations.md", + "public_observation_source:graphs/pilot_t1_l2_monolithic.yaml": "graphs/pilot_t1_l2_monolithic.yaml", + "public_observation_source:graphs/pilot_t1_l2_typed.yaml": "graphs/pilot_t1_l2_typed.yaml", + "public_observation_source:graphs/turn_plans/pilot_t1_l2_monolithic.json": "graphs/turn_plans/pilot_t1_l2_monolithic.json", + "public_observation_source:graphs/turn_plans/pilot_t1_l2_typed.json": "graphs/turn_plans/pilot_t1_l2_typed.json", + "public_observation_source:registry/manifests/pilot-policy-surface.yaml": "registry/manifests/pilot-policy-surface.yaml", + "public_observation_source:src/aisle/harness/pilot_model_cache.py": "src/aisle/harness/pilot_model_cache.py", + "public_observation_source:src/aisle/harness/pilot_policy_surface.py": "src/aisle/harness/pilot_policy_surface.py", + "research_prompt": "harness/CLAUDE.research.md", + "seed_commitment_predecessor": "analysis/freeze/cse-causal-study-v26/freeze-manifest.json", + "session_runner": "tools/campaign.py", + "session_validators": "src/aisle/harness/causal_study.py", + "simulator_bridge": "src/aisle/nodes/dora_genesis.py", + "spec": "specs/500-typed-monolithic-causal-study.md", + "treatment_integrity": "src/aisle/harness/treatment_integrity.py", + "treatment_postflight": "src/aisle/harness/treatment_postflight.py", + "treatment_table": "analysis/monolithic-control/treatment-table-v1.json" + }, + "budgets": { + "campaign_ceiling": "harness/budget.toml", + "rollouts_per_session": 20, + "session_tokens": 200000, + "session_wall_s": 3600, + "sim_episodes_per_session": 160 + }, + "campaign_id": "cse-causal-study-v27", + "commands": [ + "uv run harness stats validate --protocol analysis/freeze/cse-causal-study-v1/protocol.json --purpose power", + "uv run harness stats validate --protocol analysis/freeze/cse-causal-study-v1/protocol.json --purpose freeze --output /tmp/cse-causal-study-v22-freeze-refusal.json", + "uv run harness freeze check --manifest analysis/freeze/cse-causal-study-v26/freeze-manifest.json --allow-withheld-seeds" + ], + "decision_rules": { + "alpha": 0.05, + "equivalence_margin": { + "cost_at_equal_quality": { + "acceptance_probability": 0.1, + "heldout_quality": 0.1, + "rule": "CSE-5: both must pass or cost claims stay unconditional", + "sidedness": "two-sided 0.90" + } + }, + "multiplicity": "single primary endpoint", + "power": 0.8, + "primary_rule": "one-sided superiority at alpha 0.05 on the pooled Newcombe interval lower bound > 0 with no reversed stratum; otherwise weakened/rejected/undecidable by the #358 rule (CSE-3)", + "smallest_effect": { + "measure": "risk difference in session success", + "value": 0.25 + }, + "stopping_rule": "fixed n from the frozen power analysis using pilot-only control success; absolute floor 10 randomized assignments per arm per agent-system x task stratum (CSE-8); no optional stopping or extension" + }, + "endpoints": { + "primary": [ + { + "aggregation": "risk difference typed_dataflow - monolithic; exact arm intervals; Newcombe interval; per agent-system x task stratum before pooling (CSE-1, CSE-17)", + "direction": "higher", + "id": "E-CSE-1", + "outcome": "session_success", + "status": "inferential", + "unit": "agent_session" + } + ], + "secondary": [ + { + "aggregation": "counts by arm", + "direction": "descriptive", + "id": "E-CSE-2", + "outcome": "validation_and_launch_failure_class", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "Kaplan-Meier with censoring", + "direction": "lower", + "id": "E-CSE-3", + "outcome": "time_to_first_accepted_system", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "unconditional session estimands; no at-equal-quality wording unless CSE-5 equivalence passes", + "direction": "lower", + "id": "E-CSE-4", + "outcome": "tokens_tool_calls_wall_active_time_sim_work_rollouts", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "counts", + "direction": "descriptive", + "id": "E-CSE-5", + "outcome": "safety_interventions", + "status": "descriptive", + "unit": "agent_session" + }, + { + "aggregation": "dropped unless a blinded rubric is ratified before freeze (CSE-18)", + "direction": "n/a", + "id": "E-CSE-6", + "outcome": "artifact_legibility", + "status": "exploratory", + "unit": "agent_session" + } + ] + }, + "exclusions": { + "deviation_policy": "any behaviour-affecting change after registration is a documented deviation record and a new campaign version id; confirmatory data never revises the registered protocol", + "infrastructure": [ + "independent-controller-determined, arm-neutral facts only (CSE-14): host outage, provider interruption before first tool call, launcher crash before the session started" + ], + "rerun_policy": "a permitted rerun receives a new randomized id; the original row is retained with its exclusion reason and never replaced in place", + "treatment_integrity": [ + "contamination sentinel tripped (CSE-12)", + "model alias or CLI drift outside the frozen identity (CSE-6)", + "integrity verdict authored session-side (CSE-11)" + ] + }, + "hypotheses": [ + { + "direction": "typed_dataflow - monolithic > 0", + "id": "H-CSE-1", + "statement": "The bundled typed-dataflow surface raises the probability that a fresh coding-agent session produces a launching system passing the hidden held-out acceptance within budget, versus the equal-capability monolithic surface." + } + ], + "instrument_set": { + "agent_systems": [ + { + "binary": "claude", + "frozen": false, + "id": "claude-code", + "version_observed": "2.1.261" + }, + { + "binary": "codex", + "frozen": false, + "id": "codex-cli", + "version_observed": "0.151.0" + } + ], + "items": [ + "short_composition:TBD-by-bnd-task-band-calibration-v1", + "engineering:TBD-by-bnd-task-band-calibration-v1" + ], + "kind": "selected_task" + }, + "integrity_checks": [ + { + "gate": "CON-14 spec approval", + "kind": "external_review", + "owner_role": "project owner", + "record": null, + "status": "pending" + }, + { + "gate": "STA-12 independent statistical review", + "kind": "external_review", + "owner_role": "independent statistician", + "record": null, + "status": "pending" + }, + { + "gate": "MON-1 treatment table and expert parity (#344)", + "kind": "machine_check", + "owner_role": "parity harness; control surface exists, independent authorship and operator evidence remain pending", + "record": null, + "status": "pending" + }, + { + "gate": "BND-12 task-band freeze (#346)", + "kind": "machine_check", + "owner_role": "task-band auditor", + "record": null, + "status": "pending" + }, + { + "gate": "TRT confinement/mutation/agent-path (#353)", + "kind": "machine_check", + "owner_role": "integrity controller", + "record": null, + "status": "pending" + }, + { + "gate": "seed commitment verification", + "kind": "machine_check", + "owner_role": "holder of the original private seed and salt sources", + "record": null, + "status": "pending" + }, + { + "gate": "MON-8/MON-12 matched-session acceptance (#519)", + "kind": "machine_check", + "owner_role": "matched-session controller; complete acceptance evidence still required", + "record": null, + "status": "pending" + } + ], + "issue": 347, + "purpose": "confirmatory", + "schema_version": "aisle.freeze.declaration.v1", + "seed_commitment": { + "inherited_from": "analysis/freeze/cse-causal-study-v26/freeze-manifest.json", + "rule": "32 distinct integers in [1000,9999) drawn once from the OS CSPRNG, retained only under ~/aisle-private/freeze/cse-causal-study-v1/, disjoint by value from every committed dev/pilot seed list; revealed after the campaign closes", + "salt_source": "~/aisle-private/freeze/cse-causal-study-v1/salt.bin", + "seeds_source": "~/aisle-private/freeze/cse-causal-study-v1/seeds.json" + }, + "spec": "500", + "superseded": "cse-causal-study-v26 (the matched source set gained the ADR-67/ADR-68 engine seam: src/aisle/harness/cli.py, rollout.py and src/aisle/nodes/dora_genesis.py now select the physics engine behind AISLE_SIM_ENGINE. The protocol, seed lineage, decision rules and pending gates are unchanged, and Genesis remains the default and the only engine behind this campaign)" + }, + "declaration_sha256": "sha256:133acc8fb0533a0487cff997b85b172ee9ca113f59b09e1d40d1121069fc3054", + "frozen": false, + "gate_record_hashes": {}, + "git_head": "cabf2eb5d3be7fc960d4737cd64e449e26f5997d", + "issue": 347, + "pending_gates": [ + "BND-12 task-band freeze (#346)", + "CON-14 spec approval", + "MON-1 treatment table and expert parity (#344)", + "MON-8/MON-12 matched-session acceptance (#519)", + "STA-12 independent statistical review", + "TRT confinement/mutation/agent-path (#353)", + "seed commitment verification" + ], + "purpose": "confirmatory", + "schema_version": "aisle.freeze.manifest.v1", + "seed_commitment": "sha256:f3f4e5efe4d573e1159d93e7b53a35a50620606e3dc72d0b5d5057611a2b7522", + "spec": "500", + "status": "registered_pending_review", + "timestamp": null, + "timestamp_source": null +} diff --git a/analysis/freeze/flt-bank-calibration-v7/declaration.json b/analysis/freeze/flt-bank-calibration-v7/declaration.json new file mode 100644 index 00000000..2f2536fe --- /dev/null +++ b/analysis/freeze/flt-bank-calibration-v7/declaration.json @@ -0,0 +1,190 @@ +{ + "analysis": { + "scripts": [ + "src/aisle/harness/benchmark_statistics.py", + "src/aisle/harness/fault_bank.py", + "src/aisle/harness/fault_calibration.py", + "src/aisle/harness/fault_injector.py" + ], + "seed": 450001 + }, + "artifacts": { + "bank_commitment": "analysis/fault-bank/commitment-v2.json", + "baseline_graph": "graphs/expert_t1.yaml", + "calibration": "src/aisle/harness/fault_calibration.py", + "calibration_summary": "analysis/fault-bank/calibration-summary-v2.json", + "clean_run_episodes": "analysis/fault-bank/records/flt-clean-v2/episodes.jsonl", + "clean_run_manifest": "analysis/fault-bank/records/flt-clean-v2/manifest.json", + "injector": "src/aisle/harness/fault_injector.py", + "leakage_probe": "analysis/fault-bank/leakage-probe.json", + "leakage_prober": "src/aisle/harness/fault_leakage.py", + "leakage_report": "analysis/fault-bank/leakage-v2.json", + "public_fault_menu_h6": "tools/h6_campaign.py", + "seed_commitment_predecessor": "analysis/freeze/flt-bank-calibration-v6/freeze-manifest.json", + "spec": "specs/450-sealed-fault-bank.md", + "swap_engine": "src/aisle/harness/swap.py", + "validators": "src/aisle/harness/fault_bank.py" + }, + "budgets": { + "campaign_purpose": "excluded_pilot", + "episode_wall_s": 150, + "evidence_kind": "simulation", + "paired_seeds_per_instance": 8, + "severity_candidates_per_family": 2 + }, + "campaign_id": "flt-bank-calibration-v7", + "commands": [ + "uv run harness fault validate --bank ~/aisle-private/fault-bank/v2/bank.json --root .", + "uv run harness rollout --graph graphs/expert_t1.yaml --tier T1 --episodes 8 --seeds 0..7 --run-id flt-clean-v2 --no-idea-gate", + "uv run harness fault calibrate --bank ~/aisle-private/fault-bank/v2/bank.json --clean-run runs/flt-clean-v2 --clean-commit 265dcec --staging --raw-store ~/aisle-private/raw/fault-calibration-v2 --campaign-id flt-cal-v2 --output ~/aisle-private/fault-bank/v2/calibration-report.json", + "uv run harness fault leakage --report ~/aisle-private/fault-bank/v2/calibration-report.json --probe analysis/fault-bank/leakage-probe.json --output analysis/fault-bank/leakage-v2.json", + "uv run harness freeze check --manifest analysis/freeze/flt-bank-calibration-v6/freeze-manifest.json --allow-withheld-seeds" + ], + "decision_rules": { + "alpha": 0.05, + "equivalence_margin": { + "sham_control": { + "margin": 0.125, + "rule": "sham-minus-clean paired difference interval within +-margin" + } + }, + "power": 0.8, + "selection_rule": "least severe rung whose paired clean-minus-fault difference is at least 0.25 with at least 3 discordant seed pairs and an exact 95% lower bound above 0.5 on the clean-only share of discordant pairs; sham must show parity", + "smallest_effect": { + "measure": "paired risk difference in episode success", + "value": 0.25 + }, + "stopping_rule": "fixed 8 paired seeds per instance and severity; no severity added after seeing scored outcomes (FLT-10)" + }, + "endpoints": { + "primary": [ + { + "aggregation": "paired clean-minus-fault difference with exact interval", + "direction": "lower under fault", + "id": "E-FLT-1", + "outcome": "episode_success", + "status": "inferential", + "unit": "seed_pair" + } + ], + "secondary": [ + { + "aggregation": "counts", + "direction": "descriptive", + "id": "E-FLT-2", + "outcome": "failure_class_distribution", + "status": "descriptive", + "unit": "seed_pair" + }, + { + "aggregation": "counts", + "direction": "unchanged or lower", + "id": "E-FLT-3", + "outcome": "guard_interventions_and_safety_events", + "status": "descriptive", + "unit": "seed_pair" + }, + { + "aggregation": "does the typed evidence bundle contain a component-level signal", + "direction": "present", + "id": "E-FLT-4", + "outcome": "evidence_localizability", + "status": "descriptive", + "unit": "instance" + } + ] + }, + "exclusions": { + "deviation_policy": "any behaviour-affecting change after registration is a documented deviation record and a new campaign version id; confirmatory data never revises the registered protocol", + "infrastructure": [ + "dora launch failure before the first turn", + "watchdog expiry attributable to host load (host-load baseline retained)" + ], + "rerun_policy": "a permitted rerun receives a new randomized id; the original row is retained with its exclusion reason and never replaced in place", + "treatment_integrity": [ + "fault identity visible in any participant-facing surface (FLT-4)" + ] + }, + "hypotheses": [ + { + "direction": "fault < clean by >= 0.25", + "id": "H-FLT-1", + "statement": "Each candidate fault instance degrades the paired clean baseline episode success by at least the minimum meaningful degradation while leaving localizable evidence (FLT-9)." + }, + { + "direction": "sham == clean", + "id": "H-FLT-0", + "statement": "Sham (no-fault) controls are indistinguishable from the clean baseline at session start and in outcome (FLT-8)." + } + ], + "instrument_set": { + "items": [ + "perception", + "decision", + "motion", + "schema_metadata", + "clocking", + "runtime", + "coupled_distinct_targets_a", + "coupled_distinct_targets_b", + "sham_control" + ], + "kind": "fault_family", + "note": "instance ids, severities, and bytes are sealed outside the worktree (FLT-4); only families and their commitments are public" + }, + "integrity_checks": [ + { + "gate": "CON-7 frozen set", + "kind": "machine_check", + "owner_role": "env_hash checker", + "record": "tools/env_hash.json", + "status": "passed" + }, + { + "gate": "FLT-1/2/3/11 bank coverage", + "kind": "machine_check", + "owner_role": "harness fault validate (private manifest; commitment tracked)", + "record": "analysis/fault-bank/commitment.json", + "status": "passed" + }, + { + "gate": "FLT-4 sealed location denial", + "kind": "machine_check", + "owner_role": "#353 confinement adapter", + "record": null, + "status": "pending" + }, + { + "gate": "FLT-8 sham parity", + "kind": "machine_check", + "owner_role": "calibration report (sham at parity observed; the FLT-8 leakage probe classifier is not built)", + "record": null, + "status": "pending" + }, + { + "gate": "CON-14 SPEC 450 approval", + "kind": "external_review", + "owner_role": "project owner", + "record": null, + "status": "pending" + }, + { + "gate": "seed commitment verification", + "kind": "machine_check", + "owner_role": "holder of the original private seed and salt sources", + "record": null, + "status": "pending" + } + ], + "issue": 348, + "purpose": "calibration", + "schema_version": "aisle.freeze.declaration.v1", + "seed_commitment": { + "inherited_from": "analysis/freeze/flt-bank-calibration-v6/freeze-manifest.json", + "rule": "calibration seeds are the retained clean run's development seeds 0..7 (excluded_pilot); held-out assignment seeds for scoring are separate and sealed", + "salt_source": "~/aisle-private/freeze/flt-bank-calibration-v2/salt.bin", + "seeds_source": "~/aisle-private/freeze/flt-bank-calibration-v2/seeds.json" + }, + "spec": "450", + "superseded": "flt-bank-calibration-v6 (the calibration artifact src/aisle/harness/fault_calibration.py gained the ADR-67 `--sim-engine` passthrough and the swap engine src/aisle/harness/swap.py fenced src/aisle/sim; thresholds, seed lineage, pending gates and every other artifact are preserved)" +} diff --git a/analysis/freeze/flt-bank-calibration-v7/freeze-manifest.json b/analysis/freeze/flt-bank-calibration-v7/freeze-manifest.json new file mode 100644 index 00000000..4aa17de0 --- /dev/null +++ b/analysis/freeze/flt-bank-calibration-v7/freeze-manifest.json @@ -0,0 +1,245 @@ +{ + "analysis_script_hashes": { + "src/aisle/harness/benchmark_statistics.py": "sha256:22cde3e9d7c5cc80874c7eec30aa2898e30476d6c939c94ab3009e8b12af641e", + "src/aisle/harness/fault_bank.py": "sha256:be57dcc63453253e075ba3c6dde6498fda782ae7583bebc5955e0006ef6cfca3", + "src/aisle/harness/fault_calibration.py": "sha256:4ccd8397fb5b604aef044e85bad1d9bba531d63eb5dde4cfcbd3812801cd6ca2", + "src/aisle/harness/fault_injector.py": "sha256:0f44b950bce85b75bb1c7d8b21684aba93fdb0b6dd191d368684062964d2469d" + }, + "analysis_seed": 450001, + "artifact_hashes": { + "bank_commitment": "sha256:b4a2be405c0ad14e7f3c31ed25872b0cf2dc55fec231b9ce0dbee17ac00294c2", + "baseline_graph": "sha256:303d08b37c6f0ea3ab221778437029393e0a266094941d2124b54648a3cf48ed", + "calibration": "sha256:4ccd8397fb5b604aef044e85bad1d9bba531d63eb5dde4cfcbd3812801cd6ca2", + "calibration_summary": "sha256:f3c8f6175a59b4ab64e76b8889ee965bf8df7f6f5a8e07c9e74afb06babe9019", + "clean_run_episodes": "sha256:ad8f4816b902354dd24b087e51cb826f51bb843567399e46b62abb45b8187728", + "clean_run_manifest": "sha256:e6ba2825ff690e647e0990db6be9587a2551cfa3f844e3e7e918541543103900", + "injector": "sha256:0f44b950bce85b75bb1c7d8b21684aba93fdb0b6dd191d368684062964d2469d", + "leakage_probe": "sha256:bd057a5e3699e355a7d0147f07d01077fccb79e4ecd94c2bd031f811b8d895d8", + "leakage_prober": "sha256:5e82cb3c20550f493fb77d6256d1db1f1a302bebc2ade06015179c98a9baa7ec", + "leakage_report": "sha256:822dae77a35f3c537976f198e268d14c7342616d24ce11a1272656c40874fe35", + "public_fault_menu_h6": "sha256:e5b965ce2f4445e77fe50a1bb0548912b14ac1c1903a5f98a8f89a98aeb1ddb9", + "seed_commitment_predecessor": "sha256:fa9c240aff03f8af4518248eec38e16c940d8c9ad5b75e2a976177495ac45a7a", + "spec": "sha256:a96d43fb9bf2166568177eab48aada2dc80288325f4f55fae18f9c4ffc4f96f0", + "swap_engine": "sha256:7dcc0e48c9876be4194fb3accb863f3cf74a2f9c1faebdd61daa5293f74141db", + "validators": "sha256:be57dcc63453253e075ba3c6dde6498fda782ae7583bebc5955e0006ef6cfca3" + }, + "campaign_id": "flt-bank-calibration-v7", + "commands": [ + "uv run harness fault validate --bank ~/aisle-private/fault-bank/v2/bank.json --root .", + "uv run harness rollout --graph graphs/expert_t1.yaml --tier T1 --episodes 8 --seeds 0..7 --run-id flt-clean-v2 --no-idea-gate", + "uv run harness fault calibrate --bank ~/aisle-private/fault-bank/v2/bank.json --clean-run runs/flt-clean-v2 --clean-commit 265dcec --staging --raw-store ~/aisle-private/raw/fault-calibration-v2 --campaign-id flt-cal-v2 --output ~/aisle-private/fault-bank/v2/calibration-report.json", + "uv run harness fault leakage --report ~/aisle-private/fault-bank/v2/calibration-report.json --probe analysis/fault-bank/leakage-probe.json --output analysis/fault-bank/leakage-v2.json", + "uv run harness freeze check --manifest analysis/freeze/flt-bank-calibration-v6/freeze-manifest.json --allow-withheld-seeds" + ], + "declaration": { + "analysis": { + "scripts": [ + "src/aisle/harness/benchmark_statistics.py", + "src/aisle/harness/fault_bank.py", + "src/aisle/harness/fault_calibration.py", + "src/aisle/harness/fault_injector.py" + ], + "seed": 450001 + }, + "artifacts": { + "bank_commitment": "analysis/fault-bank/commitment-v2.json", + "baseline_graph": "graphs/expert_t1.yaml", + "calibration": "src/aisle/harness/fault_calibration.py", + "calibration_summary": "analysis/fault-bank/calibration-summary-v2.json", + "clean_run_episodes": "analysis/fault-bank/records/flt-clean-v2/episodes.jsonl", + "clean_run_manifest": "analysis/fault-bank/records/flt-clean-v2/manifest.json", + "injector": "src/aisle/harness/fault_injector.py", + "leakage_probe": "analysis/fault-bank/leakage-probe.json", + "leakage_prober": "src/aisle/harness/fault_leakage.py", + "leakage_report": "analysis/fault-bank/leakage-v2.json", + "public_fault_menu_h6": "tools/h6_campaign.py", + "seed_commitment_predecessor": "analysis/freeze/flt-bank-calibration-v6/freeze-manifest.json", + "spec": "specs/450-sealed-fault-bank.md", + "swap_engine": "src/aisle/harness/swap.py", + "validators": "src/aisle/harness/fault_bank.py" + }, + "budgets": { + "campaign_purpose": "excluded_pilot", + "episode_wall_s": 150, + "evidence_kind": "simulation", + "paired_seeds_per_instance": 8, + "severity_candidates_per_family": 2 + }, + "campaign_id": "flt-bank-calibration-v7", + "commands": [ + "uv run harness fault validate --bank ~/aisle-private/fault-bank/v2/bank.json --root .", + "uv run harness rollout --graph graphs/expert_t1.yaml --tier T1 --episodes 8 --seeds 0..7 --run-id flt-clean-v2 --no-idea-gate", + "uv run harness fault calibrate --bank ~/aisle-private/fault-bank/v2/bank.json --clean-run runs/flt-clean-v2 --clean-commit 265dcec --staging --raw-store ~/aisle-private/raw/fault-calibration-v2 --campaign-id flt-cal-v2 --output ~/aisle-private/fault-bank/v2/calibration-report.json", + "uv run harness fault leakage --report ~/aisle-private/fault-bank/v2/calibration-report.json --probe analysis/fault-bank/leakage-probe.json --output analysis/fault-bank/leakage-v2.json", + "uv run harness freeze check --manifest analysis/freeze/flt-bank-calibration-v6/freeze-manifest.json --allow-withheld-seeds" + ], + "decision_rules": { + "alpha": 0.05, + "equivalence_margin": { + "sham_control": { + "margin": 0.125, + "rule": "sham-minus-clean paired difference interval within +-margin" + } + }, + "power": 0.8, + "selection_rule": "least severe rung whose paired clean-minus-fault difference is at least 0.25 with at least 3 discordant seed pairs and an exact 95% lower bound above 0.5 on the clean-only share of discordant pairs; sham must show parity", + "smallest_effect": { + "measure": "paired risk difference in episode success", + "value": 0.25 + }, + "stopping_rule": "fixed 8 paired seeds per instance and severity; no severity added after seeing scored outcomes (FLT-10)" + }, + "endpoints": { + "primary": [ + { + "aggregation": "paired clean-minus-fault difference with exact interval", + "direction": "lower under fault", + "id": "E-FLT-1", + "outcome": "episode_success", + "status": "inferential", + "unit": "seed_pair" + } + ], + "secondary": [ + { + "aggregation": "counts", + "direction": "descriptive", + "id": "E-FLT-2", + "outcome": "failure_class_distribution", + "status": "descriptive", + "unit": "seed_pair" + }, + { + "aggregation": "counts", + "direction": "unchanged or lower", + "id": "E-FLT-3", + "outcome": "guard_interventions_and_safety_events", + "status": "descriptive", + "unit": "seed_pair" + }, + { + "aggregation": "does the typed evidence bundle contain a component-level signal", + "direction": "present", + "id": "E-FLT-4", + "outcome": "evidence_localizability", + "status": "descriptive", + "unit": "instance" + } + ] + }, + "exclusions": { + "deviation_policy": "any behaviour-affecting change after registration is a documented deviation record and a new campaign version id; confirmatory data never revises the registered protocol", + "infrastructure": [ + "dora launch failure before the first turn", + "watchdog expiry attributable to host load (host-load baseline retained)" + ], + "rerun_policy": "a permitted rerun receives a new randomized id; the original row is retained with its exclusion reason and never replaced in place", + "treatment_integrity": [ + "fault identity visible in any participant-facing surface (FLT-4)" + ] + }, + "hypotheses": [ + { + "direction": "fault < clean by >= 0.25", + "id": "H-FLT-1", + "statement": "Each candidate fault instance degrades the paired clean baseline episode success by at least the minimum meaningful degradation while leaving localizable evidence (FLT-9)." + }, + { + "direction": "sham == clean", + "id": "H-FLT-0", + "statement": "Sham (no-fault) controls are indistinguishable from the clean baseline at session start and in outcome (FLT-8)." + } + ], + "instrument_set": { + "items": [ + "perception", + "decision", + "motion", + "schema_metadata", + "clocking", + "runtime", + "coupled_distinct_targets_a", + "coupled_distinct_targets_b", + "sham_control" + ], + "kind": "fault_family", + "note": "instance ids, severities, and bytes are sealed outside the worktree (FLT-4); only families and their commitments are public" + }, + "integrity_checks": [ + { + "gate": "CON-7 frozen set", + "kind": "machine_check", + "owner_role": "env_hash checker", + "record": "tools/env_hash.json", + "status": "passed" + }, + { + "gate": "FLT-1/2/3/11 bank coverage", + "kind": "machine_check", + "owner_role": "harness fault validate (private manifest; commitment tracked)", + "record": "analysis/fault-bank/commitment.json", + "status": "passed" + }, + { + "gate": "FLT-4 sealed location denial", + "kind": "machine_check", + "owner_role": "#353 confinement adapter", + "record": null, + "status": "pending" + }, + { + "gate": "FLT-8 sham parity", + "kind": "machine_check", + "owner_role": "calibration report (sham at parity observed; the FLT-8 leakage probe classifier is not built)", + "record": null, + "status": "pending" + }, + { + "gate": "CON-14 SPEC 450 approval", + "kind": "external_review", + "owner_role": "project owner", + "record": null, + "status": "pending" + }, + { + "gate": "seed commitment verification", + "kind": "machine_check", + "owner_role": "holder of the original private seed and salt sources", + "record": null, + "status": "pending" + } + ], + "issue": 348, + "purpose": "calibration", + "schema_version": "aisle.freeze.declaration.v1", + "seed_commitment": { + "inherited_from": "analysis/freeze/flt-bank-calibration-v6/freeze-manifest.json", + "rule": "calibration seeds are the retained clean run's development seeds 0..7 (excluded_pilot); held-out assignment seeds for scoring are separate and sealed", + "salt_source": "~/aisle-private/freeze/flt-bank-calibration-v2/salt.bin", + "seeds_source": "~/aisle-private/freeze/flt-bank-calibration-v2/seeds.json" + }, + "spec": "450", + "superseded": "flt-bank-calibration-v6 (the calibration artifact src/aisle/harness/fault_calibration.py gained the ADR-67 `--sim-engine` passthrough and the swap engine src/aisle/harness/swap.py fenced src/aisle/sim; thresholds, seed lineage, pending gates and every other artifact are preserved)" + }, + "declaration_sha256": "sha256:bb231c598e56d2240a8d3c8230586b847f6a0bd014389183d381bcdae5b38ce1", + "frozen": false, + "gate_record_hashes": { + "CON-7 frozen set": "sha256:4e84ebe40d37cb26ef774e4fc6b46b54df7813ee051042308d1f409a9d9103ac", + "FLT-1/2/3/11 bank coverage": "sha256:8048b90e32d3d78cd10f8275e1d78263c4bfca71288e225667019c7cda455c85" + }, + "git_head": "6173684e8129dff85bcb3411a1f011bee4b96aab", + "issue": 348, + "pending_gates": [ + "CON-14 SPEC 450 approval", + "FLT-4 sealed location denial", + "FLT-8 sham parity", + "seed commitment verification" + ], + "purpose": "calibration", + "schema_version": "aisle.freeze.manifest.v1", + "seed_commitment": "sha256:32931dcb7c82f051a7b394ad37d4b24e59ce7e884a614d02adcdbf285eeebbca", + "spec": "450", + "status": "registered_pending_review", + "timestamp": null, + "timestamp_source": null +} diff --git a/analysis/monolithic-control/treatment-table-v1.json b/analysis/monolithic-control/treatment-table-v1.json index 8db5fec1..7ae14b20 100644 --- a/analysis/monolithic-control/treatment-table-v1.json +++ b/analysis/monolithic-control/treatment-table-v1.json @@ -1,5 +1,5 @@ { - "immutable_id": "sha256:fb41dbe66feb26993a0aada82c3ffc41692b80cb6b8806392f3ce7f20a85bc4a", + "immutable_id": "sha256:3e23d000fa6f041dcbbec4c9849d72c0f0f74caf6ed60fba7095c8ff5b256690", "rows": [ { "analysis": "the manipulated factor (SPEC 500 primary contrast)", @@ -15,9 +15,9 @@ "classification": "intentionally-different", "id": "mon1-02", "justification": "part of the bundled treatment; absence is a declared property, not missing data (MON-12)", - "monolithic": "src/aisle/harness/monolith.py#sha256:98c59d5f0887cf808888deadeb0712de37621a875a3e1d87dae0469478ddbc55", + "monolithic": "src/aisle/harness/monolith.py#sha256:a19225bef036fd797208c8c1f35cf08165aab8e0dc3ab36282172b737206bbf3", "surface": "static validation and diagnostics", - "typed": "src/aisle/harness/validate.py#sha256:55ab8640ce1dc85fac48ba5b21556cfaf1f1827c60cc2056abe12b51ae8d03e2" + "typed": "src/aisle/harness/validate.py#sha256:58e860b6b5cbfe66948cbfddae25c2de46e17001f2fce868689c4829d963a117" }, { "analysis": "treatment property", @@ -80,7 +80,7 @@ "justification": "none needed: same artifact", "monolithic": "src/aisle/nodes/monolith_broker.py#sha256:872741da1be8cfa0c89a4ee77ee4fb8a457f130a48228d39ebb8e09a84ee3d46", "surface": "trusted controller (bridge, reset, verifier, rollout client, turn barrier)", - "typed": "src/aisle/nodes/dora_genesis.py#sha256:b4f454c471e01bfee648a0a87806af50ea8d777fcf6052fdba9e0ccc3cff86a6" + "typed": "src/aisle/nodes/dora_genesis.py#sha256:1d2bfb3aaecf60e664649ed2ee9b09deded448080b7efcdc9b655af4c70fb3aa" }, { "analysis": "excluded from task authority (MON-4); no analysis treatment", @@ -114,9 +114,9 @@ "classification": "identical", "id": "mon1-13", "justification": "none needed: same artifact", - "monolithic": "src/aisle/harness/monolith.py#sha256:98c59d5f0887cf808888deadeb0712de37621a875a3e1d87dae0469478ddbc55", + "monolithic": "src/aisle/harness/monolith.py#sha256:a19225bef036fd797208c8c1f35cf08165aab8e0dc3ab36282172b737206bbf3", "surface": "episode budget, seeds, reset semantics, evaluator", - "typed": "src/aisle/harness/rollout.py#sha256:a201e4b2f66a90f600a0385de9706be4ce97149196b2f5fb63d98faac861be59" + "typed": "src/aisle/harness/rollout.py#sha256:486ceec3ee030b8ff84f96dde1b55f4b6a59a4eb49eed682d8f3896dc5edd8a0" }, { "analysis": "infrastructure-invalid records excluded in both arms; confinement status in the MON-8 tuple", diff --git a/docs/architecture.md b/docs/architecture.md index 8780ab17..70709e2d 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -37,6 +37,8 @@ ARTIFACTS git: dataflow YAML, skills/, manifests, evalcards EVIDENCE runs/: Arrow traces, videos, results, idea tree EXECUTION dora-rs runtime — typed topics, dynamic node add/remove WORLD Genesis physics scene (pharmacy desk; retail store) + optionally Nexus on the GPU (ADR-67) or rapier on the CPU + (ADR-68): same scenes, same bridge node ``` The agent never touches the simulator directly. It edits **artifacts** @@ -155,7 +157,7 @@ Safety is structural, not behavioral (H5): code can never be hot-swapped on a live dataflow (`harness swap` refuses, HAR-10). -## The world (Genesis) +## The world (Genesis, optionally Nexus) `src/aisle/scenes/` builds scenes from (seed, embodiment, tier) — a function, not a script, so reset/batching/randomization are all @@ -172,6 +174,20 @@ select task difficulty; the perception ladder (L0 oracle poses → L1 ground-truth segmentation → L2 full pixels) selects which pose source a graph may use. +The scene contract is engine neutral: the bridge only ever touches +`robot`, entity, link and camera objects through the small duck-typed +surface Genesis exposes, so another engine can realize the same scenes +behind it. **Genesis** is the default and the only engine behind the +measured record. **Nexus** (ADR-67, `src/aisle/sim/nexus_backend.py`) +realizes them on the GPU, and **rapier** (ADR-68, +`src/aisle/sim/rapier_backend.py`) on the CPU, rendering through the Nexus +viewer so the two share one camera model. `aisle.sim` dispatches +`build_scene` / `build_store` on `AISLE_SIM_ENGINE`, which +`harness rollout --sim-engine` injects into the bridge node and records in +the run manifest. The bridge node itself is shared, unchanged. Results are +not comparable across engines, and Nexus stepping determinism is not +established ([determinism](determinism.md)). + ## Verification and reset (frozen) The **oracle verifier** (SPEC 040) judges episodes from `oracle_state`: diff --git a/docs/benchmark/v1/release-audit.json b/docs/benchmark/v1/release-audit.json index 955cfca7..848d2398 100644 --- a/docs/benchmark/v1/release-audit.json +++ b/docs/benchmark/v1/release-audit.json @@ -184,7 +184,7 @@ "status": "passed" } ], - "manifest_sha256": "f088620a343c2c062543a94253187cc233250fca4a81a319b6e2bf8b9d60fdfe", + "manifest_sha256": "513617f4d2d27aa6cd85748dfa25cbee6e402ed1142605f2164f966632f58a57", "release_ready": false, "schema_version": "aisle.benchmark.release-audit.v1", "wording": "#357 stays open until every applicable criterion has reviewable evidence; internal dry runs never pass the external criteria" diff --git a/docs/benchmark/v1/version-manifest.json b/docs/benchmark/v1/version-manifest.json index 295f9109..9d95e7ea 100644 --- a/docs/benchmark/v1/version-manifest.json +++ b/docs/benchmark/v1/version-manifest.json @@ -1,6 +1,6 @@ { "benchmark_version": "aisle-benchmark-v1-draft", - "manifest_sha256": "f088620a343c2c062543a94253187cc233250fca4a81a319b6e2bf8b9d60fdfe", + "manifest_sha256": "513617f4d2d27aa6cd85748dfa25cbee6e402ed1142605f2164f966632f58a57", "missing_surfaces": [], "schema_version": "aisle.benchmark.version-manifest.v1", "status": "draft", @@ -79,7 +79,7 @@ }, "safety_boundary_validator": { "path": "src/aisle/harness/validate.py", - "sha256": "sha256:ed82967e800af6ec4d1f5336374343df38a5ba437771dd94e24b06c7dc64f481" + "sha256": "sha256:6324c0a684e4d56a8711ff7ed5851a23c1c6131f1040f3f6e80c2f896a2ac955" }, "scorer": { "path": "src/aisle/verifier", diff --git a/docs/decisions/ADR-67.md b/docs/decisions/ADR-67.md new file mode 100644 index 00000000..8761bf2a --- /dev/null +++ b/docs/decisions/ADR-67.md @@ -0,0 +1,183 @@ +# ADR-67 — A second physics engine (Nexus) behind the scene contract + +Status: PROPOSED — owner review required under CON-14; the spec-change PR it +owes (see below) must land before it can be accepted. + +## Scope: development-only engines + +Nexus, and rapier after it (ADR-68), are development engines. Their wheels +are built from dimforge branches pinned by commit in `engine-runtime.json`, +outside `pyproject.toml` and `uv.lock`, so no run on them can pass +`--env-baseline origin/main`: every such run is `--env-baseline local`, +recorded as unattested, with the engine build receipt as its only +provenance. No result from either engine enters the measured record until +all three of these hold: the engines are installed from released versions +inside the lock, a `spec-change` PR generalizes the Genesis wording of +SPEC 020 and SPEC 030 (CON-14), and the frozen baseline is re-established +under human review (CON-7). Whether that path is worth taking is a +maintainer decision this ADR does not make. + +## Context + +SPEC 020 names Genesis World as the simulator and SPEC 030 calls the bridge +`dora-genesis`, but the contracts themselves are engine neutral at the object +level: the bridge talks to `robot`, entity, link and camera objects through a +small duck-typed surface (`get_qpos`, `control_dofs_position`, `set_pos`, +`render(rgb, depth, segmentation)`, ...), and everything that decides a +scene (layout, seeded placements, DR draws, label textures, camera mounts) +is a pure function in the frozen `aisle.scenes` modules. + +We want to run the same experiments on Nexus (`dimforge/nexus`, a GPU +rigid-body engine with Python bindings) without disturbing the measured +Genesis record. + +## Decision + +1. **The engine is an explicit, attested choice.** `AISLE_SIM_ENGINE` + (`genesis` | `nexus`, default `genesis`) is declared on the bridge node + like the perception rung, injected by `harness rollout --sim-engine`, + and attested in `bridge_info` (`sim_engine`, with `genesis_version` + keeping its BRG-6 name and carrying whichever engine's version ran) and + in the run manifest (`sim_engine`). Unknown names are refused, never + defaulted (the TC-9 rule). +2. **The frozen set is untouched (CON-7).** `src/aisle/scenes/*` keeps the + Genesis builders byte-for-byte. The Nexus realization lives in + `src/aisle/sim/nexus_backend.py`, outside the fence, and REUSES the + frozen pure functions (`resolve_layout`, `sample_placements`, + `apply_occlusion`, `label_texture_image`, `wrist_mount_transform`, + `_assert_reachable`, the `SceneHandle`/`StoreHandle` dataclasses). Only + the object construction is re-implemented. A sim test pins the Nexus + placements to the frozen sampler's output. +3. **Same object surface, same conventions.** Nexus wrappers return numpy + arrays (Genesis returned tensors; `to_numpy` passes both), quaternions in + Genesis's (w, x, y, z) order, the same single-env/batched shape rules, + `segmentation_idx_dict` in the `link`-level shape TC-9 documents, and a + camera `transform` in the OpenGL look-at convention VER-8 pins, so the + verifier's calibration module needs no change. +4. **Rendering lives in the Nexus viewer, not the physics engine.** Sensor + cameras (offscreen surfaces with RGB, metric depth and per-body + segmentation passes, optionally attached to a link) were added to + `nexus_viewer3d`; the physics core only gained body/joint state read and + write access. +5. **Nexus-only constants** (Genesis-equivalent default PD gains for URDF + joints, the ground slab replacing the infinite plane, camera clip planes) + live in `src/aisle/sim/nexus_physics.toml`, outside the frozen set. +6. **The engine realization is attested by its own digest, not by widening + the fence.** `env_hash.sim_engine_hash(root, engine, build=None)` is one + sha256 over the engine name, `src/aisle/sim/**` (so nexus_physics.toml's + `substeps`, `internal_pgs_iterations`, `contact_natural_frequency` and + `friction_combine_rule` are inside it) and the engine build provenance. + It returns `{"engine", "sim_engine_hash", "n_files", "build"}` for the + run manifest, and `tools/env_hash.py --sim-engine ` emits the + same block as `report["sim"]`. Recorded, never a gate: the CON-7 verdict + stays the frozen set's. + Adding `src/aisle/sim` to `FROZEN_DIRS` was rejected. It moves the + Genesis `env_hash` from `1d83efde` (87 files) to `4cf1d9f9` (90), + i.e. it spends the attestation discontinuity that issue #283 reserves + for the composite `env_hash` with its bridging measurement (re-run M0 + and one tier curve at the new hash, show identity). Until then every + recorded Genesis manifest, `tools/env_hash.json` and every + `--env-baseline origin/main` run would have to be reissued for a change + that buys nothing on the Genesis side. +7. **The engine build is provenance, not a version string.** + `nexus_runtime.read_receipt(root=None)` is the public reader for + `.nexus-runtime-receipt.json`, returning + `{"installed", "path", "receipt", "problem"}` (`receipt` = wheel, + version, cargo feature, platform, and the nexus/rapier/kiss3d commits + with their dirty flags), never raising; `tools/nexus_runtime.py receipt` + is its CLI (exit 0 iff installed). The receipt is gitignored and local + to the machine that built the wheel, so the manifest's copy is the only + durable trace, and it is what `sim_engine_hash`'s `build` argument + takes. ADR-24's attested set is engine-aware to match: `ENGINE_DISTS` + adds the running engine's own distribution (`dimforge-nexus3d` for + Nexus) to the engine-neutral core, which previously named `genesis-world` + whatever engine ran. +8. **The realization cannot be hot-swapped.** `src/aisle/sim` joins + `harness/swap.py`'s `FROZEN_ROOTS` beside the bridge: unfenced code that + decides the physics would otherwise be the one way to change what a + running attested dataflow measures (HAR-10). + +## Consequences + +- Results across engines are NOT comparable: contact models, solver and + renderer differ. A Nexus run is a different environment; the frozen-set + hash does not encode the engine, so the manifest's `sim_engine` field and + decision 6's `sim_engine_hash` are the discriminators. The frozen set does + not encode the solver settings either, which is why the engine digest + exists. Before any Nexus result enters the measured record, + the frozen baseline must be re-established under human review (CON-7) and + SPEC 020 / SPEC 030 wording generalized by a `spec-change` PR (CON-14). +- The Genesis-fit constants in `physics.toml` (gripper gains, the SO-101 + kinematic carry latch, convex decomposition threshold) were not re-tuned; + the Nexus path inherits them. +- CON-5 on Nexus: the GPU broad phase and constraint coloring use atomics, + so bitwise run-to-run reproducibility is not established. `build_scene` + determinism (same seed, same initial state) holds; stepping determinism + must be measured before Nexus evidence is trusted. +- Fixed upstream while doing this, both consumed by Nexus through + `[patch.crates-io]` path overrides until releases carry them: + rapier3d-urdf composed URDF `rpy` as intrinsic XYZ Euler angles instead of + fixed-axis roll-pitch-yaw (`Rz * Ry * Rx`), misplacing every SO-101 link + below the shoulder (rapier branch `fix-urdf-rpy`, released in rapier 0.36.0, + which Nexus now takes from crates.io); and kiss3d replaced its + global mesh/texture/material managers whenever a second window or + offscreen surface was created, orphaning the material of objects built + before the sensor cameras existed, whose per-object uniform buffer then + grew until wgpu rejected the offsets (kiss3d branch + `fix-shared-window-managers`). +- Solver settings are the engine's, not the scene's + (`src/aisle/sim/nexus_physics.toml [sim]`). The first Nexus runs lost the + Franka pinch on lift, showed boxes creeping on their boards, and needed 16 + substeps with eight PGS iterations to come close. All of it traced to the + Nexus solver, fixed on the `aisle-backend` branch: contacts between a + multibody link and a rigid body were solved twice, once by the multibody + solver and once by the rigid-body solver against a zero-inverse-mass copy of + the link that never moved, so the fingers' friction was cancelled by a ghost + of themselves; the rigid-body sweeps ran one PGS iteration per substep + against eight for the multibody (now interleaved); contact warmstarting + matched every point of a small manifold to the first old point within 10 cm, + so a fingertip pad's four points all inherited one impulse and the solver + had to redistribute them every step (now nearest-point matching); friction + rows were only solved in the once-per-substep stabilization sweep (now + optionally in every biased iteration, `friction_in_bias_pass`); and Nexus + averaged the friction of a touching pair where Genesis takes the maximum + (`friction_combine_rule`). Replaying the recorded T0 commands + (`tools/nexus_grasp_replay.py`) the box now follows the hand with a tilt + under 3.2 degrees at 8, 4 or 2 substeps, and at Genesis's single substep + with 30 Hz contacts; resting boxes drift 0.001 mm in 3 s. AISLE runs 2 + substeps with 240 Hz contacts, 0.5 mm allowance and eight iterations, about + 3 ms of GPU per 10 ms tick on an M-series laptop. +- A Nexus run cannot pass `--env-baseline origin/main`. The wheel is + installed out of lock (`uv pip install` of a locally built archive), so + ADR-24's `uv sync --locked --check` fails and the trusted gate refuses + with DIST_DRIFT; with the attested set now engine-aware, the wheel's own + PEP 610 record (`archive_info` with no hash, a `file://` path) adds + `dimforge-nexus3d: archive install without a hash`. Nexus runs are + therefore `--env-baseline local`, recorded honestly as unattested, and + the receipt digest is the substitute provenance. Putting Nexus inside the + lock is the precondition for any attested Nexus run. +- Editing `tools/env_hash.py` at all makes the trusted mode refuse on a + branch until the change merges ("the gate tooling itself is not + trusted"), Genesis runs included. The engine attestation above therefore + lands as a normal env-change PR, not mid-campaign. +- `sim_engine_hash` covers the whole realization package, so a Nexus-only + edit also moves the Genesis digest. Over-inclusive by design: it may + report a change that did not affect the run, and can never report two + different physics as the same environment. +- The rollout manifest records `sim_engine_hash` together with the engine + build receipt. Because that receipt is the only source provenance for an + out-of-lock wheel, the gate refuses Nexus or rapier when a required solver + or renderer receipt is absent, malformed, or identifies dirty sources. +- Nexus reads generalized robot velocities back from its DoF state, and both + alternative backends re-base a fixed-root robot through the body buffer for + the SPEC 210 mobile embodiment. The published rapier 0.36.0 loader carries + the URDF fixed-axis `rpy` correction described above. + +## Alternatives rejected + +- Refactoring `pharmacy.py`/`store.py` into an engine-neutral builder: the + clean design, but it edits the frozen set and re-freezes the baseline for + a change with no measured benefit yet. +- Selecting the engine from ambient environment only: violates the + graph-attests-everything rule that the rung and the backend already + follow. diff --git a/docs/decisions/ADR-68.md b/docs/decisions/ADR-68.md new file mode 100644 index 00000000..9366295e --- /dev/null +++ b/docs/decisions/ADR-68.md @@ -0,0 +1,86 @@ +# ADR-68 — A CPU engine: rapier physics behind the Nexus renderer + +Status: PROPOSED — owner review required under CON-14, with ADR-67. +Trigger: ADR-67 left stepping determinism unestablished on the only +alternative engine. +Scope: a development-only engine, under the conditions ADR-67's scope +section sets for Nexus (pinned out-of-lock wheel, `--env-baseline local` +runs only, no measured-record results). + +## Context + +ADR-67 put a second physics engine behind the scene contract: Nexus, a GPU +rigid-body solver selected with `AISLE_SIM_ENGINE=nexus`. Genesis remains the +default and the only engine behind the measured record. + +Two engines gave AISLE a cross-check on physics but not on reproducibility. +Genesis reduces on Metal or CUDA, and Nexus colours constraints with GPU +atomics, so neither can promise a bitwise-identical step, which is what CON-5 +asks for. Nexus is rapier on the GPU: scenes are authored as rapier worlds and +baked into GPU buffers. That makes rapier on the CPU the natural third engine, +with the same contact model, the same MJCF and URDF loaders and the same joint +semantics, executing single-threaded where bitwise reproducibility is +achievable and where the physics tick is far cheaper. + +## Decision + +1. **A third engine name, attested like the others.** `AISLE_SIM_ENGINE=rapier` + joins `genesis` and `nexus` in `aisle.sim.ENGINES`; it is declared on the + bridge node, asserted by `harness rollout --sim-engine`, and recorded in the + run manifest with its own `sim_engine_hash` and build receipt. +2. **rapier owns the physics, Nexus owns the pixels.** The backend steps + `rapier3d.PhysicsWorld` on the CPU and writes the resulting poses into a + Nexus scene that is built but never stepped, then renders through the same + sensor cameras. Measured before the backend was written: writing a body pose + and a robot configuration with no solver step moves the RGB, the metric + depth and the segmentation exactly as a stepped scene does. + + The alternatives were rejected. rapier's own Panda3D testbed is windowed + only, with no offscreen buffer, no depth, no segmentation and no camera + intrinsics. A new offscreen renderer would mean calibrating a third sensor + model that VER-8 would then have to re-derive. + + The consequence is explicit: the rapier engine needs both wheels, and + `engine_available("rapier")` checks `rapier3d` and `nexus3d`. In exchange, a + rapier run and a Nexus run share the identical renderer, shadows, + antialiasing, segmentation ids and camera model, so a comparison between + them isolates the solver as the only variable. +3. **The realization lives outside the frozen set**, next to the Nexus one, and + rebuilds the same scenes from the frozen pure functions (layout, placements, + occlusion, label textures, wrist mount, reachability). Nothing under CON-7 + moves. `src/aisle/sim/**` is covered by the ADR-67 engine digest, so the + solver settings in `rapier_physics.toml` are attested even though the frozen + set does not contain them. +4. **Missing rapier Python bindings are added upstream**, on a branch in the + rapier checkout, never worked around downstream. The engine needed multibody + state writing (`apply_displacements`, `generalized_position`, the link motor + setters, armature), the MJCF name tables, an IK degree-of-freedom filter and + a counters toggle. + +## Consequences + +- Genesis stays the only engine behind the measured record. Three engines now + produce evidence that is not comparable across engines. +- `rapier_physics.toml` holds the engine-realization constants. rapier has no + counterpart to the Nexus scheduling knobs (`friction_in_bias_pass`, + `implicit_coriolis`, `substep_refresh`), and its substep analogue is the + solver iteration count. The camera and light blocks are copied from + `nexus_physics.toml` verbatim, because the renderer is the same one. +- Two out-of-lock wheels now have to be installed by hand, and any `uv sync` + removes both. +- The engine sources are pinned, not tracked. `engine-runtime.json` names the + repository, branch and full commit of the two repositories AISLE builds + wheels from, and `tools/engine_sources.py` fetches them by exact commit, the + same discipline `dora-runtime.json` applies to the Dora CLI. The crates the + engine merely links against are pinned once, in nexus's own Cargo manifest: + rapier by its published release, kiss3d by git revision rather than branch + until its fix is released: pinning them here as well would be two + sources of truth for one dependency, and a branch would make the build + irreproducible. The cost is that a new engine commit has to be pushed and + the pin bumped before a pinned install can use it; an unpushed pin fails + closed rather than building something else. `tools/rapier_runtime.py verify` reports the renderer's receipt + alongside the solver's so a half-installed environment is visible. +- A multibody is built with a 6-DoF free root that rapier collapses only during + the first `step()`, so the backend takes one warm-up step before reading the + degree-of-freedom layout. `Multibody::update_root_type` is `pub(crate)` + upstream; making it callable would remove the warm-up step. diff --git a/docs/determinism.md b/docs/determinism.md index 6422dbc3..796de18f 100644 --- a/docs/determinism.md +++ b/docs/determinism.md @@ -2,15 +2,22 @@ Contract: same (seed, cfg, platform) ⇒ bitwise-identical initial `oracle_state`. Verified by `tests/sim/test_scene.py::test_build_determinism` -on macOS arm64 (Metal backend, float32). +on macOS arm64 (Metal backend, float32), and on the optional Nexus engine +(ADR-67) by `tests/sim/test_nexus_scene.py::test_build_determinism`. Backend selection is explicit and recorded. `uv sync --extra sim` plus `harness rollout --sim-extra sim` uses Metal on Darwin and CPU elsewhere, even when a CUDA device is visible. `uv sync --extra cuda` plus `harness rollout --sim-extra cuda` uses CUDA on Linux and fails closed when the device is unavailable; it never silently retries on CPU. The selected -extra, Genesis backend, and device are persisted in `manifest.json` beside -the environment fingerprint. A pre-attestation development build executed on +extra, engine, resolved backend, and device are persisted in `manifest.json` +(`sim_extra`, `sim_engine`, `sim_backend`, `sim_device`) beside the +environment fingerprint. `sim_engine` is the discriminator between the two +engines' evidence: the frozen-set hash does not encode the engine. +On Nexus the same two extras resolve through that engine's own table +(`aisle.sim.select_nexus_backend`): `sim` is Metal on Darwin and WebGPU +elsewhere, `cuda` is Linux-only and fails closed the same way. +A pre-attestation development build executed on an NVIDIA GeForce RTX 5090 with driver 580.126.09 and PyTorch 2.13.0+cu130; that historical run is not evidence for the corrected attested path. A fresh hardware run is required before making a performance or reproducibility claim. @@ -23,6 +30,27 @@ Known platform caveats — recorded here rather than hidden (SCN-7): reduction order may diverge once physics steps run (relevant from T05 onward; measure before promising cross-platform reproducibility). - genesis is initialized once per process (backend fixed at first - build_scene call); mixing backends in one process is unsupported. + build_scene call); mixing backends in one process is unsupported. Nexus + does the same in `_ensure_nexus` (`src/aisle/sim/nexus_backend.py`): one + engine per process, and a later build asking for another backend is + refused. That engine also holds one live renderable scene: a newer + `build_scene` supersedes the previous scene's render nodes and frees its + cameras, so the older handle keeps its physics readbacks and camera poses + but its cameras raise on render. +- Nexus stepping determinism is not established (ADR-67). Only build + determinism is covered, by `tests/sim/test_nexus_scene.py`; the GPU broad + phase and constraint coloring use atomics, so run-to-run bitwise equality + after the first step is unmeasured. Nexus evidence must not be treated as + reproducible until it is measured, and Genesis remains the only engine + behind the measured record. +- The rapier engine (ADR-68) is the one with stepping determinism, and it is + measured: `tests/sim/test_rapier_scene.py::test_bitwise_step_determinism` + drops a box into contact in two identically seeded worlds, steps both 200 + times and requires bitwise identical `oracle_state` and joint coordinates. + It steps single-threaded on the CPU by configuration + (`num_threads = 1` in `src/aisle/sim/rapier_physics.toml`), which is what + removes the only ordering variation it has. Cross-platform bit equality is + a separate claim and is not measured; rapier's `determinism` cargo feature + (`tools/rapier_runtime.py install --determinism`) is what that would need. - CUDA startup errors propagate; AISLE never silently retries initialization on CPU. Metal-vs-CUDA post-step divergence has not yet been quantified. diff --git a/docs/generated/project-inventory.md b/docs/generated/project-inventory.md index c1499e02..36310448 100644 --- a/docs/generated/project-inventory.md +++ b/docs/generated/project-inventory.md @@ -34,7 +34,7 @@ stayed outside it). | `skills/` | registered agent-authored skills | — | | `skills_pending_review/` | recovered campaign skills, not registered | — | | `specs/` | numbered specs with MUST ids (000 = constitution) | — | -| `src/` | the aisle package: scenes, bridge, verifier, reset, harness, nodes | `src/aisle/embodiment.py`, `src/aisle/kinematics.py`, `src/aisle/mobility`, `src/aisle/nodes/budget_guard.py`, `src/aisle/reset`, `src/aisle/scenes`, `src/aisle/topics.py`, `src/aisle/turn_node.py`, `src/aisle/turns.py`, `src/aisle/verifier` | +| `src/` | the aisle package: scenes, bridge, sim engines, verifier, reset, harness, nodes | `src/aisle/embodiment.py`, `src/aisle/kinematics.py`, `src/aisle/mobility`, `src/aisle/nodes/budget_guard.py`, `src/aisle/reset`, `src/aisle/scenes`, `src/aisle/topics.py`, `src/aisle/turn_node.py`, `src/aisle/turns.py`, `src/aisle/verifier` | | `templates/` | scaffolding for new artifacts | — | | `tests/` | unit / sim / graph suites | — | | `tools/` | CI, env_hash, trace_check, campaign runners | — | @@ -46,39 +46,39 @@ stayed outside it). | Graphs | 27 | | Capability manifests | 43 | | CLI command entries | 39 | -| ADR files | 86 | +| ADR files | 88 | ## Graphs -| Graph | Scene/scenario | Embodiment | Perception | Nodes | -|---|---|---|---|---:| -| [graphs/agent_campaign.yaml](../../graphs/agent_campaign.yaml) | store / S3 | mobile | L0 (default) | 8 | -| [graphs/eval_hybrid_t1.yaml](../../graphs/eval_hybrid_t1.yaml) | pharmacy (default) / — | franka (default) | L1 | 11 | -| [graphs/eval_ik_transfer_v2.yaml](../../graphs/eval_ik_transfer_v2.yaml) | pharmacy (default) / — | franka (default) | L1 | 10 | -| [graphs/eval_s1_driver_v2.yaml](../../graphs/eval_s1_driver_v2.yaml) | store / S1 | mobile | L0 (default) | 10 | -| [graphs/eval_s3_driver_v1.yaml](../../graphs/eval_s3_driver_v1.yaml) | store / S3 | mobile | L0 (default) | 8 | -| [graphs/eval_t2_stack.yaml](../../graphs/eval_t2_stack.yaml) | pharmacy (default) / — | franka (default) | L2 | 11 | -| [graphs/eval_vla_smolvla_so101.yaml](../../graphs/eval_vla_smolvla_so101.yaml) | pharmacy (default) / — | so101 | L1 | 8 | -| [graphs/eval_vla_smolvla_so101_lockstep.yaml](../../graphs/eval_vla_smolvla_so101_lockstep.yaml) | pharmacy (default) / — | so101 | L1 | 8 | -| [graphs/eval_vla_smolvla_t1.yaml](../../graphs/eval_vla_smolvla_t1.yaml) | pharmacy (default) / — | franka (default) | L1 | 8 | -| [graphs/expert_s1.yaml](../../graphs/expert_s1.yaml) | store / S1 | mobile | L0 (default) | 10 | -| [graphs/expert_t0.yaml](../../graphs/expert_t0.yaml) | pharmacy (default) / — | franka (default) | L0 (default) | 10 | -| [graphs/expert_t1.yaml](../../graphs/expert_t1.yaml) | pharmacy (default) / — | franka (default) | L1 | 10 | -| [graphs/expert_t1_behavioral.yaml](../../graphs/expert_t1_behavioral.yaml) | pharmacy (default) / — | franka (default) | L1 | 10 | -| [graphs/expert_t1_l2.yaml](../../graphs/expert_t1_l2.yaml) | pharmacy (default) / — | franka (default) | L2 | 10 | -| [graphs/expert_t2.yaml](../../graphs/expert_t2.yaml) | pharmacy (default) / — | franka (default) | L2 | 11 | -| [graphs/expert_t3.yaml](../../graphs/expert_t3.yaml) | pharmacy (default) / — | franka (default) | L1 | 10 | -| [graphs/expert_t4.yaml](../../graphs/expert_t4.yaml) | pharmacy (default) / — | franka (default) | L1 | 11 | -| [graphs/expert_t4_inc2.yaml](../../graphs/expert_t4_inc2.yaml) | pharmacy (default) / — | franka (default) | L1 | 12 | -| [graphs/monolithic_t1.yaml](../../graphs/monolithic_t1.yaml) | pharmacy (default) / — | franka (default) | L1 | 7 | -| [graphs/monolithic_t1_l2.yaml](../../graphs/monolithic_t1_l2.yaml) | pharmacy (default) / — | franka (default) | L2 | 7 | -| [graphs/pilot_t1_l2_monolithic.yaml](../../graphs/pilot_t1_l2_monolithic.yaml) | pharmacy (default) / — | franka (default) | L2 | 8 | -| [graphs/pilot_t1_l2_typed.yaml](../../graphs/pilot_t1_l2_typed.yaml) | pharmacy (default) / — | franka (default) | L2 | 11 | -| [graphs/shield_t0_none_adversary.yaml](../../graphs/shield_t0_none_adversary.yaml) | pharmacy (default) / — | franka (default) | L0 (default) | 12 | -| [graphs/shield_t0_oracle.yaml](../../graphs/shield_t0_oracle.yaml) | pharmacy (default) / — | franka (default) | L0 (default) | 11 | -| [graphs/shield_t0_oracle_adversary.yaml](../../graphs/shield_t0_oracle_adversary.yaml) | pharmacy (default) / — | franka (default) | L0 (default) | 12 | -| [graphs/shield_t1_sensor.yaml](../../graphs/shield_t1_sensor.yaml) | pharmacy (default) / — | franka (default) | L1 | 11 | -| [graphs/shield_t1_sensor_adversary.yaml](../../graphs/shield_t1_sensor_adversary.yaml) | pharmacy (default) / — | franka (default) | L1 | 12 | +| Graph | Scene/scenario | Embodiment | Perception | Engine | Nodes | +|---|---|---|---|---|---:| +| [graphs/agent_campaign.yaml](../../graphs/agent_campaign.yaml) | store / S3 | mobile | L0 (default) | genesis (default) | 8 | +| [graphs/eval_hybrid_t1.yaml](../../graphs/eval_hybrid_t1.yaml) | pharmacy (default) / — | franka (default) | L1 | genesis (default) | 11 | +| [graphs/eval_ik_transfer_v2.yaml](../../graphs/eval_ik_transfer_v2.yaml) | pharmacy (default) / — | franka (default) | L1 | genesis (default) | 10 | +| [graphs/eval_s1_driver_v2.yaml](../../graphs/eval_s1_driver_v2.yaml) | store / S1 | mobile | L0 (default) | genesis (default) | 10 | +| [graphs/eval_s3_driver_v1.yaml](../../graphs/eval_s3_driver_v1.yaml) | store / S3 | mobile | L0 (default) | genesis (default) | 8 | +| [graphs/eval_t2_stack.yaml](../../graphs/eval_t2_stack.yaml) | pharmacy (default) / — | franka (default) | L2 | genesis (default) | 11 | +| [graphs/eval_vla_smolvla_so101.yaml](../../graphs/eval_vla_smolvla_so101.yaml) | pharmacy (default) / — | so101 | L1 | genesis (default) | 8 | +| [graphs/eval_vla_smolvla_so101_lockstep.yaml](../../graphs/eval_vla_smolvla_so101_lockstep.yaml) | pharmacy (default) / — | so101 | L1 | genesis (default) | 8 | +| [graphs/eval_vla_smolvla_t1.yaml](../../graphs/eval_vla_smolvla_t1.yaml) | pharmacy (default) / — | franka (default) | L1 | genesis (default) | 8 | +| [graphs/expert_s1.yaml](../../graphs/expert_s1.yaml) | store / S1 | mobile | L0 (default) | genesis (default) | 10 | +| [graphs/expert_t0.yaml](../../graphs/expert_t0.yaml) | pharmacy (default) / — | franka (default) | L0 (default) | genesis (default) | 10 | +| [graphs/expert_t1.yaml](../../graphs/expert_t1.yaml) | pharmacy (default) / — | franka (default) | L1 | genesis (default) | 10 | +| [graphs/expert_t1_behavioral.yaml](../../graphs/expert_t1_behavioral.yaml) | pharmacy (default) / — | franka (default) | L1 | genesis (default) | 10 | +| [graphs/expert_t1_l2.yaml](../../graphs/expert_t1_l2.yaml) | pharmacy (default) / — | franka (default) | L2 | genesis (default) | 10 | +| [graphs/expert_t2.yaml](../../graphs/expert_t2.yaml) | pharmacy (default) / — | franka (default) | L2 | genesis (default) | 11 | +| [graphs/expert_t3.yaml](../../graphs/expert_t3.yaml) | pharmacy (default) / — | franka (default) | L1 | genesis (default) | 10 | +| [graphs/expert_t4.yaml](../../graphs/expert_t4.yaml) | pharmacy (default) / — | franka (default) | L1 | genesis (default) | 11 | +| [graphs/expert_t4_inc2.yaml](../../graphs/expert_t4_inc2.yaml) | pharmacy (default) / — | franka (default) | L1 | genesis (default) | 12 | +| [graphs/monolithic_t1.yaml](../../graphs/monolithic_t1.yaml) | pharmacy (default) / — | franka (default) | L1 | genesis (default) | 7 | +| [graphs/monolithic_t1_l2.yaml](../../graphs/monolithic_t1_l2.yaml) | pharmacy (default) / — | franka (default) | L2 | genesis (default) | 7 | +| [graphs/pilot_t1_l2_monolithic.yaml](../../graphs/pilot_t1_l2_monolithic.yaml) | pharmacy (default) / — | franka (default) | L2 | genesis (default) | 8 | +| [graphs/pilot_t1_l2_typed.yaml](../../graphs/pilot_t1_l2_typed.yaml) | pharmacy (default) / — | franka (default) | L2 | genesis (default) | 11 | +| [graphs/shield_t0_none_adversary.yaml](../../graphs/shield_t0_none_adversary.yaml) | pharmacy (default) / — | franka (default) | L0 (default) | genesis (default) | 12 | +| [graphs/shield_t0_oracle.yaml](../../graphs/shield_t0_oracle.yaml) | pharmacy (default) / — | franka (default) | L0 (default) | genesis (default) | 11 | +| [graphs/shield_t0_oracle_adversary.yaml](../../graphs/shield_t0_oracle_adversary.yaml) | pharmacy (default) / — | franka (default) | L0 (default) | genesis (default) | 12 | +| [graphs/shield_t1_sensor.yaml](../../graphs/shield_t1_sensor.yaml) | pharmacy (default) / — | franka (default) | L1 | genesis (default) | 11 | +| [graphs/shield_t1_sensor_adversary.yaml](../../graphs/shield_t1_sensor_adversary.yaml) | pharmacy (default) / — | franka (default) | L1 | genesis (default) | 12 | ### Graph node membership @@ -169,10 +169,10 @@ stayed outside it). | `harness exposure ledger` | `--run`, `--campaign-id`, `--source-map`, `--output` | | `harness exposure wording` | `--output`, `--root` | | `harness fault assign` | `--bank`, `--seed-file`, `--block`, `--session`, `--cells`, `--output` | -| `harness fault calibrate` | `--bank`, `--clean-run`, `--clean-commit`, `--graph`, `--seeds`, `--tier`, `--embodiment`, `--perception`, `--staging`, `--raw-store`, `--campaign-id`, `--only`, `--output`, `--root` | +| `harness fault calibrate` | `--bank`, `--clean-run`, `--clean-commit`, `--graph`, `--seeds`, `--tier`, `--embodiment`, `--perception`, `--staging`, `--raw-store`, `--campaign-id`, `--only`, `--output`, `--root`, `--sim-engine` | | `harness fault leakage` | `--report`, `--probe`, `--output` | | `harness fault validate` | `--bank`, `--root`, `--tool-root` | -| `harness fleet` | `--graph`, `--agents`, `--episodes`, `--seeds`, `--out`, `--timeout-s`, `--root` | +| `harness fleet` | `--graph`, `--agents`, `--episodes`, `--seeds`, `--out`, `--timeout-s`, `--root`, `--sim-engine` | | `harness freeze build` | `--declaration`, `--output`, `--timestamp`, `--timestamp-source`, `--root` | | `harness freeze check` | `--manifest`, `--root`, `--allow-withheld-seeds` | | `harness hardware dry-run` | `--seed`, `--output` | @@ -181,16 +181,16 @@ stayed outside it). | `harness monolith describe` | `--embodiment` | | `harness monolith interface` | `--root`, `--candidate` | | `harness monolith parity` | `--typed`, `--monolithic`, `--output`, `--root`, `--candidate` | -| `harness monolith run` | `--module`, `--tier`, `--embodiment`, `--episodes`, `--seeds`, `--run-id`, `--timeout-s`, `--no-idea-gate`, `--root`, `--template`, `--verifier`, `--reset`, `--worker-config`, `--worker-config-sha256` | +| `harness monolith run` | `--module`, `--tier`, `--embodiment`, `--episodes`, `--seeds`, `--run-id`, `--timeout-s`, `--no-idea-gate`, `--root`, `--template`, `--verifier`, `--reset`, `--sim-engine`, `--worker-config`, `--worker-config-sha256` | | `harness monolith table` | `--write`, `--root`, `--candidate` | | `harness perception audit` | `--run`, `--envelope`, `--output` | | `harness probe` | `--dataflow`, `--topic`, `--for`, `--root` | | `harness report close` | `--id`, `--observed`, `--verdict`, `--root` | | `harness report log` | `--idea`, `--parent`, `--expect`, `--root` | -| `harness rollout` | `--graph`, `--tier`, `--embodiment`, `--episodes`, `--seeds`, `--reset`, `--verifier`, `--sim-extra`, `--root`, `--no-idea-gate`, `--perception`, `--run-id`, `--per-episode-wall-s`, `--env-baseline`, `--timeout-s` | +| `harness rollout` | `--graph`, `--tier`, `--embodiment`, `--episodes`, `--seeds`, `--reset`, `--verifier`, `--sim-extra`, `--sim-engine`, `--root`, `--no-idea-gate`, `--perception`, `--run-id`, `--per-episode-wall-s`, `--env-baseline`, `--timeout-s` | | `harness semantic corpus` | `--seed`, `--per-condition`, `--output` | | `harness semantic run` | `--corpus`, `--analysis-seed`, `--output` | -| `harness skill register` | ``, `--root`, `--sandbox`, `--run-id` | +| `harness skill register` | ``, `--root`, `--sandbox`, `--sim-engine`, `--run-id` | | `harness stats analyze` | `--protocol`, `--records`, `--output` | | `harness stats power` | `--protocol`, `--output` | | `harness stats validate` | `--protocol`, `--purpose`, `--output` | @@ -273,6 +273,8 @@ inference. | [docs/decisions/ADR-64.md](../decisions/ADR-64.md) | ADR-64 — a halt ends the episode: no in-episode resume | ACCEPTED 2026-09-10 (owner sign-off per CON-10, after merge of #563; | | [docs/decisions/ADR-65.md](../decisions/ADR-65.md) | ADR-65: Frontend conformance is a bound evidence gate | PROPOSED — engineering interpretation for #536; no study admission. | | [docs/decisions/ADR-66.md](../decisions/ADR-66.md) | ADR-66 — pilot first: a labelled pilot evidence tier and the execution order | ACCEPTED 2026-09-11 (owner sign-off per CON-10, triage table accepted | +| [docs/decisions/ADR-67.md](../decisions/ADR-67.md) | ADR-67 — A second physics engine (Nexus) behind the scene contract | PROPOSED — owner review required under CON-14; the spec-change PR it | +| [docs/decisions/ADR-68.md](../decisions/ADR-68.md) | ADR-68 — A CPU engine: rapier physics behind the Nexus renderer | PROPOSED — owner review required under CON-14, with ADR-67. | | [docs/decisions/ADR-7.md](../decisions/ADR-7.md) | ADR-7: T05 bridge interpretations and measured performance (SPEC 030) | ACCEPTED | | [docs/decisions/ADR-8.md](../decisions/ADR-8.md) | ADR-8: T06 verifier/reset interpretations (SPEC 040) | ACCEPTED | | [docs/decisions/ADR-9.md](../decisions/ADR-9.md) | ADR-9: T07 budget-guard interpretations (SPEC 080) | ACCEPTED | diff --git a/docs/getting-started.md b/docs/getting-started.md index 1a8ec374..3dbfc584 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -109,6 +109,32 @@ uv run --extra sim --locked harness rollout --graph graphs/expert_t0.yaml --tier overrides (both recorded in the run manifest). Research agents run without them: rollouts then require an open idea-tree entry (HAR-8) and a trusted frozen-set baseline (ADR-21). +- While the run waits on dora, a progress line goes to stderr every 15 s: + the phase (scene build before the first trace byte, then the running + episode against its wall budget), the last verdict, and the run deadline. + `AISLE_ROLLOUT_PROGRESS_S` changes the cadence; `0` silences it. Stdout + stays the single JSON report (CON-8). Once physics runs, the line also + carries the recent performance: wall time of the physics step call and of + the whole bridge tick (the step plus the state reads that block on the + GPU, which is the honest cost when the step call is an asynchronous + submit), the engine's GPU time per step when it reports one (Nexus does, + Genesis does not), render time per frame, and the real-time factor. +- The bridge writes those numbers per 100-step window to + `runs//sim_timing.jsonl`, and the manifest's `sim_timing` block + aggregates them over the run, so two engines can be compared on the same + graph and seeds. +- The per-episode wall clamp is the tier's budget (150 s for T0/T1) plus a + scene-build grace on the first episode of a launch. The grace is + engine-derived: 420 s for Genesis, which compiles kernels (9m30s for T0), + and 60 s for Nexus and rapier, which build a scene in seconds, so a wedged + Nexus episode clamps about a minute past its tier budget. + `--per-episode-wall-s N` overrides the tier budget. +- `AISLE_DEBUG_VIEW=side` (either engine) adds an operator camera after the + build, looking at the shelf front from the tray side, and + writes `runs//debug_view.mp4` at 10 fps, useful for seeing a grasp + slip in profile. `AISLE_DEBUG_VIEW=px,py,pz;lx,ly,lz` sets an explicit eye + and look-at in the base frame. The recorded traces and `overhead.mp4` are + unchanged: this camera is not a topic. - Results land in `runs//`: per-episode results JSON, Arrow traces, and videos. `runs/` is gitignored; every run is reproducible from (graph hash, env hash, seed list) (CON-5). @@ -124,6 +150,101 @@ work, and see `docs/troubleshooting.md` if runs behave strangely (leaked simulator processes from a previous killed run are the most common cause). +## 3b. Optional: run the scene on the Nexus engine (ADR-67) + +Genesis is the default and the only engine behind the measured record. The +graphs can also run on [Nexus](https://github.com/dimforge/nexus) (GPU +rigid bodies, Metal on macOS) for development: the bridge picks the engine +from `AISLE_SIM_ENGINE`, which `harness rollout --sim-engine nexus` injects +into the bridge node and records in the manifest. Results are not +comparable across engines. + +How far the Nexus path is actually exercised, as of today: + +- The **pharmacy desk** scene with the franka and so101 embodiments, the + **retail store** scene and the **mobile** embodiment are all covered by + `tests/sim/test_nexus_scene.py` (placements, IK and home pose, the + overhead/wrist passes, the realized wrist calibration, stepping, teleport + reset, re-basing, batched builds). +- The **L0 and L1** rungs run end to end: single-seed expert rollouts of + T0, T1 and T4 succeed, and `test_l1_estimate_matches_nexus_ground_truth` + pins the estimator against ground truth inside 1 mm. +- The **L2** rung does not work on Nexus. The open-vocabulary detector + refuses every frame (`identity margin -0.016 under the 0.01 floor`), so + expert_t1_l2 and expert_t2 fail with `never_grasped`. The cause is render + fidelity rather than physics: Nexus box pixels come back at 0.47 + saturation against a declared 0.73 albedo, and the frame is flatter than + Genesis's. +- **Nexus stepping determinism is not established** (ADR-67): only build + determinism is. See [determinism](determinism.md) before reading anything + reproducible into a Nexus run. + +Nexus is not part of the lock, so its Python module is built from source. +`engine-runtime.json` pins that source the way `dora-runtime.json` pins the +Dora CLI: repository, branch and full commit. Nothing else has to be checked +out first. + +```bash +uv run --no-sync python tools/nexus_runtime.py install +``` + +That fetches the pinned nexus commit into `.engine-sources/` (gitignored), +builds with `maturin` (`--features metal` on macOS), installs the wheel with +`uv pip`, and writes a receipt naming the commit it built. The rapier and +kiss3d crates the engine links against are NOT fetched here: nexus's own +Cargo manifest takes the published rapier 0.36.0 from crates.io and patches +kiss3d from git by revision, so cargo resolves and caches both, and +`engine-runtime.json` does not pin them a second time. + +Working across local checkouts instead? Pass `--nexus ../nexus`. Any path +argument switches the whole build to local sources, so you never get a +half-pinned, half-local mix. Then: + +```bash +uv run --extra sim --locked pytest -m sim tests/sim/test_nexus_scene.py +``` + +```bash +uv run --extra sim --locked harness rollout --graph graphs/expert_t0.yaml --tier T0 \ + --episodes 2 --seeds 0..1 --no-idea-gate --env-baseline local --sim-engine nexus +``` + +Plain `uv sync` removes the wheel (it is not in the lock); reinstall with the +command above. `AISLE_SIM_BACKEND` accepts `metal`, `webgpu`, `cuda` or `cpu` +for Nexus, matching the features the wheel was built with. + +The solver settings Nexus needs for the pick-and-place (substeps, contact +stiffness, PGS iterations) live in `src/aisle/sim/nexus_physics.toml`; +`tools/nexus_grasp_replay.py --run runs/` replays a recorded run's joint +and gripper commands into a fresh Nexus scene under overrides, which is how +those values were chosen and how a grasp regression is reproduced offline. + +## 3c. Optional: run the scene on the rapier CPU engine (ADR-68) + +The third engine steps the same scenes with +[rapier](https://github.com/dimforge/rapier) on the CPU and renders them +through the Nexus viewer, so a rapier run and a Nexus run differ only in the +solver. Select it with `harness rollout --sim-engine rapier`. + +It needs BOTH wheels: the Nexus one above for the renderer, and the rapier +Python bindings, which are also outside the lock and pinned in +`engine-runtime.json`: + +```bash +uv run --no-sync python tools/rapier_runtime.py install +``` + +`--rapier ../rapier` builds from a local checkout instead. + +`--no-sync` matters here: a syncing `uv run` would reinstall the locked +environment first and take the Nexus wheel back out, which is the renderer +this engine needs. + +`tools/rapier_runtime.py verify` reports the solver's receipt and the +renderer's together, so a half-installed environment is visible before a run +rather than at the first render. Its engine constants live in +`src/aisle/sim/rapier_physics.toml`, alongside the Nexus ones. + ## 4. Where to go next - `docs/physical-ai-primer.md` — the concepts behind the project diff --git a/docs/harness-guide.md b/docs/harness-guide.md index a21a3f85..1bb9f9fa 100644 --- a/docs/harness-guide.md +++ b/docs/harness-guide.md @@ -48,6 +48,28 @@ Flags you will actually use: `--tier` (T0..T4, S1..S3), `--embodiment `--env-baseline local` (trust the local frozen set) — are recorded in the run manifest; research agents run without them. +`--sim-extra {sim,cuda}` picks the attested dependency/backend selection. +`--sim-engine {genesis,nexus,rapier}` picks the physics engine realizing +the scene (ADR-67, ADR-68); it defaults to `genesis` and is recorded in the +manifest as `sim_engine`, beside a digest of the engine realization and the +wheel's build receipt. `nexus` (GPU) and `rapier` (CPU, rendering through +the Nexus viewer) refuse before launch when their wheels are not installed +(see [getting started](getting-started.md) §3b and §3c), and results are not +comparable across engines. A graph whose bridge node declares +`AISLE_SIM_ENGINE` owns the choice: passing a different `--sim-engine` is +refused rather than silently overridden. The same flag is on +`harness monolith run`, `harness fault calibrate`, `harness skill register` +and `harness fleet`. + +```bash +uv run harness rollout --graph graphs/expert_t0.yaml --tier T0 \ + --episodes 2 --seeds 0..1 --no-idea-gate --env-baseline local --sim-engine nexus +``` + +The scene-build wall grace added to the first episode of each launch is +engine-derived: 420 s for Genesis, which compiles kernels, and 60 s for the +other two, which build a scene in seconds. + ## traces — query recorded evidence ```bash diff --git a/docs/monolithic/treatment-table.md b/docs/monolithic/treatment-table.md index 448b941b..b0e10a36 100644 --- a/docs/monolithic/treatment-table.md +++ b/docs/monolithic/treatment-table.md @@ -5,16 +5,16 @@ Regenerate with `uv run harness monolith table --write`; CI checks it. | id | surface | class | typed arm | monolithic arm | justification | analysis treatment | |---|---|---|---|---|---|---| | mon1-01 | agent deliverable | intentionally-different | capability manifests + dora dataflow YAML + node modules of the agent region
`graphs/expert_t1.yaml` 303d08b37c6f
`registry/manifests/segmented-pose.yaml` db56206033d4
`registry/manifests/grasp-planner-topdown.yaml` fb4d4696b908
`registry/manifests/ik-trajectory.yaml` e2b498f5a47d
`registry/manifests/task-state-machine.yaml` 5af70b46a412
`src/aisle/nodes/segmented_pose.py` 78b748f4f0dc
`src/aisle/nodes/grasp_topdown.py` 3f24e0a56571
`src/aisle/nodes/ik_trajectory.py` 44e8186838d9
`src/aisle/nodes/task_state_machine.py` ad7fcc819c7b | one orchestration module against the primitive API
`experts/monolithic/expert_t1.py` 97ac76fcec0d
`docs/monolithic/allowlist.json` 975d05332e97 | this is the treatment: bundled typed-dataflow engineering surface versus one ordinary module | the manipulated factor (SPEC 500 primary contrast) | -| mon1-02 | static validation and diagnostics | intentionally-different | harness validate (VAL catalog: schema, topology, motion gating, turn plan)
`src/aisle/harness/validate.py` ed82967e800a | none; Python syntax/import/runtime errors only, via harness monolith check
`src/aisle/harness/monolith.py` a12428eab867 | part of the bundled treatment; absence is a declared property, not missing data (MON-12) | recorded as treatment property; check/run attempt and error events compared descriptively | +| mon1-02 | static validation and diagnostics | intentionally-different | harness validate (VAL catalog: schema, topology, motion gating, turn plan)
`src/aisle/harness/validate.py` 6324c0a684e4 | none; Python syntax/import/runtime errors only, via harness monolith check
`src/aisle/harness/monolith.py` d898a3734ade | part of the bundled treatment; absence is a declared property, not missing data (MON-12) | recorded as treatment property; check/run attempt and error events compared descriptively | | mon1-03 | registry search and resolution | intentionally-different | registry manifests, harness registry search/lint
`src/aisle/harness/registry.py` 214da7bb8db2 | absent; the primitive API is the only capability catalog
`docs/monolithic/primitive-api.md` f2c686ac4cb3 | part of the bundled treatment | treatment property | | mon1-04 | documentation given to the agent | intentionally-different | research-agent contract and the typed graph/manifest docs
`harness/CLAUDE.research.md` e736bbd6caa7 | primitive API document and module contract
`docs/monolithic/primitive-api.md` f2c686ac4cb3
`docs/monolithic/README.md` ebfc1733c9b8 | each arm documents its own representation; task, goals, budgets and evaluator text are shared | documentation hashes in the MON-8 treatment tuple; representation-specific instructions listed here rather than in prose | | mon1-05 | robot primitives (pose, grasp, trajectory, executor) | identical | called in-process by the typed nodes
`src/aisle/nodes/segmented_pose.py` 78b748f4f0dc
`src/aisle/nodes/grasp_topdown.py` 3f24e0a56571
`src/aisle/nodes/ik_trajectory.py` 44e8186838d9 | the same function objects re-exported by aisle.monolith.primitives
`src/aisle/monolith/primitives.py` e4b3ef279708 | none needed: same artifact | held fixed; pinned by tests/unit/test_monolith_surface.py (same objects) | | mon1-06 | semantic observations | representation-equivalent | dora edges into the agent region (bridge_info, seg_overhead, depth_overhead, joint_state, episode_goal, episode_result, violation, reset_done, 1 Hz tick)
`graphs/expert_t1.yaml` 303d08b37c6f | the same fields as on_event(event) dicts from the broker
`graphs/monolithic_t1.yaml` a43b46fb6f7c
`docs/monolithic/interface-map.json` 4dffcb8a0bcb | none needed: same semantic fields, different envelope | held fixed; exactness checked by harness monolith interface | | mon1-07 | actions and authority | representation-equivalent | joint_cmd / gripper_cmd edges into budget-guard, episode_feedback to rollout-client
`graphs/expert_t1.yaml` 303d08b37c6f | action dicts {joint_cmd}, {gripper_cmd}, {feedback} validated and published by the broker on the same edges
`src/aisle/nodes/monolith_broker.py` 6fb86d2fdcae | none needed: same semantic fields, different envelope | held fixed; same names, arity and privilege (interface map) | | mon1-08 | guard route and limits | identical | budget-guard node, harness/limits + budget.toml
`src/aisle/nodes/budget_guard.py` e15bc5f70b29
`harness/budget.toml` bcbb79aa146a | the same node consuming monolith-broker/joint_cmd and gripper_cmd
`graphs/monolithic_t1.yaml` a43b46fb6f7c | none needed: same artifact | held fixed (MON-5) | -| mon1-09 | trusted controller (bridge, reset, verifier, rollout client, turn barrier) | identical | as wired in expert_t1.yaml
`src/aisle/nodes/dora_genesis.py` 196837073251
`src/aisle/reset/service.py` 289a8c80374d
`src/aisle/verifier/oracle.py` c4ce22de0b88
`src/aisle/harness/rollout_client.py` 18808db0aa2d
`src/aisle/nodes/turn_barrier.py` fd075658ecc8 | the same nodes, same wiring, in monolithic_t1.yaml
`src/aisle/nodes/monolith_broker.py` 6fb86d2fdcae | none needed: same artifact | held fixed; graphs differ only inside the agent region (tests/graph/test_monolithic_graph.py) | +| mon1-09 | trusted controller (bridge, reset, verifier, rollout client, turn barrier) | identical | as wired in expert_t1.yaml
`src/aisle/nodes/dora_genesis.py` b4d5e6eab8e6
`src/aisle/reset/service.py` 289a8c80374d
`src/aisle/verifier/oracle.py` c4ce22de0b88
`src/aisle/harness/rollout_client.py` 18808db0aa2d
`src/aisle/nodes/turn_barrier.py` fd075658ecc8 | the same nodes, same wiring, in monolithic_t1.yaml
`src/aisle/nodes/monolith_broker.py` 6fb86d2fdcae | none needed: same artifact | held fixed; graphs differ only inside the agent region (tests/graph/test_monolithic_graph.py) | | mon1-10 | transport metadata | intentionally-different | nodes see dora metadata (sim_time_ns, seq, env_id, turn stamps, h/w/enc)
`src/aisle/topics.py` 431a0e0bee4c | the module sees sim_time_ns only; frames arrive reshaped
`src/aisle/nodes/monolith_broker.py` 6fb86d2fdcae | runtime envelope needed only by the typed transport; carries no task authority | excluded from task authority (MON-4); no analysis treatment | | mon1-11 | turn accounting | intentionally-different | four lockstep participants in the agent region
`graphs/turn_plans/expert_t1.json` 6f0c8997c748 | one participant (the broker)
`graphs/turn_plans/monolithic_t1.json` d268bcb32285 | runtime representation of the same synchronous step; ordering of the four typed nodes is fixed by the plan, so the merged participant observes the same data per turn | no task-outcome treatment; wall-time is reported per arm, not compared causally | | mon1-12 | in-episode retry budget | intentionally-different | AISLE_MAX_RETRIES=8 attested in the graph
`graphs/expert_t1.yaml` 303d08b37c6f | module-internal decision, bounded by the same episode timeout
`experts/monolithic/expert_t1.py` 97ac76fcec0d | the typed arm attests retries in YAML because that is where its decisions live; the episode budget (timeout, seeds) is held by the rollout runner in both arms | retries reported per episode in both arms (episode_feedback.retries) | -| mon1-13 | episode budget, seeds, reset semantics, evaluator | identical | harness rollout --graph graphs/expert_t1.yaml
`src/aisle/harness/rollout.py` 371789bd5093 | harness monolith run --module (calls the same rollout())
`src/aisle/harness/monolith.py` a12428eab867 | none needed: same artifact | held fixed | +| mon1-13 | episode budget, seeds, reset semantics, evaluator | identical | harness rollout --graph graphs/expert_t1.yaml
`src/aisle/harness/rollout.py` 568f98bb6593 | harness monolith run --module (calls the same rollout())
`src/aisle/harness/monolith.py` d898a3734ade | none needed: same artifact | held fixed | | mon1-14 | confinement | intentionally-different | issue #353 OS-level policy (pending)
`specs/460-actuation-threat-model.md` 27c3701afb0b | broker import/builtins hook + integrity check now; #353 policy pending
`src/aisle/monolith/confinement.py` f207d08bac41 | the module runs in the broker's process, so an in-process denial layer is needed in addition to #353 | infrastructure-invalid records excluded in both arms; confinement status in the MON-8 tuple | diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index cb43cbcc..7df8d0f2 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -14,6 +14,29 @@ uv sync --extra sim Anything that touches the simulator needs the extra; plain sync is only for pure-unit work. +That sync also removes the optional engine wheels, silently: neither +`nexus3d` nor `rapier3d` is in the +lock ([getting started](getting-started.md) §3b and §3c), so any `uv sync` +drops them and `--sim-engine nexus` then refuses at the `sim_engine` gate +with `simulation engine 'nexus' is not installed in this environment`. +Reinstall them from the commits `engine-runtime.json` pins: + +```bash +uv run --no-sync python tools/nexus_runtime.py install +``` + +```bash +uv run --no-sync python tools/rapier_runtime.py install +``` + +Keep `--no-sync` on those: a syncing `uv run` reinstalls the locked +environment first and takes the other engine wheel back out. + +A pinned install that fails with `upload-pack: not our ref` means the pinned +commit is not on the remote, usually because a local engine commit has not +been pushed yet. The fetch refuses rather than silently building something +else; push the branch, or pass a local checkout path to build from it. + ## Leaked simulator processes (the first thing to check) A timeout-killed or crashed `dora run` leaves orphaned node processes @@ -25,7 +48,7 @@ Before debugging ANY perf/timing weirdness: ```bash uptime # load average sane for an idle box? -ps aux | grep -E "dora|genesis" | grep -v grep +ps aux | grep -E "dora|genesis|nexus|rapier" | grep -v grep ``` Kill leftovers by their run working directory rather than pattern- @@ -97,6 +120,28 @@ The refusal JSON says why; the common ones: / cargo build during a run) contend for the GPU/CPU and corrupt timing. +## Nexus engine refusals (ADR-67, optional engine) + +- **`this nexus scene was superseded by a newer build_scene in this + process`**: the Nexus engine holds one live renderable scene per + process. A newer build takes the viewer's render nodes, and the older + handle's cameras raise from then on (its physics readbacks still work, + and the viewer frees those cameras, so a process can build any number of + scenes). + Build one scene per process, or read from the newest handle. The sim + tests build a fresh scene per test for exactly this reason. +- **`nexus already initialized with backend 'X'; build_scene requires + 'Y'`**: like `gs.init`, the backend is fixed at the first build and + cannot be mixed within a process. Set `AISLE_SIM_BACKEND` once + (`metal`, `webgpu`, `cuda` or `cpu`) and restart. +- **`the installed nexus3d was built without the metal feature`** (or + `cuda`): the wheel was built for another GPU API. Rebuild it with + `tools/nexus_runtime.py install`, which defaults the feature per host + (`metal` on macOS, `webgpu` elsewhere) and takes `--feature` to override. +- Nexus stepping determinism is not established, so a Nexus run that does + not replicate is not by itself a bug report; see + [determinism](determinism.md). + ## Nondeterminism (same seed, different result) Determinism is a spec requirement (CON-5) with a dedicated writeup: diff --git a/engine-runtime.json b/engine-runtime.json new file mode 100644 index 00000000..db92779f --- /dev/null +++ b/engine-runtime.json @@ -0,0 +1,18 @@ +{ + "schema_version": 1, + "comment": "ADR-67/ADR-68 engine sources: the two repositories AISLE builds wheels from. kiss3d is absent on purpose, and so are the rapier crates the engine links: nexus's own Cargo manifest patches kiss3d from git by rev and takes the published rapier 0.36.0 release from crates.io, so cargo resolves both and this file does not pin them twice.", + "sources": { + "nexus": { + "repository": "https://github.com/dimforge/nexus.git", + "branch": "aisle-backend", + "commit": "b1b3049992895593fe01afa35bba56d3d622afdc", + "builds": "the nexus3d wheel (the GPU engine and its viewer)" + }, + "rapier": { + "repository": "https://github.com/dimforge/rapier.git", + "branch": "python-multibody-control", + "commit": "2144f3e5c7f27f26d18ba8bef2bf8590ecf1aca8", + "builds": "the rapier3d wheel (bindings/python/rapier-py-3d), the CPU engine's bindings" + } + } +} diff --git a/src/aisle/harness/cli.py b/src/aisle/harness/cli.py index 499a56ee..67fb8067 100644 --- a/src/aisle/harness/cli.py +++ b/src/aisle/harness/cli.py @@ -39,6 +39,22 @@ def _git_sha(root: Path) -> str: ).stdout.strip() +def _add_sim_engine_flag(parser: argparse.ArgumentParser) -> None: + """The one `--sim-engine` definition (ADR-67) every sim-launching + subcommand shares, so no entry point can drift to another engine set or + silently force genesis. `None` asserts nothing: the graph's declaration + wins, and an undeclared graph runs the default engine.""" + from aisle.sim import ENGINES + + parser.add_argument( + "--sim-engine", + default=None, + choices=list(ENGINES), + help="physics engine realizing the scene (ADR-67, ADR-68); genesis is " + "the default; omit to honour the engine the graph's bridge declares", + ) + + def build_parser() -> argparse.ArgumentParser: """The full CLI surface (CON-8). Exposed so the research contract's copy-paste examples are TESTED against the real argparse tree (T17): @@ -75,6 +91,7 @@ def build_parser() -> argparse.ArgumentParser: choices=["sim", "cuda"], help="attested dependency/backend selection: portable sim or Linux CUDA", ) + _add_sim_engine_flag(roll) roll.add_argument("--root", type=Path, default=DEFAULT_ROOT) roll.add_argument( "--no-idea-gate", @@ -143,6 +160,7 @@ def build_parser() -> argparse.ArgumentParser: "no quality claim, never safety_class motion, never counted as a " "library skill. Promote by re-registering without this flag.", ) + _add_sim_engine_flag(skr) sw = subparsers.add_parser("swap", help="hot-swap a node on a live dataflow (HAR-10)") sw.add_argument("--graph", type=Path, required=True) @@ -160,6 +178,7 @@ def build_parser() -> argparse.ArgumentParser: fl.add_argument("--out", type=Path, default=None, help="output dir (default runs/fleet-)") fl.add_argument("--timeout-s", type=float, default=1200.0) fl.add_argument("--root", type=Path, default=DEFAULT_ROOT) + _add_sim_engine_flag(fl) pr = subparsers.add_parser("probe", help="attach a temporary topic inspector (HAR-11)") pr.add_argument("--dataflow", required=True) @@ -295,6 +314,7 @@ def build_parser() -> argparse.ArgumentParser: ) mono_run.add_argument("--verifier", default="oracle", choices=["oracle", "realistic"]) mono_run.add_argument("--reset", default="teleport", choices=["teleport", "behavioral"]) + _add_sim_engine_flag(mono_run) mono_check = monolith_sub.add_parser("check", help="compile/construct the module; no sim") mono_check.add_argument("--module", type=Path, required=True) mono_check.add_argument("--embodiment", default="franka", choices=["franka", "so101"]) @@ -353,6 +373,7 @@ def build_parser() -> argparse.ArgumentParser: fault_calibrate.add_argument("--only", default=None, help="comma list of opaque ids") fault_calibrate.add_argument("--output", type=Path, default=None) fault_calibrate.add_argument("--root", type=Path, default=DEFAULT_ROOT) + _add_sim_engine_flag(fault_calibrate) fault_leakage = fault_sub.add_parser("leakage", help="FLT-8 sham-vs-fault leakage probe") fault_leakage.add_argument("--report", type=Path, required=True, help="calibration report") fault_leakage.add_argument("--probe", type=Path, required=True, help="frozen probe declaration") @@ -461,6 +482,7 @@ def main() -> int: template=mono.TEMPLATE_GRAPH if args.template is None else args.template, verifier=args.verifier, reset_mode=args.reset, + sim_engine=args.sim_engine, ) elif args.monolith_command == "check": from aisle.monolith.worker_config import configured_worker_factory @@ -852,6 +874,7 @@ def main() -> int: perception=args.perception, sim_extra=args.sim_extra, per_episode_wall_s=args.per_episode_wall_s, + sim_engine=args.sim_engine, ) return emit_report(report, lambda level, e: f"rollout {level}: {e}") @@ -875,15 +898,39 @@ def main() -> int: import time as time_module from aisle.harness.fleet import run_fleet - from aisle.harness.rollout import parse_seed_range, scrub_bringup_env + from aisle.harness.rollout import ( + engine_check, + parse_seed_range, + resolve_sim_identity, + scrub_bringup_env, + ) from aisle.harness.validate import validate as validate_graph + graph_path = args.graph if args.graph.is_absolute() else args.root / args.graph out_dir = args.out or (args.root / "runs" / f"fleet-{int(time_module.time())}") + # ADR-67: the engine is the RUNNER's choice, not the shell's, and it + # rides the graph as it does for rollout: a bridge's own + # AISLE_SIM_ENGINE would win over this process's env, so a conflict + # with the flag is refused, as is an engine that is not installed. + # The scrub below strips both variables; they are re-applied from here. + engine_gate = engine_check(args.root, graph_path, args.sim_engine) + identity = ( + resolve_sim_identity("sim", engine_gate["engine"]) if engine_gate["ok"] else engine_gate + ) + if not identity["ok"]: + print(json.dumps({"ok": False, "refused": identity})) + return 1 + sim_engine = engine_gate["engine"] + sim_backend = identity["sim_backend"] procs = [] def launch(graph_path): - env = {**scrub_bringup_env(dict(os.environ))} + env = { + **scrub_bringup_env(dict(os.environ)), + "AISLE_SIM_ENGINE": sim_engine, + "AISLE_SIM_BACKEND": sim_backend, + } proc = subprocess.Popen( ["dora", "run", str(graph_path), "--uv"], cwd=args.root, @@ -896,7 +943,7 @@ def launch(graph_path): return proc.poll report = run_fleet( - args.graph if args.graph.is_absolute() else args.root / args.graph, + graph_path, args.agents, args.episodes, parse_seed_range(args.seeds), @@ -904,6 +951,9 @@ def launch(graph_path): args.timeout_s, launch, root=args.root, + sim_engine=sim_engine, + sim_backend=sim_backend, + sim_device=identity["sim_device"], ) # validate the STAMPED graph and attach the verdict (VAL gates) stamped = validate_graph(Path(report["graph"]), args.root, "franka", False) @@ -937,6 +987,7 @@ def launch(graph_path): now=datetime.date.today().isoformat(), run_id=args.run_id, sandbox=args.sandbox, + sim_engine=args.sim_engine, ) except RegistrationError as refused: print(json.dumps({"ok": False, "error": str(refused)})) diff --git a/src/aisle/harness/fault_calibration.py b/src/aisle/harness/fault_calibration.py index 41d06b18..38d1d3ec 100644 --- a/src/aisle/harness/fault_calibration.py +++ b/src/aisle/harness/fault_calibration.py @@ -206,10 +206,15 @@ def run_rung( perception: str, run_id: str, env: dict | None = None, + sim_engine: str | None = None, ) -> dict[str, Any]: """Materialize into a fresh git worktree at the clean commit and run the rollout there with --root so validation resolves the staged sources. - Returns the receipt, the rollout result, and the episode rows.""" + Returns the receipt, the rollout result, and the episode rows. + + sim_engine (ADR-67) is forwarded to the rollout so a calibration ladder + runs the engine the campaign selected, not whichever the runner defaults + to; None leaves the graph's own declaration in charge.""" subprocess.run( ["git", "worktree", "add", "-q", "--detach", str(worktree), clean_commit], cwd=root, @@ -244,6 +249,10 @@ def run_rung( "--run-id", run_id, ] + if sim_engine is not None: + from aisle.sim import normalize_engine + + cmd += ["--sim-engine", normalize_engine(sim_engine)] proc = subprocess.run( cmd, cwd=root, capture_output=True, text=True, env={**os.environ, **(env or {})} ) @@ -329,6 +338,7 @@ def calibrate_bank(args, bank: dict) -> dict[str, Any]: perception=args.perception, run_id=run_id, env=env, + sim_engine=getattr(args, "sim_engine", None), ) raw_dest = Path(args.raw_store) / run_id if Path(outcome["run_dir"]).exists(): diff --git a/src/aisle/harness/fleet.py b/src/aisle/harness/fleet.py index 663958e1..0c497196 100644 --- a/src/aisle/harness/fleet.py +++ b/src/aisle/harness/fleet.py @@ -164,10 +164,17 @@ def run_fleet( timeout_s: float, launch, root: Path | None = None, + sim_engine: str | None = None, + sim_backend: str | None = None, + sim_device: str | None = None, ) -> dict: """Generate, launch (via the injected `launch(graph_path, env)`), wait for every agent's results, aggregate. `launch` returns a poll() - callable; injection keeps this pure enough to unit-test.""" + callable; injection keeps this pure enough to unit-test. + + sim_engine/sim_backend/sim_device are recorded in the report (ADR-67, + CON-5): the caller owns the child env, so the report is where a fleet run + says which physics it measured.""" import yaml doc = yaml.safe_load(graph.read_text()) @@ -222,4 +229,7 @@ def run_fleet( report = aggregate(results_paths) report["graph"] = str(graph_path) report["wall_s"] = round(time.monotonic() - started, 1) + report["sim_engine"] = sim_engine + report["sim_backend"] = sim_backend + report["sim_device"] = sim_device return report diff --git a/src/aisle/harness/monolith.py b/src/aisle/harness/monolith.py index 0a6450f1..eee85149 100644 --- a/src/aisle/harness/monolith.py +++ b/src/aisle/harness/monolith.py @@ -141,8 +141,12 @@ def run( template: str = TEMPLATE_GRAPH, verifier: str = "oracle", reset_mode: str = "teleport", + sim_engine: str | None = None, ) -> dict: - """Stamp the admitted monolithic template and roll it out through the trusted runner.""" + """Stamp the admitted monolithic template and roll it out through the trusted runner. + + sim_engine (ADR-67) reaches the runner unchanged: the launcher must not + force genesis on a module the operator asked to run on another engine.""" if tier != "T1": return { "ok": False, @@ -209,6 +213,7 @@ def run( timeout_s=timeout_s, embodiment=embodiment, **({"record_simulator_work": True} if record_simulator_work else {}), + sim_engine=sim_engine, ) return {**report, "module": pre, "campaign_purpose": CAMPAIGN_PURPOSE} diff --git a/src/aisle/harness/rollout.py b/src/aisle/harness/rollout.py index e4569e18..68a152c0 100644 --- a/src/aisle/harness/rollout.py +++ b/src/aisle/harness/rollout.py @@ -38,9 +38,20 @@ # traces; post-data = a dead bridge freezes the stream) EPISODE_TIMEOUT_S = 60 GENESIS_BUILD_BUDGET_S = 420 +# ADR-67: Nexus assembles the same scene in seconds (no kernel compilation), +# so a wedged first episode must clamp in a minute rather than seven. Sized +# well above the measured build to cover wheel import and GPU adapter start. +NEXUS_BUILD_BUDGET_S = 60 PER_EPISODE_BUDGET_S = 150 PRE_DATA_STALL_S = 600 +# The pre-data grace exists because the scene build emits no traces; on Nexus +# that window is seconds, so two minutes already covers a slow adapter start. +NEXUS_PRE_DATA_STALL_S = 120 STALL_S = 180 +# Operator progress cadence on stderr while a run waits on dora (HAR-1). CON-8 +# keeps stdout JSON-only, so this never touches stdout. AISLE_ROLLOUT_PROGRESS_S +# overrides it; 0 disables. +PROGRESS_INTERVAL_S = 15.0 # retail tiers (RS-6, ADR-18): store-sim rtf ~0.1 on the dev machine — the # fixed-seed S1 episode runs ~101.5 sim s / ~25 wall min plus a ~2.5 min # store build (first green run: 28:39 total). The desk budgets above would @@ -68,6 +79,22 @@ def tier_budgets(tier: str) -> tuple[int, int]: return EPISODE_TIMEOUT_S, PER_EPISODE_BUDGET_S +def build_budget_s(engine: str) -> int: + """The default scene-build wall grace for an engine (ADR-67, HAR-1). + + Genesis compiles kernels for minutes, which is what the 420 s default was + sized for; Nexus builds the same scene in seconds, and rapier builds the + same Nexus scene to render through (ADR-68).""" + return NEXUS_BUILD_BUDGET_S if engine in ("nexus", "rapier") else GENESIS_BUILD_BUDGET_S + + +def pre_data_stall_s(engine: str) -> int: + """How long a launch may produce no traces at all before it counts as + stalled (ADR-67). Same reasoning as `build_budget_s`: the silent window is + the scene build, so it is engine-sized rather than Genesis-sized.""" + return NEXUS_PRE_DATA_STALL_S if engine in ("nexus", "rapier") else PRE_DATA_STALL_S + + # A7 wall-budget sizing (issue #160 item 6): in `--verifier realistic` # nothing ends an episode early — the reset/goal are downstream of the # verifier's own verdict, which it renders at sim-budget expiry — so EVERY @@ -151,17 +178,221 @@ def compute_metrics(episodes: list[dict]) -> dict: } -def resolve_sim_identity(sim_extra: str) -> dict: +def progress_interval_s(env: dict | None = None) -> float: + """The stderr progress cadence: AISLE_ROLLOUT_PROGRESS_S (seconds, 0 or a + negative value disables), else PROGRESS_INTERVAL_S. Malformed values are + refused loudly rather than silently muting the report.""" + raw = (os.environ if env is None else env).get("AISLE_ROLLOUT_PROGRESS_S") + if raw is None or not str(raw).strip(): + return PROGRESS_INTERVAL_S + try: + return float(raw) + except ValueError as exc: + raise ValueError( + f"AISLE_ROLLOUT_PROGRESS_S must be a number of seconds, got {raw!r}" + ) from exc + + +def _hms(seconds: float) -> str: + seconds = max(0, int(seconds)) + if seconds >= 3600: + return f"{seconds // 3600}h{(seconds % 3600) // 60:02d}m" + if seconds >= 60: + return f"{seconds // 60}m{seconds % 60:02d}s" + return f"{seconds}s" + + +def progress_report( + *, + now: float, + started: float, + deadline: float, + lines: int, + episodes: int, + seeds: list[int], + last_line_t: float, + episode_grace_s: float, + traces_size: int, + last_record: dict | None, + relaunches: int = 0, + timing: dict | None = None, +) -> str: + """One human-readable progress line for stderr while the run waits on + dora (HAR-1 operator feedback). Before the first trace byte the scene is + still building; afterwards it names the running episode, how long it has + run against its per-episode wall budget, the last verdict, and the run + deadline, so an operator can tell a long build from a wedged episode + without reading the traces.""" + elapsed = now - started + head = f"[rollout +{_hms(elapsed)}]" + if lines >= episodes: + phase = f"all {episodes} episodes recorded, waiting for the client to exit" + else: + seed = seeds[lines] if lines < len(seeds) else None + seed_txt = f" (seed {seed})" if seed is not None else "" + running = now - last_line_t + if traces_size <= 0: + phase = ( + f"episode {lines + 1}/{episodes}{seed_txt}: building the scene, " + f"no traces yet ({_hms(running)})" + ) + else: + phase = ( + f"episode {lines + 1}/{episodes}{seed_txt} running {_hms(running)} " + f"of {_hms(episode_grace_s)} wall budget, traces {traces_size / 1e6:.1f} MB" + ) + parts = [head, phase] + if last_record: + status = last_record.get("status", "?") + failure = last_record.get("failure") + verdict = status if not failure else f"{status} ({failure})" + t_end = last_record.get("t_end") + t_txt = f" at t={float(t_end):.1f}s" if isinstance(t_end, (int, float)) else "" + episode, seed = last_record.get("episode", "?"), last_record.get("seed", "?") + parts.append(f"last: episode {episode} seed {seed} {verdict}{t_txt}") + if relaunches: + parts.append(f"relaunches {relaunches}") + phrase = timing_phrase(timing or {}) + if phrase: + parts.append(phrase) + parts.append(f"run deadline in {_hms(deadline - now)}") + return f"{parts[0]} " + "; ".join(parts[1:]) + + +def summarize_timing(rows: list[dict]) -> dict: + """Aggregate the bridge's timing sidecar (`sim_timing.jsonl`, one row per + window of steps) into run-level means, weighted by step and frame counts: + physics wall time per step, engine GPU time per step where the engine + reports it, render wall time per frame, and the real-time factor (sim + seconds per wall second of bridge ticks). A row's tick already spans the + step, the renders and the publishes, so render time is added only for + older rows that carry no tick. Empty when there is nothing to aggregate.""" + steps = sum(int(r.get("steps", 0)) for r in rows) + frames = sum(int(r.get("frames", 0)) for r in rows) + if steps <= 0: + return {} + step_wall = sum(float(r.get("step_ms_mean") or 0.0) * int(r.get("steps", 0)) for r in rows) + tick_wall = sum( + float(r.get("tick_ms_mean") or r.get("step_ms_mean") or 0.0) * int(r.get("steps", 0)) + for r in rows + ) + gpu_rows = [r for r in rows if r.get("gpu_ms_mean") is not None] + gpu_steps = sum(int(r.get("steps", 0)) for r in gpu_rows) + gpu_wall = sum(float(r["gpu_ms_mean"]) * int(r.get("steps", 0)) for r in gpu_rows) + render_wall = sum(float(r.get("render_ms_mean") or 0.0) * int(r.get("frames", 0)) for r in rows) + sim_s = sum(float(r.get("sim_window_s") or 0.0) for r in rows) + wall_s = ( + sum( + float(r["tick_ms_mean"]) * int(r.get("steps", 0)) + if r.get("tick_ms_mean") is not None + else float(r.get("step_ms_mean") or 0.0) * int(r.get("steps", 0)) + + float(r.get("render_ms_mean") or 0.0) * int(r.get("frames", 0)) + for r in rows + ) + / 1000.0 + ) + out = { + "engine": rows[-1].get("engine"), + "steps": steps, + "frames": frames, + "step_ms_mean": step_wall / steps, + "step_ms_max": max(float(r.get("step_ms_max") or 0.0) for r in rows), + "tick_ms_mean": tick_wall / steps, + "render_ms_mean": (render_wall / frames) if frames else None, + "gpu_ms_mean": (gpu_wall / gpu_steps) if gpu_steps else None, + "rtf": (sim_s / wall_s) if wall_s > 0 else None, + } + return out + + +def read_timing_rows(run_dir: Path) -> list[dict]: + """The bridge's timing sidecar rows under `run_dir` (every launch's), or + an empty list when the bridge wrote none.""" + rows: list[dict] = [] + for path in sorted(run_dir.rglob("sim_timing.jsonl")): + for line in path.read_text().splitlines(): + line = line.strip() + if not line: + continue + try: + rows.append(json.loads(line)) + except json.JSONDecodeError: + continue + return rows + + +def timing_phrase(summary: dict) -> str: + """The performance clause of a progress line: wall time of the physics + step call and of the whole tick (step plus the state reads that sync the + GPU), GPU time per step when the engine reports it, render time per frame, + and the real-time factor.""" + if not summary: + return "" + parts = [f"physics {summary['step_ms_mean']:.1f} ms/step"] + if summary.get("tick_ms_mean") is not None: + parts.append(f"tick {summary['tick_ms_mean']:.1f} ms") + if summary.get("gpu_ms_mean") is not None: + parts.append(f"gpu {summary['gpu_ms_mean']:.1f} ms") + if summary.get("render_ms_mean") is not None: + parts.append(f"render {summary['render_ms_mean']:.1f} ms/frame") + if summary.get("rtf") is not None: + parts.append(f"rtf {summary['rtf']:.2f}x") + return ", ".join(parts) + + +def _last_record(raw: bytes) -> dict | None: + """The last parseable episode row of a results file, or None.""" + for line in reversed(raw.splitlines()): + line = line.strip() + if not line: + continue + try: + return json.loads(line) + except json.JSONDecodeError: + return None + return None + + +def sim_device_for(engine: str, backend: str) -> str: + """The live device an (engine, backend) pair runs on (CON-5, ADR-67). + + Derived from the resolved pair rather than from the host OS: Genesis's + Metal backend runs through torch MPS, while Nexus names its GPU adapter by + the backend itself, so a Nexus webgpu or metal run is never attested + ``cpu`` just because the host is not a Mac.""" + if backend == "cpu": + return "cpu" + if engine == "genesis" and backend == "metal": + return "mps" + return backend + + +def resolve_sim_identity(sim_extra: str, sim_engine: str = "genesis") -> dict: """Resolve the requested lock extra to a fail-closed backend/device. CON-5 requires this live hardware fact to ride with the run identity; the portable ``sim`` selection deliberately never probes into CUDA. + sim_engine (ADR-67) picks whose backend table answers; an engine whose + package is not installed refuses here, before anything launches. """ - from aisle.scenes.pharmacy import select_genesis_backend + from aisle.sim import engine_available, normalize_engine, select_sim_backend + + try: + sim_engine = normalize_engine(sim_engine) + except ValueError as exc: + return {"ok": False, "gate": "sim_engine", "detail": str(exc)} + if not engine_available(sim_engine): + return { + "ok": False, + "gate": "sim_engine", + "detail": f"simulation engine {sim_engine!r} is not installed in this environment", + } system = platform_module.system() cuda_available = False - device = "mps" if system == "Darwin" else "cpu" + # only the CUDA probe names a concrete device; every other pair is derived + # from the resolved backend below + device = None if sim_extra == "cuda": if system != "Linux": return { @@ -183,14 +414,15 @@ def resolve_sim_identity(sim_extra: str) -> dict: "detail": f"cannot inspect the requested CUDA device: {exc}", } try: - backend = select_genesis_backend(sim_extra, system, cuda_available) + backend = select_sim_backend(sim_engine, sim_extra, system, cuda_available) except ValueError as exc: return {"ok": False, "gate": "sim_backend", "detail": str(exc)} return { "ok": True, "sim_extra": sim_extra, + "sim_engine": sim_engine, "sim_backend": backend, - "sim_device": device, + "sim_device": device or sim_device_for(sim_engine, backend), } @@ -398,6 +630,7 @@ def run_gates( sim_extra: str = "sim", graph_snapshot: bytes | None = None, typed_stage: tuple[Path, dict] | None = None, + sim_engine: str = "genesis", ) -> dict: """HAR-2: refuse on env-hash mismatch (TRUSTED baseline by default, ADR-21: the baseline commit is fetched from the remote SERVER and @@ -418,7 +651,7 @@ def run_gates( "'origin/main', a full campaign-pinned main-history OID, or the " "logged dev override 'local' are accepted (ADR-21, issue #91)", } - sim_identity = resolve_sim_identity(sim_extra) + sim_identity = resolve_sim_identity(sim_extra, sim_engine) if not sim_identity["ok"]: return sim_identity baseline_oid = None @@ -426,6 +659,9 @@ def run_gates( # ADR-24: rollouts need the sim extra — declare the selection so the # trusted checker attests THIS environment shape (HAR-2) hash_cmd += ["--extras", sim_extra] + # ADR-67: the frozen set is engine neutral, so the engine digest and the + # engine's own build provenance ride beside env_hash rather than in it + hash_cmd += ["--sim-engine", sim_identity["sim_engine"]] if env_baseline != "local": baseline_oid, err = resolve_trusted_baseline(root, env_baseline) if err: @@ -529,6 +765,10 @@ def run_gates( "env_attested": bool((dist or {}).get("attested")), "dist_problems": (dist or {}).get("problems") or [], "dist_inventory": (dist or {}).get("inventory"), + # ADR-67: {"engine", "sim_engine_hash", "n_files", "build"} — the + # realization package plus the out-of-lock engine's build receipt, + # which is the only trace of which engine sources produced the run + "sim_engine_build": hash_report.get("sim"), "budget": remaining, } if typed_stage is not None: @@ -625,6 +865,13 @@ def realistic_verifier_node(root: Path, run_dir: Path, doc: dict, timeout_s: flo } +class EngineConflict(RuntimeError): + """The graph declares a physics engine the run asked to override (ADR-67). + + Its own type so the refusal reports the `sim_engine` gate rather than the + `perception` one the other instrumentation failure carries.""" + + def instrumented_graph( graph: Path, root: Path, @@ -637,6 +884,7 @@ def instrumented_graph( graph_snapshot: bytes | None = None, typed_stage: tuple[Path, dict] | None = None, work_launch: int | None = None, + sim_engine: str | None = None, ) -> Path: """The input graph plus a trace-recorder node (HAR-4) with absolutized node paths, written under the run dir (dora's cwd becomes the run dir, @@ -703,6 +951,23 @@ def instrumented_graph( for node in doc["nodes"]: if node["id"] in bridge_ids: node["env"] = {**(node.get("env") or {}), "AISLE_SIM_BACKEND": sim_backend} + if sim_engine is not None: + for node in doc["nodes"]: + if node["id"] not in bridge_ids: + continue + env = node.get("env") or {} + # ADR-67: the engine is DECLARED on the bridge, like the rung, so + # the graph hash attests which physics produced a result. A + # declaration this runner would overwrite is refused (engine_check + # catches it before launch; this is the same rule at the last + # writer). + declared = str(env.get("AISLE_SIM_ENGINE") or "").strip().lower() + if "AISLE_SIM_ENGINE" in env and declared != sim_engine: + raise EngineConflict( + f"{node['id']!r} declares AISLE_SIM_ENGINE={env['AISLE_SIM_ENGINE']!r}, " + f"but the run requested engine {sim_engine!r} (ADR-67)" + ) + node["env"] = {**env, "AISLE_SIM_ENGINE": sim_engine} forbidden = FORBIDDEN_BY_RUNG.get(rung, ()) # HAR-4: EVERY declared endpoint, keyed __ so two # producers of the same topic name (e.g. reset_done from both the @@ -856,6 +1121,17 @@ def instrumented_graph( # graph YAML, could never detect the divergence. Scrubbed with the variable # that introduced the same hazard rather than after the first bad run. "AISLE_PERCEPTION", + # ADR-67: which physics engine realizes the scene. The bridge reads it via + # select_engine(os.environ), and `harness fleet` builds its child env from + # this scrub, so an ambient AISLE_SIM_ENGINE=nexus in an operator's shell + # silently swapped the physics of a fleet run while every hash attested + # clean. Each runner that launches a sim now sets it back explicitly, + # AFTER the scrub, so the choice is the runner's. + "AISLE_SIM_ENGINE", + # CON-5/BRG-6: the engine's backend, resolved by the gate from the + # attested dependency extra. Ambient leakage would attest one backend and + # run another, which is the same hazard as the engine above. + "AISLE_SIM_BACKEND", ) @@ -1057,6 +1333,70 @@ def perception_check( return {"ok": True, "rung": rung} +def engine_check( + root: Path, + graph: Path, + requested: str | None, + graph_snapshot: bytes | None = None, +) -> dict: + """ADR-67: the physics engine rides the graph's sim bridge, exactly as the + perception rung does (TC-9), so the graph hash attests which engine + produced a result. `--sim-engine` therefore ASSERTS the declaration rather + than replacing it: a conflict is refused before any gate, budget + reservation, or launch, a graph that declares nothing takes the request + (default genesis), and a request of nothing takes the declaration. + Returns {"ok": True, "engine": , "declared": }.""" + from aisle.harness.registry import load_manifests + from aisle.harness.validate import graph_perception_rung, load_graph + from aisle.sim import DEFAULT_ENGINE, normalize_engine + + try: + requested = None if requested is None else normalize_engine(requested) + except ValueError as exc: + return {"ok": False, "gate": "sim_engine", "detail": str(exc)} + fallback = {"ok": True, "engine": requested or DEFAULT_ENGINE, "declared": None} + nodes, _ = load_graph(graph, graph_snapshot) + manifest_list, manifest_errors = load_manifests(root) + if nodes is None or manifest_errors: + # run_gates' validate owns the verdict on a broken graph or registry; + # this check only refuses what it can actually read + return fallback + _, bridge_ids, _ = graph_perception_rung(nodes, {m["id"]: m for _, m in manifest_list}) + by_id = {node["id"]: node for node in nodes} + declared: set[str] = set() + for bridge_id in sorted(bridge_ids): + env = by_id[bridge_id].get("env") or {} + if "AISLE_SIM_ENGINE" not in env: + continue + try: + declared.add(normalize_engine(str(env.get("AISLE_SIM_ENGINE") or ""))) + except ValueError as exc: + return { + "ok": False, + "gate": "sim_engine", + "detail": f"{bridge_id!r} declares an unusable AISLE_SIM_ENGINE: {exc}", + } + if len(declared) > 1: + return { + "ok": False, + "gate": "sim_engine", + "detail": f"conflicting engines {sorted(declared)} declared across the sim " + f"bridges {sorted(bridge_ids)} (ADR-67): a graph with two engines attests neither", + } + if not declared: + return fallback + engine = declared.pop() + if requested is not None and requested != engine: + return { + "ok": False, + "gate": "sim_engine", + "detail": f"--sim-engine {requested} asserted, but the graph declares engine " + f"{engine} (ADR-67: the engine rides the graph, where the graph hash attests it)", + "hint": f"run a graph whose sim bridge declares AISLE_SIM_ENGINE: {requested}", + } + return {"ok": True, "engine": engine, "declared": engine} + + def rollout( root: Path, graph: Path, @@ -1076,8 +1416,16 @@ def rollout( per_episode_wall_s: int | None = None, typed_stage_factory=None, record_simulator_work: bool = False, + sim_engine: str | None = None, ) -> dict: - """HAR-1: the full run. Returns the report dict (CON-8: caller emits).""" + """HAR-1: the full run. Returns the report dict (CON-8: caller emits). + + sim_engine ASSERTS the engine the graph's bridge declares (ADR-67); None + means no assertion, so the declaration wins and an undeclared graph runs + the default engine. The scene-build wall grace the first episode of every + launch gets (and that each launch adds to the run budget) is + engine-derived (`build_budget_s`), since Genesis builds in minutes and + Nexus in seconds.""" # A relative root (`--root .`) must be pinned to THIS process's cwd: # dora runs with cwd = the run dir, so relative AISLE_RESULTS / # AISLE_TRACE_DIR strings would resolve to a nested runs//runs// @@ -1140,6 +1488,10 @@ def rollout( perception_gate = perception_check(registry_root, graph, perception, graph_snapshot) if not perception_gate["ok"]: return {"ok": False, "refused": perception_gate} + engine_gate = engine_check(root, graph, sim_engine, graph_snapshot) + if not engine_gate["ok"]: + return {"ok": False, "refused": engine_gate} + sim_engine = engine_gate["engine"] authored_graph_hash = hashlib.sha256(graph_snapshot).hexdigest() gates = run_gates( root, @@ -1152,6 +1504,7 @@ def rollout( sim_extra, graph_snapshot=graph_snapshot, **({"typed_stage": typed_stage} if typed_stage is not None else {}), + sim_engine=sim_engine, ) if not gates["ok"]: return { @@ -1232,7 +1585,10 @@ def rollout( **({"work_launch": 0} if record_simulator_work else {}), graph_snapshot=graph_snapshot, **({"typed_stage": typed_stage} if typed_stage is not None else {}), + sim_engine=gates["sim_engine"], ) + except EngineConflict as exc: + return {"ok": False, "refused": {"gate": "sim_engine", "detail": str(exc)}} except RuntimeError as exc: return { "ok": False, @@ -1261,17 +1617,21 @@ def rollout( "AISLE_TIER": tier, # M0-5: the embodiment profile swap rides on env, zero YAML edits "AISLE_EMBODIMENT": embodiment, - # CON-5: the gate resolved this from the explicitly selected, - # attested dependency extra. It never comes from ambient state. - "AISLE_SIM_BACKEND": gates["sim_backend"], "AISLE_TIMEOUT_S": str(episode_timeout_s), "AISLE_RESULTS": str(results_path), # RST-2: the client stamps every reset request with the mode "AISLE_RESET_MODE": reset_mode, } ) + # CON-5 / ADR-67: applied AFTER the scrub, which now strips both from the + # ambient environment. The gate resolved them from the explicitly selected + # extra and the graph's declaration; a developer shell never may. + env["AISLE_SIM_BACKEND"] = gates["sim_backend"] + env["AISLE_SIM_ENGINE"] = gates["sim_engine"] started = time.monotonic() - run_budget_s = timeout_s or (GENESIS_BUILD_BUDGET_S + per_episode_budget_s * episodes) + build_grace_s = build_budget_s(gates["sim_engine"]) + stall_grace_s = pre_data_stall_s(gates["sim_engine"]) + run_budget_s = timeout_s or (build_grace_s + per_episode_budget_s * episodes) if env_baseline != "local": # ADR-21 round 3: the run is CAPPED to the campaign's remaining # wall budget — a single long rollout cannot blow through the @@ -1357,6 +1717,8 @@ def audit_stopped_stage(): lines_at_count = 0 last_lines = 0 last_line_t = time.monotonic() + progress_every = progress_interval_s() + last_progress_t = time.monotonic() try: proc = spawn_launch(0) while time.monotonic() < deadline: @@ -1408,9 +1770,31 @@ def audit_stopped_stage(): if size != last_size: last_size = size last_growth = time.monotonic() - elif time.monotonic() - last_growth > (PRE_DATA_STALL_S if last_size <= 0 else STALL_S): + elif time.monotonic() - last_growth > (stall_grace_s if last_size <= 0 else STALL_S): stalled = True break + grace = per_episode_budget_s + (build_grace_s if lines == lines_at_launch else 0) + now = time.monotonic() + if progress_every > 0 and now - last_progress_t >= progress_every: + last_progress_t = now + print( + progress_report( + now=now, + started=started, + deadline=deadline, + lines=lines, + episodes=episodes, + seeds=seeds, + last_line_t=last_line_t, + episode_grace_s=grace, + traces_size=size, + last_record=_last_record(raw), + relaunches=relaunches, + timing=summarize_timing(read_timing_rows(run_dir)[-3:]), + ), + file=sys.stderr, + flush=True, + ) # per-episode WALL clamp (ADR-23, W/S2 holdout wedge): traces # kept GROWING while one episode ran 4 h, so the stall detector # above never fired and the wedged episode ate the whole run @@ -1419,9 +1803,6 @@ def audit_stopped_stage(): # first of a launch); past that it is killed, recorded as a # synthetic wall_clamp failure, and the run RELAUNCHES with the # remaining seeds so they still get scored. - grace = per_episode_budget_s + ( - GENESIS_BUILD_BUDGET_S if lines == lines_at_launch else 0 - ) if time.monotonic() - last_line_t > grace: _terminate(proc) # stale nodes from the killed launch are CONCURRENT WRITERS @@ -1506,6 +1887,7 @@ def audit_stopped_stage(): **({"work_launch": relaunches} if record_simulator_work else {}), graph_snapshot=graph_snapshot, **({"typed_stage": typed_stage} if typed_stage is not None else {}), + sim_engine=gates["sim_engine"], ) except RuntimeError as exc: # fail closed mid-run too: a registry broken since the @@ -1520,7 +1902,7 @@ def audit_stopped_stage(): # each relaunch pays a fresh build: extend the deadline by # the build grace (still bounded by the campaign wall cap), # else consecutive wedges cut the tail seeds (PR #58 review) - deadline += GENESIS_BUILD_BUDGET_S + deadline += build_grace_s if hard_cap_s is not None: deadline = min(deadline, started + hard_cap_s) pending_typed_stage = typed_stage @@ -1623,8 +2005,17 @@ def audit_stopped_stage(): # CON-5 / SCN-7: dependency selection and the resolved live device # distinguish CPU/Metal/CUDA physics evidence with no hidden probe. "sim_extra": gates["sim_extra"], + "sim_engine": gates["sim_engine"], "sim_backend": gates["sim_backend"], "sim_device": gates["sim_device"], + # ADR-67: env_hash cannot tell two engines (or two solver settings) + # apart, so the engine digest and its build receipt ride here + "sim_engine_build": gates.get("sim_engine_build"), + # ADR-67: the bridge's step/render/GPU timing, aggregated over the run, + # so engines are compared on the same graph and seeds + "sim_timing": summarize_timing(read_timing_rows(run_dir)), + # the first-episode build grace this run clamped against (HAR-1) + "build_grace_s": build_grace_s, "seeds": seeds, "reset": reset_mode, "verifier": verifier, diff --git a/src/aisle/harness/skill.py b/src/aisle/harness/skill.py index b857a798..aba64595 100644 --- a/src/aisle/harness/skill.py +++ b/src/aisle/harness/skill.py @@ -184,9 +184,13 @@ def _lint_or(root: Path, message: str) -> None: raise RegistrationError(f"{message}: {report.get('errors', [])[:3]}") -def run_skill_eval(skill: Skill, root: Path, run_rollout, run_id: str) -> float: +def run_skill_eval( + skill: Skill, root: Path, run_rollout, run_id: str, sim_engine: str | None = None +) -> float: """The shipped mini-rollout against the STAGED candidate: returns the - measured pass rate (pass1).""" + measured pass rate (pass1). sim_engine (ADR-67) rides through to the + runner so the eval is not pinned to genesis; None honours whatever the + eval graph declares.""" cfg = skill.eval_cfg report = run_rollout( root=root, @@ -201,6 +205,7 @@ def run_skill_eval(skill: Skill, root: Path, run_rollout, run_id: str) -> float: no_idea_gate=True, # registration machinery, logged (ADR-22) embodiment=str(cfg["embodiment"]), env_baseline=str(cfg.get("env_baseline", "local")), + sim_engine=sim_engine, ) if not report.get("ok"): detail = report.get("refused") or report.get("error") or "no episodes" @@ -215,9 +220,11 @@ def register_skill( now: str, run_id: str | None = None, sandbox: bool = False, + sim_engine: str | None = None, ) -> dict: """validate → STAGE → lint → eval → evalcard → final lint (§8.4); - every failure rolls the registry back exactly. + every failure rolls the registry back exactly. `sim_engine` (ADR-67) + selects the physics engine the eval rollout runs on. `sandbox=True` (ADR-40, #265) takes the §9.4 lower rung: admit the id so graphs referencing it VALIDATE, while making no quality claim at all — no @@ -285,7 +292,7 @@ def rollback() -> None: ), } eval_run_id = run_id or f"skill-{skill.manifest['id']}-{now}-{uuid.uuid4().hex[:6]}" - pass_rate = run_skill_eval(skill, root, run_rollout, eval_run_id) + pass_rate = run_skill_eval(skill, root, run_rollout, eval_run_id, sim_engine=sim_engine) minimum = float(skill.eval_cfg["min_pass_rate"]) if pass_rate < minimum: raise RegistrationError( diff --git a/src/aisle/harness/swap.py b/src/aisle/harness/swap.py index 4c7262b3..dec4937e 100644 --- a/src/aisle/harness/swap.py +++ b/src/aisle/harness/swap.py @@ -46,6 +46,12 @@ # wholesale can drop the TC-9 rung declaration (env rides the node) and # turn an L1 run into an L0 one while each mutation validates "src/aisle/nodes/dora_genesis.py", + # ADR-67: the engine realization is the bridge's other live half — the + # module that decides which physics steps the scene, and (nexus_physics + # .toml) the solver settings it steps with. env_hash does not fence it, + # so a live swap would be the one way to change the physics of a running + # attested dataflow. + "src/aisle/sim", ) diff --git a/src/aisle/harness/typed_validation.py b/src/aisle/harness/typed_validation.py index 2d31e9cb..d0b05669 100644 --- a/src/aisle/harness/typed_validation.py +++ b/src/aisle/harness/typed_validation.py @@ -31,6 +31,10 @@ "aisle/harness/registry.py", "aisle/harness/validate.py", "aisle/harness/cli.py", + # ADR-67: the CLI's `--sim-engine` choices and the validator's engine + # check both read the engine registry, so the bundle carries it. It + # imports no simulator (CON-12), so this adds no weight to the sandbox. + "aisle/sim/__init__.py", ) _SOURCE_ROOT = Path(__file__).resolve().parents[2] _BOOTSTRAP = ( diff --git a/src/aisle/harness/validate.py b/src/aisle/harness/validate.py index 6c1c7f3b..fd65f818 100644 --- a/src/aisle/harness/validate.py +++ b/src/aisle/harness/validate.py @@ -746,6 +746,9 @@ def validate_nodes( # TC-9/VAL-8: the perception rung is a property of the GRAPH, read once rung, bridge_ids, rung_errors = graph_perception_rung(nodes, manifests) errors.extend(rung_errors) + # ADR-67: the engine is declared on the bridge like the rung, so a typo + # must be caught here rather than at launch + errors.extend(graph_sim_engine_errors(nodes, bridge_ids)) # ADR-32 §1: T4 goal blinding is likewise a property of the graph errors.extend(_dialogue_blinding_errors(nodes)) for node in nodes: @@ -1413,6 +1416,38 @@ def graph_perception_rung(nodes: list, manifests: dict) -> tuple[str, list[str], return rung, bridge_ids, errors +def graph_sim_engine_errors(nodes: list, bridge_ids: list[str]) -> list[dict]: + """VAL-8 companion for ADR-67: a sim bridge declaring AISLE_SIM_ENGINE + must name an engine the harness knows. + + Deliberately minimal next to the rung: the engine forbids no topics, so + the only thing a graph can get silently wrong is the name. An unknown one + refuses at bridge config time hours later, or worse, defaults elsewhere; + naming it here costs one dictionary lookup.""" + from aisle.sim import ENGINES, normalize_engine + + errors: list[dict] = [] + by_id = {node["id"]: node for node in nodes} + for bridge_id in sorted(set(bridge_ids)): + env = by_id.get(bridge_id, {}).get("env") or {} + if "AISLE_SIM_ENGINE" not in env: + continue + declared = env.get("AISLE_SIM_ENGINE") + try: + normalize_engine(str(declared or "")) + except ValueError: + errors.append( + _entry( + "SIM_ENGINE_UNKNOWN", + {"node": bridge_id}, + f"unknown simulation engine {declared!r} declared by {bridge_id!r} (ADR-67)", + f"set AISLE_SIM_ENGINE to one of {list(ENGINES)}, or drop the key to " + "let the runner choose", + ) + ) + return errors + + FORBIDDEN_BY_RUNG = { "L0": (), # TC-9: at L1 pose must be ESTIMATED from segmentation + depth, so the diff --git a/src/aisle/nodes/dora_genesis.py b/src/aisle/nodes/dora_genesis.py index 11f81f9a..fafe8655 100644 --- a/src/aisle/nodes/dora_genesis.py +++ b/src/aisle/nodes/dora_genesis.py @@ -254,6 +254,11 @@ class BridgeConfig: # CON-5: rollout resolves and attests this backend before launch. None is # the deterministic portable default for direct/debug graph execution. sim_backend: str | None = None + # ADR-67/ADR-68: which physics engine realizes the scene + # (AISLE_SIM_ENGINE): `genesis` (the frozen builders), `nexus` or + # `rapier`. Graph-declared like the rung, injected by rollout, attested + # in bridge_info and the manifest. + sim_engine: str = "genesis" # T2: printed med labels rendered as box textures. Graph-declared # (AISLE_LABELS on the bridge node) so the graph hash attests whether a # run's pixels carried text -- same reasoning as the rung; ambient env @@ -311,13 +316,15 @@ def parse_bridge_config(env: dict) -> BridgeConfig: ) if occlusion_raw not in ("0", "1", "true", "false", "yes", "no"): raise ValueError(f"unknown AISLE_OCCLUSION value {occlusion_raw!r}; use 0/1") + from aisle.sim import normalize_engine, validate_backend + + sim_engine = normalize_engine(env.get("AISLE_SIM_ENGINE")) sim_backend = env.get("AISLE_SIM_BACKEND") if sim_backend is not None: sim_backend = str(sim_backend).strip().lower() - if sim_backend not in ("cpu", "metal", "cuda"): - raise ValueError( - f"unknown simulation backend {sim_backend!r}; expected cpu, metal, or cuda" - ) + # refuse rather than default: a backend the engine cannot run must + # not silently attest another one (same rule as the rung) + validate_backend(sim_engine, sim_backend) return BridgeConfig( perception=perception, seed=int(env.get("AISLE_SEED", "0")), @@ -329,6 +336,7 @@ def parse_bridge_config(env: dict) -> BridgeConfig: step_without_reset=env.get("AISLE_STEP_WITHOUT_RESET", "0").strip().lower() in ("1", "true", "yes"), sim_backend=sim_backend, + sim_engine=sim_engine, labels=labels_raw in ("1", "true", "yes"), shuffle_colors=shuffle_raw in ("1", "true", "yes"), occlusion=occlusion_raw in ("1", "true", "yes"), @@ -540,6 +548,7 @@ def make_bridge_info( perception: str, segmentation_ids: dict, sim_backend: str, + sim_engine: str, ) -> str: """BRG-6 + BRG-8: the startup contract announcement, as a JSON string. @@ -567,7 +576,13 @@ def make_bridge_info( silently selects other geometry (measured: robot links with identical pixel counts across different layouts). Publishing the map is what keeps a consumer from having to guess. sim_backend is required for the same - recorded-vs-actual reason: CUDA and CPU traces must not look identical.""" + recorded-vs-actual reason: CUDA and CPU traces must not look identical. + + sim_engine (ADR-67) names the physics engine; genesis_version keeps its + BRG-6 name and carries the version of whichever engine ran. It is + required for the same recorded-vs-actual reason as sim_backend: a + defaulted "genesis" is a valid name, so the wrong engine would be + attested with nothing to catch it.""" return json.dumps( { "contract": "v0", @@ -581,6 +596,7 @@ def make_bridge_info( "env_hash": env_hash, "step_without_reset": step_without_reset, "sim_backend": sim_backend, + "sim_engine": sim_engine, "calibration": calibration, } ) @@ -686,6 +702,200 @@ def render_frames(cameras, due, work): return frames +def timing_sidecar_path(env: dict) -> Path | None: + """Where the bridge writes its timing rows: beside the run's results file + (`AISLE_RESULTS`, set by `harness rollout`), or nowhere for a graph + launched by hand (`dora run` has no run directory to attest into).""" + results = env.get("AISLE_RESULTS") + return Path(results).parent / "sim_timing.jsonl" if results else None + + +class StepTimer: + """Wall and engine timing of the bridge's physics steps and camera renders, + flushed as one JSON row per window of steps to a sidecar file so a run's + performance can be compared across engines (ADR-67) without a new topic: + the topic contract (SPEC 010) and the frozen graphs stay untouched. + + A row carries the window's step count, mean/max wall time of the physics + step call, mean wall time of the whole tick (step plus every state read + and publish that followed it: the reads block on the GPU, so the tick is + the honest cost while the step call alone can be an asynchronous submit), + the render frame count and mean render wall time, the sim seconds covered, + and, when the engine reports it (`scene.perf_stats()`), the mean GPU time + per step. `sim_window_s` over the summed tick and render time is the + real-time factor. The clock is injected (CON-5).""" + + def __init__( + self, + engine: str, + path: Path | None, + flush_every: int = 100, + clock: Callable[[], float] = time.perf_counter, + ): + self.engine = engine + self.path = path + self.flush_every = max(1, int(flush_every)) + self.clock = clock + self.rows: list[dict] = [] + self._reset() + + def _reset(self) -> None: + self.steps = 0 + self.step_wall = 0.0 + self.step_max = 0.0 + self.tick_wall = 0.0 + self.frames = 0 + self.render_wall = 0.0 + self.gpu_ms = 0.0 + self.gpu_steps = 0 + self.window_start_ns: int | None = None + + def record_step( + self, + wall_s: float, + sim_time_ns: int, + perf: dict | None = None, + tick_s: float | None = None, + ) -> None: + if self.window_start_ns is None: + self.window_start_ns = sim_time_ns + self.steps += 1 + self.step_wall += wall_s + self.step_max = max(self.step_max, wall_s) + self.tick_wall += tick_s if tick_s is not None else wall_s + if perf and perf.get("gpu_ms") is not None: + self.gpu_ms += float(perf["gpu_ms"]) + self.gpu_steps += 1 + if self.steps >= self.flush_every: + self.flush(sim_time_ns) + + def record_render(self, wall_s: float) -> None: + self.frames += 1 + self.render_wall += wall_s + + def flush(self, sim_time_ns: int) -> dict | None: + if self.steps == 0: + return None + row = { + "engine": self.engine, + "sim_time_s": sim_time_ns / 1e9, + "sim_window_s": (sim_time_ns - (self.window_start_ns or sim_time_ns)) / 1e9, + "steps": self.steps, + "step_ms_mean": 1000.0 * self.step_wall / self.steps, + "step_ms_max": 1000.0 * self.step_max, + "tick_ms_mean": 1000.0 * self.tick_wall / self.steps, + "frames": self.frames, + "render_ms_mean": (1000.0 * self.render_wall / self.frames) if self.frames else None, + "gpu_ms_mean": (self.gpu_ms / self.gpu_steps) if self.gpu_steps else None, + } + self.rows.append(row) + if self.path is not None: + with open(self.path, "a") as f: + f.write(json.dumps(row) + "\n") + self._reset() + return row + + +class DebugVideo: + """Appends rendered frames to an mp4 beside the run's results (10 fps, + like the recorder's overhead.mp4); opened on the first frame, closed by + the bridge on shutdown.""" + + def __init__(self, path: Path): + self.path = path + self._writer = None + + def append(self, frame: np.ndarray) -> None: + if self._writer is None: + import imageio.v2 as imageio + + self._writer = imageio.get_writer(self.path, fps=10, macro_block_size=1) + self._writer.append_data(np.asarray(frame, dtype=np.uint8)) + + def close(self) -> None: + if self._writer is not None: + self._writer.close() + self._writer = None + + +def genesis_simulator_source() -> Path: + """Genesis's own simulator module, for the physics journal's source + binding. Imported here rather than at the top of the bridge so a run on + another engine never needs Genesis installed (ADR-67).""" + import genesis + + return Path(genesis.__file__).parent / "engine/simulator.py" + + +def parse_debug_view(env: dict, layout: dict) -> tuple[list[float], list[float]] | None: + """The optional operator-only debug camera (AISLE_DEBUG_VIEW): `side` + looks at the shelf front from the tray side, so a grasp and its slip are + seen in profile rather than from above; `px,py,pz;lx,ly,lz` is an + explicit eye and look-at in the robot base frame. Unset or empty means + none. This camera feeds `debug_view.mp4` beside the run's results only: + it is not a topic, so the contract (SPEC 010), the graphs and the + recorded traces are unchanged, and the frozen overhead camera (SCN-5) + keeps producing `overhead.mp4`.""" + raw = str(env.get("AISLE_DEBUG_VIEW") or "").strip().lower() + if not raw: + return None + if raw == "side": + shelf = layout["shelf"] + front_x = shelf["pos"][0] - shelf["level_size"][0] / 2 + top_z = shelf["pos"][2] + max(shelf["level_heights"]) + eye = [front_x - 0.15, -1.1, top_z + 0.15] + lookat = [front_x + 0.1, -0.1, top_z * 0.6] + return eye, lookat + try: + eye_s, lookat_s = raw.split(";") + eye = [float(v) for v in eye_s.split(",")] + lookat = [float(v) for v in lookat_s.split(",")] + except ValueError as exc: + raise ValueError( + f"AISLE_DEBUG_VIEW must be 'side' or 'px,py,pz;lx,ly,lz', got {raw!r}" + ) from exc + if len(eye) != 3 or len(lookat) != 3: + raise ValueError(f"AISLE_DEBUG_VIEW needs two xyz triples, got {raw!r}") + return eye, lookat + + +# the operator debug camera: its own resolution and field of view, kept out +# of the frozen scene's camera table because nothing on the wire uses it +DEBUG_CAMERA_RES = (640, 480) +DEBUG_CAMERA_FOV = 55 + + +def add_debug_camera(scene, sim_engine: str, eye, lookat): + """A rasterizer-only camera added to a built scene. The Nexus backend + accepts cameras at any time, and so does the rapier backend, which renders + through the same viewer (ADR-68). Genesis fixes the scene's cameras at + build (`Scene.add_camera` asserts unbuilt), but its visualizer still + registers a `debug=True` camera afterwards: it renders through the + rasterizer only, outside the batch renderer and raytracer, exactly what an + operator view needs. Either way the frozen scene builder is not touched.""" + if sim_engine in ("nexus", "rapier"): + return scene.add_camera(DEBUG_CAMERA_RES, DEBUG_CAMERA_FOV, pos=eye, lookat=lookat) + camera = scene.visualizer.add_camera( + res=DEBUG_CAMERA_RES, + pos=tuple(eye), + lookat=tuple(lookat), + up=(0.0, 0.0, 1.0), + model="pinhole", + fov=DEBUG_CAMERA_FOV, + aperture=2.0, + focus_dist=None, + GUI=False, + spp=256, + denoise=False, + near=0.1, + far=20.0, + env_idx=None, + debug=True, + ) + camera.build() + return camera + + def main( clock: Callable[[], float] = time.perf_counter, work_clock: Callable[[], int] = time.monotonic_ns, @@ -694,7 +904,6 @@ def main( wall clock ad hoc.""" from contextlib import ExitStack - import genesis import pyarrow as pa from dora import Node @@ -703,15 +912,14 @@ def main( from aisle.scenes.pharmacy import ( SceneCfg, _ensure_genesis, - build_scene, desk_scan_obstacles, load_physics, oracle_state, resolve_layout, sample_placements, - select_genesis_backend, to_numpy, ) + from aisle.sim import build_scene, build_store, engine_version, select_sim_backend cfg = parse_bridge_config(os.environ) require_single_env_for_mobile(cfg.embodiment, cfg.n_envs) @@ -728,15 +936,18 @@ def main( dt_ns=int(dt * 1e9), n_envs=cfg.n_envs, clock=work_clock, - physics_source_path=Path(genesis.__file__).parent / "engine/simulator.py" - if "AISLE_SIM_WORK_PATH" in os.environ + physics_source_path=genesis_simulator_source() + if "AISLE_SIM_WORK_PATH" in os.environ and cfg.sim_engine == "genesis" else None, ) as work, ExitStack() as physics_observer, ): def observed_build(builder, **kwargs): - if "AISLE_SIM_WORK_PATH" in os.environ: + # ADR-67: the physics journal observes Genesis's own Simulator + # class, so it applies to that engine alone; the others report + # their step cost through the timing sidecar instead. + if "AISLE_SIM_WORK_PATH" in os.environ and cfg.sim_engine == "genesis": _ensure_genesis(cfg.sim_backend) from genesis.engine.simulator import Simulator @@ -751,7 +962,6 @@ def observed_build(builder, **kwargs): is_store = cfg.scene == "store" if is_store: from aisle.scenes.store import ( - build_store, generate_episode, load_planogram, store_oracle_state, @@ -763,6 +973,7 @@ def observed_build(builder, **kwargs): "build", observed_build, build_store, + engine=cfg.sim_engine, seed=cfg.seed, scenario=cfg.scenario, embodiment=cfg.embodiment, @@ -774,6 +985,7 @@ def observed_build(builder, **kwargs): "build", observed_build, build_scene, + engine=cfg.sim_engine, seed=cfg.seed, embodiment=cfg.embodiment, n_envs=cfg.n_envs, @@ -819,6 +1031,19 @@ def observed_build(builder, **kwargs): node = Node() sim_time_ns = 0 turn_id = 0 + step_timer = StepTimer(cfg.sim_engine, timing_sidecar_path(os.environ), clock=clock) + perf_stats = getattr(handle.scene, "perf_stats", None) + # operator debug camera (AISLE_DEBUG_VIEW): added after the build so the + # frozen scene stays untouched; frames go to debug_view.mp4, never on + # the wire + debug_cam = None + debug_video = None + debug_view = parse_debug_view(os.environ, resolve_layout(physics, cfg.embodiment)) + if debug_view is not None: + eye, lookat = debug_view + debug_cam = add_debug_camera(handle.scene, cfg.sim_engine, eye, lookat) + sidecar = timing_sidecar_path(os.environ) + debug_video = DebugVideo((sidecar.parent if sidecar else Path.cwd()) / "debug_view.mp4") try: turn_epoch = int(os.environ.get("AISLE_TURN_EPOCH", "1")) except ValueError as exc: @@ -853,14 +1078,15 @@ def emit_bridge(topic: str, value, metadata: dict) -> None: embodiment=cfg.embodiment, n_dof=n_dof, n_envs=cfg.n_envs, - genesis_version=genesis.__version__, + genesis_version=engine_version(cfg.sim_engine), env_hash=compute_env_hash(root), step_without_reset=cfg.step_without_reset, calibration=realized_calibration(handle, physics, is_store), perception=cfg.perception, segmentation_ids=segmentation_ids, sim_backend=cfg.sim_backend - or select_genesis_backend("sim", platform.system()), + or select_sim_backend(cfg.sim_engine, "sim", platform.system()), + sim_engine=cfg.sim_engine, ) ] ), @@ -958,7 +1184,17 @@ def env_slice(tensor, env_id: int) -> np.ndarray: return data[env_id] if cfg.n_envs > 1 else data.reshape(-1) def render_due(due: list[str]) -> dict[str, np.ndarray]: - return render_frames(handle.cams, due, work) + render_t0 = clock() + frames = render_frames(handle.cams, due, work) + # timed before the operator view, so AISLE_DEBUG_VIEW never skews + # the cross-engine render numbers + if frames: + step_timer.record_render(clock() - render_t0) + # the operator view rides the overhead tick so it never adds a + # render pass of its own, and never reaches the wire + if "rgb_overhead" in due and debug_video is not None: + debug_video.append(debug_cam.render()[0]) + return frames def publish(topic: str, frames: dict[str, np.ndarray] | None = None) -> None: # TC-9: the rung's topic set is the SINGLE source of truth for what @@ -1234,287 +1470,324 @@ def finish_turn(actions, shutdown: bool): if stop: return - for event in bridge_events(): - if event["type"] != "INPUT": - continue - input_id = event["id"] - metadata = event.get("metadata") or {} - if input_id == "tick": - if awaiting_first_reset: + try: + for event in bridge_events(): + if event["type"] != "INPUT": continue - held_envs = ( - [env for env in range(cfg.n_envs) if quarantine.held(env)] - if home_hold is not None - else [] - ) - quarantine.tick() - # mobile keeps the whole-scene semantics (single-env by spec) - settling = bool(held_envs) and len(held_envs) == cfg.n_envs - if settling: - # every env settling (single-env case unchanged): hold the - # arm(s) at home and DROP stale in-flight joint_cmds so - # they cannot drive the just-homed arm off home - commands.drain() - batched = home_hold if cfg.n_envs == 1 else np.tile(home_hold, (cfg.n_envs, 1)) - robot.control_dofs_position(batched) - elif held_envs: - # fleet mode: only the quarantined envs hold and drain — - # the others' episodes keep executing - for env in held_envs: - commands.drain(env) - robot.control_dofs_position(home_hold[None, :], envs_idx=[env]) - apply_commands() - else: - apply_commands() - if is_mobile: - # KINEMATIC GRASP ATTACH (T15 rounds 14-18, ADR-18): the - # kinematic base teleports the arm, and repeated pinning - # destroyed the physical pinch (round 18: the box dropped - # the moment physics resumed). Standard sim solution: from - # grip close to finger open the held box rides the HAND - # LINK every tick — physics never needs to hold it. Latch - # captures the hand-frame offset; release hands the box - # back to physics at the drop hover. - if is_store and hand_link is not None: - fingers = float(np.mean(to_numpy(robot.get_qpos()).reshape(-1)[-2:])) - hand_pos = to_numpy(hand_link.get_pos()).reshape(-1)[:3] - hq = to_numpy(hand_link.get_quat()).reshape(-1)[:4] - # yaw of the wrist-down flange (w,x,y,z quat) - hand_yaw = float( - np.arctan2( - 2 * (hq[0] * hq[3] + hq[1] * hq[2]), - 1 - 2 * (hq[2] * hq[2] + hq[3] * hq[3]), - ) + input_id = event["id"] + metadata = event.get("metadata") or {} + if input_id == "tick": + if awaiting_first_reset: + continue + held_envs = ( + [env for env in range(cfg.n_envs) if quarantine.held(env)] + if home_hold is not None + else [] + ) + quarantine.tick() + # mobile keeps the whole-scene semantics (single-env by spec) + settling = bool(held_envs) and len(held_envs) == cfg.n_envs + if settling: + # every env settling (single-env case unchanged): hold the + # arm(s) at home and DROP stale in-flight joint_cmds so + # they cannot drive the just-homed arm off home + commands.drain() + batched = ( + home_hold if cfg.n_envs == 1 else np.tile(home_hold, (cfg.n_envs, 1)) ) - if held_item is None and fingers < 0.025: - best_id, best_d = None, 0.15 - for item_id, entity in handle.items.items(): - p = to_numpy(entity.get_pos()).reshape(-1)[:3] - d = float(np.linalg.norm(p - hand_pos)) - if d < best_d: - best_id, best_d = item_id, d - if best_id is not None: - p = to_numpy(handle.items[best_id].get_pos()).reshape(-1)[:3] - q = to_numpy(handle.items[best_id].get_quat()).reshape(-1)[:4] - item_yaw = 2.0 * float(np.arctan2(float(q[3]), float(q[0]))) - cos_h, sin_h = np.cos(-hand_yaw), np.sin(-hand_yaw) - dx, dy = p[0] - hand_pos[0], p[1] - hand_pos[1] - held_item = best_id - held_offset = ( - float(dx * cos_h - dy * sin_h), - float(dx * sin_h + dy * cos_h), - float(p[2] - hand_pos[2]), - float(item_yaw - hand_yaw), + robot.control_dofs_position(batched) + elif held_envs: + # fleet mode: only the quarantined envs hold and drain — + # the others' episodes keep executing + for env in held_envs: + commands.drain(env) + robot.control_dofs_position(home_hold[None, :], envs_idx=[env]) + apply_commands() + else: + apply_commands() + if is_mobile: + # KINEMATIC GRASP ATTACH (T15 rounds 14-18, ADR-18): the + # kinematic base teleports the arm, and repeated pinning + # destroyed the physical pinch (round 18: the box dropped + # the moment physics resumed). Standard sim solution: from + # grip close to finger open the held box rides the HAND + # LINK every tick — physics never needs to hold it. Latch + # captures the hand-frame offset; release hands the box + # back to physics at the drop hover. + if is_store and hand_link is not None: + fingers = float(np.mean(to_numpy(robot.get_qpos()).reshape(-1)[-2:])) + hand_pos = to_numpy(hand_link.get_pos()).reshape(-1)[:3] + hq = to_numpy(hand_link.get_quat()).reshape(-1)[:4] + # yaw of the wrist-down flange (w,x,y,z quat) + hand_yaw = float( + np.arctan2( + 2 * (hq[0] * hq[3] + hq[1] * hq[2]), + 1 - 2 * (hq[2] * hq[2] + hq[3] * hq[3]), + ) + ) + if held_item is None and fingers < 0.025: + best_id, best_d = None, 0.15 + for item_id, entity in handle.items.items(): + p = to_numpy(entity.get_pos()).reshape(-1)[:3] + d = float(np.linalg.norm(p - hand_pos)) + if d < best_d: + best_id, best_d = item_id, d + if best_id is not None: + p = to_numpy(handle.items[best_id].get_pos()).reshape(-1)[:3] + q = to_numpy(handle.items[best_id].get_quat()).reshape(-1)[:4] + item_yaw = 2.0 * float(np.arctan2(float(q[3]), float(q[0]))) + cos_h, sin_h = np.cos(-hand_yaw), np.sin(-hand_yaw) + dx, dy = p[0] - hand_pos[0], p[1] - hand_pos[1] + held_item = best_id + held_offset = ( + float(dx * cos_h - dy * sin_h), + float(dx * sin_h + dy * cos_h), + float(p[2] - hand_pos[2]), + float(item_yaw - hand_yaw), + ) + print(f"carry latch: {best_id}", file=sys.stderr) + elif held_item is not None and fingers > 0.035: + print(f"carry release: {held_item}", file=sys.stderr) + held_item = None + if held_item is not None: + off = held_offset + cos_h, sin_h = np.cos(hand_yaw), np.sin(hand_yaw) + held_entity = handle.items[held_item] + held_entity.set_pos( + np.array( + [ + hand_pos[0] + off[0] * cos_h - off[1] * sin_h, + hand_pos[1] + off[0] * sin_h + off[1] * cos_h, + hand_pos[2] + off[2], + ], + dtype=np.float32, + ) ) - print(f"carry latch: {best_id}", file=sys.stderr) - elif held_item is not None and fingers > 0.035: - print(f"carry release: {held_item}", file=sys.stderr) - held_item = None - if held_item is not None: - off = held_offset - cos_h, sin_h = np.cos(hand_yaw), np.sin(hand_yaw) - held_entity = handle.items[held_item] - held_entity.set_pos( - np.array( - [ - hand_pos[0] + off[0] * cos_h - off[1] * sin_h, - hand_pos[1] + off[0] * sin_h + off[1] * cos_h, - hand_pos[2] + off[2], - ], - dtype=np.float32, + hh = (hand_yaw + off[3]) / 2 + held_entity.set_quat( + np.array([np.cos(hh), 0.0, 0.0, np.sin(hh)], dtype=np.float32) ) + held_entity.zero_all_dofs_velocity() + # MOB-1/ADR-13: integrate the base from the latest base_cmd + # (held at rest during the post-reset settle) and re-base the + # arm's root before stepping + cmd = [0.0, 0.0] if settling else base_cmd + new_pose = integrate_base_pose(base_pose, cmd, dt) + # re-base ONLY when the base actually moved (T15 round 13): + # an every-tick set_pos/set_quat perturbs the solver state + # each step and the gravity-loaded wrist joints chronically + # lagged ~0.1-0.7 rad — the fingers plowed instead of + # pinching. A stationary base leaves the arm's PD untouched, + # matching the (proven) desk behavior. + if new_pose != base_pose: + base_pose = new_pose + half = base_pose[2] / 2 + robot.set_pos( + np.array([base_pose[0], base_pose[1], 0.0], dtype=np.float32) ) - hh = (hand_yaw + off[3]) / 2 - held_entity.set_quat( - np.array([np.cos(hh), 0.0, 0.0, np.sin(hh)], dtype=np.float32) + robot.set_quat( + np.array([np.cos(half), 0.0, 0.0, np.sin(half)], dtype=np.float32) + ) + if so101_latch is not None: + finger = float(to_numpy(robot.get_qpos()).reshape(-1)[finger_idx[0]]) + grip = float( + np.clip( + (gripper_open - finger) / (gripper_open - gripper_close), 0.0, 1.0 ) - held_entity.zero_all_dofs_velocity() - # MOB-1/ADR-13: integrate the base from the latest base_cmd - # (held at rest during the post-reset settle) and re-base the - # arm's root before stepping - cmd = [0.0, 0.0] if settling else base_cmd - new_pose = integrate_base_pose(base_pose, cmd, dt) - # re-base ONLY when the base actually moved (T15 round 13): - # an every-tick set_pos/set_quat perturbs the solver state - # each step and the gravity-loaded wrist joints chronically - # lagged ~0.1-0.7 rad — the fingers plowed instead of - # pinching. A stationary base leaves the arm's PD untouched, - # matching the (proven) desk behavior. - if new_pose != base_pose: - base_pose = new_pose - half = base_pose[2] / 2 - robot.set_pos(np.array([base_pose[0], base_pose[1], 0.0], dtype=np.float32)) - robot.set_quat( - np.array([np.cos(half), 0.0, 0.0, np.sin(half)], dtype=np.float32) ) - if so101_latch is not None: - finger = float(to_numpy(robot.get_qpos()).reshape(-1)[finger_idx[0]]) - grip = float( - np.clip((gripper_open - finger) / (gripper_open - gripper_close), 0.0, 1.0) + hand_pos = to_numpy(so101_hand_link.get_pos()).reshape(-1)[:3] + hand_quat = to_numpy(so101_hand_link.get_quat()).reshape(-1)[:4] + tcp_pos = frame_point_wxyz(hand_pos, hand_quat, so101_tcp_offset) + candidates = { + name: ( + to_numpy(entity.get_pos()).reshape(-1)[:3], + to_numpy(entity.get_quat()).reshape(-1)[:4], + ) + for name, entity in handle.boxes.items() + } + before = so101_latch.held_name + attached = so101_latch.update(grip, tcp_pos, hand_quat, candidates) + if before != so101_latch.held_name: + action = "latch" if so101_latch.held_name is not None else "release" + print( + f"so101 carry {action}: {before or so101_latch.held_name}", + file=sys.stderr, + ) + if attached is not None: + name, pos, quat = attached + entity = handle.boxes[name] + entity.set_pos(pos) + entity.set_quat(quat) + entity.zero_all_dofs_velocity() + step_t0 = clock() + work.call("step", handle.scene.step) # BRG-7: exceptions crash the node loudly + step_wall = clock() - step_t0 + sim_time_ns += int(dt * 1e9) + due = scheduler.due() + frames = render_due(due) + for topic in due: + publish(topic, frames) + # after the publishes: the state reads they force are the + # device sync the bare step call may not include, on either + # GPU engine. The tick is read before perf_stats(), because on + # Nexus that call syncs the viewer and would bias the + # cross-engine comparison it exists to serve. + tick_s = clock() - step_t0 + step_timer.record_step( + step_wall, + sim_time_ns, + perf_stats() if perf_stats else None, + tick_s=tick_s, ) - hand_pos = to_numpy(so101_hand_link.get_pos()).reshape(-1)[:3] - hand_quat = to_numpy(so101_hand_link.get_quat()).reshape(-1)[:4] - tcp_pos = frame_point_wxyz(hand_pos, hand_quat, so101_tcp_offset) - candidates = { - name: ( - to_numpy(entity.get_pos()).reshape(-1)[:3], - to_numpy(entity.get_quat()).reshape(-1)[:4], - ) - for name, entity in handle.boxes.items() - } - before = so101_latch.held_name - attached = so101_latch.update(grip, tcp_pos, hand_quat, candidates) - if before != so101_latch.held_name: - action = "latch" if so101_latch.held_name is not None else "release" - print( - f"so101 carry {action}: {before or so101_latch.held_name}", - file=sys.stderr, + elif input_id == "joint_cmd": + payload = np.asarray( + event["value"].to_numpy(zero_copy_only=False), dtype=np.float32 + ).reshape(-1) + if payload.shape[0] != n_dof: + raise ValueError( + f"joint_cmd must be Float32[{n_dof}], " + f"got length {payload.shape[0]} (TC-5)" ) - if attached is not None: - name, pos, quat = attached - entity = handle.boxes[name] - entity.set_pos(pos) - entity.set_quat(quat) - entity.zero_all_dofs_velocity() - work.call("step", handle.scene.step) # BRG-7: exceptions crash the node loudly - sim_time_ns += int(dt * 1e9) - due = scheduler.due() - frames = render_due(due) - for topic in due: - publish(topic, frames) - elif input_id == "joint_cmd": - payload = np.asarray( - event["value"].to_numpy(zero_copy_only=False), dtype=np.float32 - ).reshape(-1) - if payload.shape[0] != n_dof: - raise ValueError( - f"joint_cmd must be Float32[{n_dof}], got length {payload.shape[0]} (TC-5)" - ) - commands.push("joint", metadata.get("env_id"), payload) - env_id = int(metadata.get("env_id", 0)) - dropped_counts["joint"][env_id] = dropped_counts["joint"].get(env_id, 0) + int( - metadata.get("dropped", 0) - ) - elif input_id == "gripper_cmd": - payload = np.asarray( - event["value"].to_numpy(zero_copy_only=False), dtype=np.float32 - ).reshape(-1) - if payload.shape[0] != 1 or not 0.0 <= float(payload[0]) <= 1.0: - raise ValueError( - f"gripper_cmd must be Float32[1] in [0, 1], got {payload!r} (TC table)" - ) - commands.push("gripper", metadata.get("env_id"), payload) - env_id = int(metadata.get("env_id", 0)) - dropped_counts["gripper"][env_id] = dropped_counts["gripper"].get(env_id, 0) + int( - metadata.get("dropped", 0) - ) - elif input_id == "base_cmd": - # MOB-1: latest diff-drive command [v, omega]; integrated each - # tick (the guard has already clamped it, MOB-3) - payload = np.asarray( - event["value"].to_numpy(zero_copy_only=False), dtype=np.float32 - ).reshape(-1) - if payload.shape[0] != 2: - raise ValueError( - f"base_cmd must be Float32[2] (v, omega), got {payload!r} (MOB-1)" + commands.push("joint", metadata.get("env_id"), payload) + env_id = int(metadata.get("env_id", 0)) + dropped_counts["joint"][env_id] = dropped_counts["joint"].get(env_id, 0) + int( + metadata.get("dropped", 0) ) - base_cmd = [float(payload[0]), float(payload[1])] - env_id = int(metadata.get("env_id", 0)) - dropped_counts["base"][env_id] = dropped_counts["base"].get(env_id, 0) + int( - metadata.get("dropped", 0) - ) - elif input_id == "reset": - with work.operation("reset"): - started = clock() - payload = np.asarray(event["value"].to_numpy(zero_copy_only=False)).reshape(-1) + elif input_id == "gripper_cmd": + payload = np.asarray( + event["value"].to_numpy(zero_copy_only=False), dtype=np.float32 + ).reshape(-1) + if payload.shape[0] != 1 or not 0.0 <= float(payload[0]) <= 1.0: + raise ValueError( + f"gripper_cmd must be Float32[1] in [0, 1], got {payload!r} (TC table)" + ) + commands.push("gripper", metadata.get("env_id"), payload) + env_id = int(metadata.get("env_id", 0)) + dropped_counts["gripper"][env_id] = dropped_counts["gripper"].get( + env_id, 0 + ) + int(metadata.get("dropped", 0)) + elif input_id == "base_cmd": + # MOB-1: latest diff-drive command [v, omega]; integrated each + # tick (the guard has already clamped it, MOB-3) + payload = np.asarray( + event["value"].to_numpy(zero_copy_only=False), dtype=np.float32 + ).reshape(-1) if payload.shape[0] != 2: raise ValueError( - f"reset payload must be UInt32[2], got {payload.shape} (TC-6)" + f"base_cmd must be Float32[2] (v, omega), got {payload!r} (MOB-1)" ) - reset_seed, mode = int(payload[0]), int(payload[1]) - if mode not in (0, 1): - raise ValueError(f"reset mode must be 0 or 1, got {mode} (TC-6)") - if not metadata.get("request_id"): - raise ValueError("reset request missing request_id metadata (TC-6)") - # TC-6: no observation may interleave reset -> reset_done; the - # loop is single-threaded, so replying before returning to the - # event loop guarantees ordering - if mode == 1: - raise NotImplementedError("behavioral reset lands with SPEC 040 (T06)") - # fleet mode (BRG-5): a reset carrying env_id teleports only - # that env's slice; the shared publish grid is NOT re-anchored - # (it belongs to every env — fleet cadence is anchored to the - # first whole-scene reset, a documented fleet-mode difference - # from ADR-25's per-episode anchoring) - reset_env = metadata.get("env_id") if cfg.n_envs > 1 else None - if reset_env is not None: - if isinstance(reset_env, bool) or not isinstance(reset_env, int): + base_cmd = [float(payload[0]), float(payload[1])] + env_id = int(metadata.get("env_id", 0)) + dropped_counts["base"][env_id] = dropped_counts["base"].get(env_id, 0) + int( + metadata.get("dropped", 0) + ) + elif input_id == "reset": + with work.operation("reset"): + started = clock() + payload = np.asarray(event["value"].to_numpy(zero_copy_only=False)).reshape( + -1 + ) + if payload.shape[0] != 2: raise ValueError( - f"reset env_id must be an int, got {reset_env!r} (BRG-5)" + f"reset payload must be UInt32[2], got {payload.shape} (TC-6)" ) - if not 0 <= reset_env < cfg.n_envs: - raise ValueError( - f"reset env_id {reset_env} outside [0, {cfg.n_envs}) (BRG-5)" + reset_seed, mode = int(payload[0]), int(payload[1]) + if mode not in (0, 1): + raise ValueError(f"reset mode must be 0 or 1, got {mode} (TC-6)") + if not metadata.get("request_id"): + raise ValueError("reset request missing request_id metadata (TC-6)") + # TC-6: no observation may interleave reset -> reset_done; the + # loop is single-threaded, so replying before returning to the + # event loop guarantees ordering + if mode == 1: + raise NotImplementedError("behavioral reset lands with SPEC 040 (T06)") + # fleet mode (BRG-5): a reset carrying env_id teleports only + # that env's slice; the shared publish grid is NOT re-anchored + # (it belongs to every env — fleet cadence is anchored to the + # first whole-scene reset, a documented fleet-mode difference + # from ADR-25's per-episode anchoring) + reset_env = metadata.get("env_id") if cfg.n_envs > 1 else None + if reset_env is not None: + if isinstance(reset_env, bool) or not isinstance(reset_env, int): + raise ValueError( + f"reset env_id must be an int, got {reset_env!r} (BRG-5)" + ) + if not 0 <= reset_env < cfg.n_envs: + raise ValueError( + f"reset env_id {reset_env} outside [0, {cfg.n_envs}) (BRG-5)" + ) + teleport_reset(reset_seed, reset_env) + awaiting_first_reset = False + if reset_env is None: + # CON-5/ADR-25: re-anchor the publish-cadence grid to the + # reset, so which ticks fire the sub-100 Hz topics (poses, + # oracle_state, base_pose...) is a function of the episode, + # not of the wall tick the request happened to land on + scheduler = RateScheduler(topic_rates, dt) + if so101_latch is not None: + if so101_latch.held_name is not None: + print( + f"so101 carry release: {so101_latch.held_name} (reset)", + file=sys.stderr, + ) + so101_latch.reset() + if is_mobile: + # MOB-1/ADR-13: re-home the base to the store-frame start and + # drop the in-flight base command (mirrors the arm re-home). + # The robot ROOT moves too (PR #21): the tick handler re-bases + # only when the integrated pose CHANGES, so a variable-only + # re-home would leave the physical base at the pre-reset pose + base_pose = [ + float(v) for v in profile.get("base_start", [0.0, 0.0, 0.0]) + ] + base_cmd = [0.0, 0.0] + half = base_pose[2] / 2 + robot.set_pos( + np.array([base_pose[0], base_pose[1], 0.0], dtype=np.float32) ) - teleport_reset(reset_seed, reset_env) - awaiting_first_reset = False - if reset_env is None: - # CON-5/ADR-25: re-anchor the publish-cadence grid to the - # reset, so which ticks fire the sub-100 Hz topics (poses, - # oracle_state, base_pose...) is a function of the episode, - # not of the wall tick the request happened to land on - scheduler = RateScheduler(topic_rates, dt) - if so101_latch is not None: - if so101_latch.held_name is not None: - print( - f"so101 carry release: {so101_latch.held_name} (reset)", - file=sys.stderr, + robot.set_quat( + np.array([np.cos(half), 0.0, 0.0, np.sin(half)], dtype=np.float32) ) - so101_latch.reset() - if is_mobile: - # MOB-1/ADR-13: re-home the base to the store-frame start and - # drop the in-flight base command (mirrors the arm re-home). - # The robot ROOT moves too (PR #21): the tick handler re-bases - # only when the integrated pose CHANGES, so a variable-only - # re-home would leave the physical base at the pre-reset pose - base_pose = [float(v) for v in profile.get("base_start", [0.0, 0.0, 0.0])] - base_cmd = [0.0, 0.0] - half = base_pose[2] / 2 - robot.set_pos(np.array([base_pose[0], base_pose[1], 0.0], dtype=np.float32)) - robot.set_quat( - np.array([np.cos(half), 0.0, 0.0, np.sin(half)], dtype=np.float32) + if held_item is not None: + # a mid-carry reset hands the item back to physics: the + # latch would otherwise pin the respawned item to the hand + print(f"carry release: {held_item} (reset)", file=sys.stderr) + held_item = None + reply_env = reset_env if reset_env is not None else 0 + emit_bridge( + "reset_done", + pa.array(np.array([1], dtype=np.uint32)), + _metadata( + sim_time_ns, + reply_env, + seq.update( + { + ("reset_done", reply_env): seq.get( + ("reset_done", reply_env), 0 + ) + + 1 + } + ) + or seq[("reset_done", reply_env)], + request_id=metadata.get("request_id", ""), + seed=reset_seed, + mode=mode, + t_reset_ms=int((clock() - started) * 1000), + ), ) - if held_item is not None: - # a mid-carry reset hands the item back to physics: the - # latch would otherwise pin the respawned item to the hand - print(f"carry release: {held_item} (reset)", file=sys.stderr) - held_item = None - reply_env = reset_env if reset_env is not None else 0 - emit_bridge( - "reset_done", - pa.array(np.array([1], dtype=np.uint32)), - _metadata( - sim_time_ns, - reply_env, - seq.update( - { - ("reset_done", reply_env): seq.get(("reset_done", reply_env), 0) - + 1 - } - ) - or seq[("reset_done", reply_env)], - request_id=metadata.get("request_id", ""), - seed=reset_seed, - mode=mode, - t_reset_ms=int((clock() - started) * 1000), - ), - ) - # the injected state IS the post-reset observation: snapshot it - # before any physics step so the first oracle_state after reset - # is a pure function of the seed (TC-A2, CON-5); reset_done was - # already sent, so nothing interleaves the service pair (TC-6) - for topic in reset_publish_topics(topic_rates): - publish(topic) + # the injected state IS the post-reset observation: snapshot it + # before any physics step so the first oracle_state after reset + # is a pure function of the seed (TC-A2, CON-5); reset_done was + # already sent, so nothing interleaves the service pair (TC-6) + for topic in reset_publish_topics(topic_rates): + publish(topic) + + finally: + # the last partial timing window and the operator video, whatever + # ended the loop + step_timer.flush(sim_time_ns) + if debug_video is not None: + debug_video.close() if __name__ == "__main__": diff --git a/src/aisle/sim/__init__.py b/src/aisle/sim/__init__.py new file mode 100644 index 00000000..b3925b50 --- /dev/null +++ b/src/aisle/sim/__init__.py @@ -0,0 +1,180 @@ +"""Physics-engine selection for the bridge and the harness (ADR-67). + +The scene contract (SPEC 020) and the bridge contract (SPEC 030) are engine +neutral at the object level: the bridge talks to `robot`, entity, link and +camera objects through the small duck-typed surface Genesis exposes +natively. This package names the supported engines, resolves the attested +backend for each, and dispatches scene construction: + +- ``genesis``: the frozen `aisle.scenes` builders, unchanged (CON-7). +- ``nexus``: `aisle.sim.nexus_backend`, which rebuilds the same scenes from + the frozen pure functions (layout, placements, textures) on the Nexus + GPU engine behind the same object surface. +- ``rapier``: `aisle.sim.rapier_backend` (ADR-68), the same scenes stepped by + rapier on the CPU. It borrows the Nexus viewer as its renderer, so it needs + both wheels, and gains a deterministic single-threaded step in exchange. + +Nothing here imports a simulator at module level (CON-12): the engine name +alone is a plain string decision, and every simulator import happens inside +the builder that needs it. +""" + +from __future__ import annotations + +import os +from collections.abc import Mapping +from typing import Any + +ENGINES: tuple[str, ...] = ("genesis", "nexus", "rapier") +DEFAULT_ENGINE = "genesis" +ENGINE_ENV_VAR = "AISLE_SIM_ENGINE" + +# Backend names each engine accepts through AISLE_SIM_BACKEND (BRG-6). +# rapier steps on the CPU and takes no other backend; its renderer is the +# Nexus viewer, whose adapter choice is not the physics backend (ADR-68). +ENGINE_BACKENDS: dict[str, tuple[str, ...]] = { + "genesis": ("cpu", "metal", "cuda"), + "nexus": ("webgpu", "metal", "cuda", "cpu"), + "rapier": ("cpu",), +} + +# The importable module each engine needs. rapier needs two: the solver and +# the Nexus viewer it renders through. +ENGINE_MODULES: dict[str, tuple[str, ...]] = { + "genesis": ("genesis",), + "nexus": ("nexus3d",), + "rapier": ("rapier3d", "nexus3d"), +} + + +def normalize_engine(name: str | None) -> str: + """Validate an engine name; None means the default. Unknown names are + refused rather than defaulted, for the same reason TC-9 refuses an + unknown perception rung: a typo must not silently attest another engine.""" + if name is None: + return DEFAULT_ENGINE + engine = name.strip().lower() + if engine not in ENGINES: + raise ValueError(f"unknown simulation engine {name!r}; expected one of {ENGINES}") + return engine + + +def select_engine(env: Mapping[str, str] | None = None) -> str: + """The engine named by ``AISLE_SIM_ENGINE`` (default ``genesis``).""" + env = os.environ if env is None else env + return normalize_engine(env.get(ENGINE_ENV_VAR)) + + +def select_nexus_backend(sim_extra: str, platform_name: str, cuda_available: bool = False) -> str: + """Nexus counterpart of `select_genesis_backend`: the portable ``sim`` + selection maps to Metal on macOS and WebGPU elsewhere; ``cuda`` is the + Linux-only explicit opt-in and fails closed without a device.""" + if sim_extra == "sim": + return "metal" if platform_name == "Darwin" else "webgpu" + if sim_extra != "cuda": + raise ValueError(f"unknown simulation extra {sim_extra!r}; expected 'sim' or 'cuda'") + if platform_name != "Linux": + raise ValueError("the locked CUDA simulation extra is supported only on Linux") + if not cuda_available: + raise ValueError("the CUDA simulation extra requires an available CUDA device") + return "cuda" + + +def select_rapier_backend(sim_extra: str, platform_name: str, cuda_available: bool = False) -> str: + """rapier steps on the CPU on every platform (ADR-68), so the portable + extra resolves to ``cpu`` and the CUDA extra has nothing to select.""" + if sim_extra == "sim": + return "cpu" + if sim_extra != "cuda": + raise ValueError(f"unknown simulation extra {sim_extra!r}; expected 'sim' or 'cuda'") + raise ValueError("the rapier engine is CPU only; the CUDA extra selects no rapier backend") + + +def select_sim_backend( + engine: str, sim_extra: str, platform_name: str, cuda_available: bool = False +) -> str: + """Resolve (engine, extra, platform) to one attested backend name.""" + engine = normalize_engine(engine) + if engine == "genesis": + from aisle.scenes.pharmacy import select_genesis_backend + + return select_genesis_backend(sim_extra, platform_name, cuda_available) + if engine == "rapier": + return select_rapier_backend(sim_extra, platform_name, cuda_available) + return select_nexus_backend(sim_extra, platform_name, cuda_available) + + +def validate_backend(engine: str, backend: str | None) -> str | None: + """Refuse a backend name the engine does not know (bridge config).""" + engine = normalize_engine(engine) + if backend is not None and backend not in ENGINE_BACKENDS[engine]: + raise ValueError( + f"unknown simulation backend {backend!r} for engine {engine!r}; " + f"expected one of {ENGINE_BACKENDS[engine]}" + ) + return backend + + +def engine_available(engine: str) -> bool: + """Whether every Python package the engine needs is importable + (collection-safe: uses find_spec, never imports the simulator). rapier + needs its renderer too, so a missing nexus3d refuses it here rather than + at the first render (ADR-68).""" + import importlib.util + + modules = ENGINE_MODULES[normalize_engine(engine)] + return all(importlib.util.find_spec(module) is not None for module in modules) + + +def engine_version(engine: str) -> str: + """The installed simulator's version string, for `bridge_info`. The + rapier engine reports its solver's version; its renderer is attested + separately through the run's engine build receipt (ADR-68).""" + engine = normalize_engine(engine) + if engine == "genesis": + import genesis + + return str(genesis.__version__) + from importlib.metadata import PackageNotFoundError, version + + dist, module = ( + ("rapier3d", "rapier3d") if engine == "rapier" else ("dimforge-nexus3d", "nexus3d") + ) + try: + return version(dist) + except PackageNotFoundError: + import importlib + + return str(getattr(importlib.import_module(module), "__version__", "unknown")) + + +def build_scene(engine: str, *args: Any, **kwargs: Any): + """`aisle.scenes.pharmacy.build_scene` on the chosen engine.""" + engine = normalize_engine(engine) + if engine == "genesis": + from aisle.scenes.pharmacy import build_scene as genesis_build_scene + + return genesis_build_scene(*args, **kwargs) + if engine == "rapier": + from aisle.sim.rapier_backend import build_scene as rapier_build_scene + + return rapier_build_scene(*args, **kwargs) + from aisle.sim.nexus_backend import build_scene as nexus_build_scene + + return nexus_build_scene(*args, **kwargs) + + +def build_store(engine: str, *args: Any, **kwargs: Any): + """`aisle.scenes.store.build_store` on the chosen engine.""" + engine = normalize_engine(engine) + if engine == "genesis": + from aisle.scenes.store import build_store as genesis_build_store + + return genesis_build_store(*args, **kwargs) + if engine == "rapier": + from aisle.sim.rapier_backend import build_store as rapier_build_store + + return rapier_build_store(*args, **kwargs) + from aisle.sim.nexus_backend import build_store as nexus_build_store + + return nexus_build_store(*args, **kwargs) diff --git a/src/aisle/sim/nexus_backend.py b/src/aisle/sim/nexus_backend.py new file mode 100644 index 00000000..ee717df0 --- /dev/null +++ b/src/aisle/sim/nexus_backend.py @@ -0,0 +1,1357 @@ +"""Nexus engine backend (ADR-67): the frozen scenes, rebuilt on Nexus. + +`build_scene` / `build_store` here are the Nexus counterparts of the frozen +`aisle.scenes.pharmacy.build_scene` / `aisle.scenes.store.build_store`. They +call the SAME frozen pure functions for everything that decides the scene +(layout resolution, seeded placements, occlusion, colors, label textures, +camera mounts) and only replace the Genesis object construction. The +objects they return (scene, robot, entities, links, cameras) implement the +duck-typed surface the bridge and the frozen helpers already use on Genesis +objects: same method names, same argument shapes, quaternions in (w, x, y, +z) order, numpy arrays where Genesis returns tensors (`to_numpy` passes +them through). + +Nexus renders through its viewer: sensor cameras are offscreen surfaces on +the viewer's scene graph, so every camera concern (pose, attachment, +depth, segmentation) stays out of the physics engine. + +`nexus3d` is imported lazily (CON-12): importing this module is sim-free. +""" + +from __future__ import annotations + +import io +import math +import platform +import random +import tomllib +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +import numpy as np + +from aisle.embodiment import profile_dof_indices +from aisle.scenes.pharmacy import ( + FRANKA_EE_LINK, + FRANKA_MJCF, + SO101_URDF, + SceneCfg, + SceneHandle, + _assert_reachable, + _rotation_to_quat_wxyz, + apply_occlusion, + label_texture_image, + level_x_span, + load_meds, + load_physics, + resolve_layout, + sample_placements, + wrist_mount_transform, +) +from aisle.sim import select_nexus_backend + +_SIM_DIR = Path(__file__).parent +_SCENES_ASSETS = _SIM_DIR.parent / "scenes" / "assets" +_ENGINE: NexusEngine | None = None +_BACKGROUND_SEG_ID = -1 + + +def load_nexus_physics() -> dict: + with open(_SIM_DIR / "nexus_physics.toml", "rb") as f: + return tomllib.load(f) + + +def franka_mjcf_path() -> Path: + """The Genesis-bundled Franka MJCF (SCN-4), located without importing + genesis: the same file both engines load.""" + import importlib.util + + spec = importlib.util.find_spec("genesis") + if spec is None or not spec.submodule_search_locations: + raise FileNotFoundError("genesis is not installed; the franka MJCF ships with it (SCN-4)") + path = Path(next(iter(spec.submodule_search_locations))) / "assets" / FRANKA_MJCF + if not path.exists(): + raise FileNotFoundError(f"franka MJCF missing from the genesis package: {path}") + return path + + +# --- math helpers (numpy, float64; wire conversions happen in the callers) -- + + +def quat_wxyz_to_matrix(q) -> np.ndarray: + w, x, y, z = (float(v) for v in q) + return np.array( + [ + [1 - 2 * (y * y + z * z), 2 * (x * y - z * w), 2 * (x * z + y * w)], + [2 * (x * y + z * w), 1 - 2 * (x * x + z * z), 2 * (y * z - x * w)], + [2 * (x * z - y * w), 2 * (y * z + x * w), 1 - 2 * (x * x + y * y)], + ], + dtype=np.float64, + ) + + +def matrix_to_quat_wxyz(r) -> np.ndarray: + """Rotation matrix to (w, x, y, z): the frozen scene module's own + branch-stable conversion, so both engines agree bit for bit.""" + return _rotation_to_quat_wxyz(np.asarray(r, dtype=np.float64)) + + +def pos_lookat_up_to_transform(pos, lookat, up=(0.0, 0.0, 1.0)) -> np.ndarray: + """Genesis's look-at camera transform (OpenGL frame: -Z forward, +Y up): + `z = pos - lookat`, `x = cross(up, z)`, `y = cross(z, x)`. Mirrors the + pinned Genesis `_np_z_up_to_R`, including its degenerate branch, so the + realized calibration matches what VER-8 re-derives.""" + pos = np.asarray(pos, dtype=np.float64) + z = pos - np.asarray(lookat, dtype=np.float64) + norm = float(np.linalg.norm(z)) + if norm > 1e-6: + z = z / norm + else: + z = np.array([0.0, 1.0, 0.0]) if abs(up[1]) < 0.5 else np.array([0.0, 0.0, 1.0]) + x = np.cross(np.asarray(up, dtype=np.float64), z) + xn = float(np.linalg.norm(x)) + if xn > np.finfo(np.float32).eps: + x = x / xn + y = np.cross(z, x) + else: + # colinear z and up: Genesis falls back to the identity GL frame + x = np.array([1.0, 0.0, 0.0]) + y = np.array([0.0, 1.0, 0.0]) + z = np.array([0.0, 0.0, 1.0]) + transform = np.eye(4, dtype=np.float64) + transform[:3, 0], transform[:3, 1], transform[:3, 2], transform[:3, 3] = x, y, z, pos + return transform + + +def _per_dof(values, dofs: list, name: str) -> np.ndarray: + """Genesis gain-shape rule: one value for every addressed DoF, or one per + DoF. Any other length is a malformed gains profile, refused rather than + silently broadcasting its first entry.""" + values = np.atleast_1d(np.asarray(values, dtype=np.float64)) + if values.ndim != 1 or values.size not in (1, len(dofs)): + raise ValueError( + f"{name} has {values.size} entries for {len(dofs)} DoFs; give one or one per DoF" + ) + return values + + +def _broadcast_rows(values, n_envs: int, envs_idx) -> tuple[np.ndarray, list[int]]: + """Genesis call-shape rule: `(k,)` or `(1, k)` broadcasts to every + addressed env, `(n, k)` gives one row per addressed env.""" + rows = np.atleast_2d(np.asarray(values, dtype=np.float64)) + envs = list(range(n_envs)) if envs_idx is None else [int(e) for e in envs_idx] + if rows.shape[0] == 1 and len(envs) > 1: + rows = np.repeat(rows, len(envs), axis=0) + if rows.shape[0] != len(envs): + raise ValueError(f"{rows.shape[0]} rows for {len(envs)} environments") + return rows, envs + + +def _squeeze_envs(rows: np.ndarray, n_envs: int) -> np.ndarray: + """Genesis return-shape rule: single-env scenes return `(k,)`, batched + ones `(n_envs, k)`.""" + rows = np.asarray(rows, dtype=np.float32) + return rows[0] if n_envs == 1 else rows + + +def _png_bytes(image: np.ndarray) -> bytes: + from PIL import Image + + buffer = io.BytesIO() + Image.fromarray(np.asarray(image, dtype=np.uint8)).save(buffer, format="PNG") + return buffer.getvalue() + + +def checkerboard_texture(squares: int, color_a, color_b, px_per_square: int = 64) -> np.ndarray: + """An RGB checkerboard of `squares` x `squares` cells alternating the two + colors (RGB or RGBA in [0, 1]), for the floor slab's textured cube.""" + squares = max(1, int(squares)) + a = (np.asarray(color_a[:3], dtype=np.float64) * 255).round().astype(np.uint8) + b = (np.asarray(color_b[:3], dtype=np.float64) * 255).round().astype(np.uint8) + rows, cols = np.indices((squares, squares)) + cells = ((rows + cols) % 2).astype(bool) + image = np.where(cells[..., None], b, a).astype(np.uint8) + return np.repeat(np.repeat(image, px_per_square, axis=0), px_per_square, axis=1) + + +def load_uv_cube() -> tuple[list[list[float]], list[list[int]], list[list[float]]]: + """The committed UV-mapped unit cube (T2 labels) as (vertices, faces, + uvs) with one vertex per (position, uv) pair, as the renderer wants.""" + positions: list[list[float]] = [] + uvs: list[list[float]] = [] + verts: dict[tuple[int, int], int] = {} + out_pos: list[list[float]] = [] + out_uv: list[list[float]] = [] + faces: list[list[int]] = [] + for line in (_SCENES_ASSETS / "unit_cube_uv.obj").read_text().splitlines(): + parts = line.split() + if not parts or parts[0].startswith("#"): + continue + if parts[0] == "v": + positions.append([float(v) for v in parts[1:4]]) + elif parts[0] == "vt": + uvs.append([float(v) for v in parts[1:3]]) + elif parts[0] == "f": + face = [] + for token in parts[1:4]: + vi, ti = (int(t) - 1 for t in token.split("/")[:2]) + key = (vi, ti) + if key not in verts: + verts[key] = len(out_pos) + out_pos.append(positions[vi]) + out_uv.append(uvs[ti]) + face.append(verts[key]) + faces.append(face) + return out_pos, faces, out_uv + + +# --- engine (process-wide GPU context) ------------------------------------ + + +class NexusEngine: + """One headless Nexus viewer (the GPU context and render scene graph) and + compute pipeline per process, like `_ensure_genesis`'s one `gs.init`. + Scenes share it; the viewer's scene graph is cleared between builds.""" + + def __init__(self, backend_name: str, headless: bool = True): + import nexus3d + + self.nexus3d = nexus3d + self.backend_name = backend_name + self.headless = headless + viewer = nexus3d.NexusViewer(64, 64, headless=headless) + if backend_name == "metal": + if not hasattr(viewer, "with_metal"): + raise RuntimeError("the installed nexus3d was built without the metal feature") + viewer = viewer.with_metal() + elif backend_name == "cuda": + if not hasattr(viewer, "with_cuda"): + raise RuntimeError("the installed nexus3d was built without the cuda feature") + viewer = viewer.with_cuda() + elif backend_name == "cpu": + viewer = viewer.with_cpu() + viewer.init_backend() + # one node per body: the depth and segmentation passes only see + # non-instanced nodes, and per-body ids need them + viewer.set_sensor_rendering(True) + nexus_physics = load_nexus_physics() + camera = nexus_physics["camera"] + viewer.set_sensor_antialiasing(int(camera["msaa_samples"])) + viewer.set_sensor_shadow_softness(float(camera["shadow_softness"])) + viewer.set_sensor_shadow_range( + float(camera["shadow_first_cascade_m"]), float(camera["shadow_distance_m"]) + ) + viewer.set_sensor_shadow_resolution( + int(camera["shadow_resolution"]), int(camera["shadow_atlas_layers"]) + ) + viewer.add_directional_light(nexus3d.Vec3(*nexus_physics["light"]["direction"])) + self.viewer = viewer + self.pipeline = nexus3d.NexusPipeline() + self.pipeline.preload_pipelines(viewer) + self.scene_count = 0 + # the live scene's sensor cameras; each holds its own render targets + # and shadow atlas on the GPU until it is released + self.live_cameras: list[NexusCamera] = [] + + def release_cameras(self) -> None: + """Free the live scene's sensor cameras. Called when a new scene + supersedes it, whose cameras could no longer render anyway, so a + process can build any number of scenes without exhausting the GPU.""" + for camera in self.live_cameras: + camera.release() + self.live_cameras = [] + + +def _ensure_nexus(backend_name: str | None = None, headless: bool = True) -> NexusEngine: + """Nexus twin of `_ensure_genesis`: create the process-wide engine once, + refuse a later build asking for another backend (CON-5).""" + global _ENGINE + system = platform.system() + backend_name = backend_name or select_nexus_backend("sim", system) + if backend_name == "cuda" and system != "Linux": + raise ValueError("the CUDA backend is supported only on Linux") + if _ENGINE is None: + _ENGINE = NexusEngine(backend_name, headless=headless) + elif _ENGINE.backend_name != backend_name: + raise RuntimeError( + f"nexus already initialized with backend {_ENGINE.backend_name!r}; " + f"build_scene requires {backend_name!r}" + ) + return _ENGINE + + +# --- scene objects (the Genesis duck-typed surface) ------------------------ + + +class NexusEntity: + """A free rigid body per environment (a med box, a board, a tray).""" + + def __init__(self, scene: NexusScene, idx: int, name: str, handles: list, fixed: bool): + self.scene = scene + self.idx = idx + self.name = name + self.handles = handles # one RigidBodyHandle per env + self.fixed = fixed + self.link_bodies = [[h] for h in handles] # per env, for segmentation ids + + def _rows(self) -> np.ndarray: + pos, quat = self.scene.body_poses() + rows = [self.scene.gpu_index(env, h) for env, h in enumerate(self.handles)] + return pos[rows], quat[rows] + + def get_pos(self) -> np.ndarray: + return _squeeze_envs(self._rows()[0], self.scene.n_envs) + + def get_quat(self) -> np.ndarray: + return _squeeze_envs(self._rows()[1], self.scene.n_envs) + + def get_dofs_velocity(self) -> np.ndarray: + lin, ang = self.scene.body_velocities() + rows = [self.scene.gpu_index(env, h) for env, h in enumerate(self.handles)] + return _squeeze_envs(np.concatenate([lin[rows], ang[rows]], axis=-1), self.scene.n_envs) + + def set_pos(self, pos, envs_idx=None) -> None: + rows, envs = _broadcast_rows(pos, self.scene.n_envs, envs_idx) + _, quat = self._rows() + for row, env in zip(rows, envs, strict=True): + self.scene.set_body_pose(env, self.handles[env], row[:3], quat[env]) + + def set_quat(self, quat, envs_idx=None) -> None: + rows, envs = _broadcast_rows(quat, self.scene.n_envs, envs_idx) + pos, _ = self._rows() + for row, env in zip(rows, envs, strict=True): + self.scene.set_body_pose(env, self.handles[env], pos[env], row[:4]) + + def zero_all_dofs_velocity(self, envs_idx=None) -> None: + envs = range(self.scene.n_envs) if envs_idx is None else envs_idx + for env in envs: + self.scene.set_body_velocity(int(env), self.handles[int(env)]) + + +@dataclass +class NexusJoint: + name: str + n_dofs: int + dofs_idx_local: list[int] + + +class NexusLink: + """One robot link; poses come from the robot's GPU readback cache.""" + + def __init__(self, robot: NexusRobot, index: int, name: str): + self.robot = robot + self.index = index + self.name = name + + def get_pos(self) -> np.ndarray: + return _squeeze_envs(self.robot._link_poses()[0][:, self.index], self.robot.scene.n_envs) + + def get_quat(self) -> np.ndarray: + return _squeeze_envs(self.robot._link_poses()[1][:, self.index], self.robot.scene.n_envs) + + +class NexusRobot: + """One articulated robot, loaded once per environment.""" + + def __init__(self, scene: NexusScene, robots: list, name: str): + self.scene = scene + self.robots = robots # one nexus3d.Robot per env, identical layouts + self.name = name + first = robots[0] + self.n_dofs = int(first.n_dofs) + self.n_qs = self.n_dofs + self.links = [NexusLink(self, i, n) for i, n in enumerate(first.link_names)] + self.joints = [ + NexusJoint(name=n, n_dofs=int(nd), dofs_idx_local=list(range(off, off + nd))) + for n, nd, off in zip( + first.joint_names, first.joint_ndofs, first.joint_dof_offsets, strict=True + ) + ] + self.link_bodies = [list(r.link_body_handles) for r in robots] # per env + self._state_cache: dict[int, tuple] | None = None + # last root pose written by set_pos/set_quat, per env. The GPU link + # workspace only picks a written root up on the next step, so the + # second half of a set_pos + set_quat pair would otherwise read the + # stale root back and undo the first half (SPEC 210 re-basing). + self._root_writes: dict[int, tuple[np.ndarray, np.ndarray]] = {} + nexus_defaults = load_nexus_physics()["robot"] + for robot in robots: + robot.kp = [k if k > 0 else nexus_defaults["default_kp"] for k in robot.kp] + robot.kv = [v if v > 0 else nexus_defaults["default_kv"] for v in robot.kv] + self._targets = None + + # -- structure ------------------------------------------------------- + + def get_joint(self, name: str) -> NexusJoint: + for joint in self.joints: + if joint.name == name: + return joint + raise KeyError(f"robot {self.name!r} has no joint {name!r}") + + def get_link(self, name: str) -> NexusLink: + for link in self.links: + if link.name == name: + return link + raise KeyError(f"robot {self.name!r} has no link {name!r}") + + def get_dofs_limit(self) -> tuple[np.ndarray, np.ndarray]: + first = self.robots[0] + return ( + np.asarray(first.dof_lower, dtype=np.float32), + np.asarray(first.dof_upper, dtype=np.float32), + ) + + # -- state ----------------------------------------------------------- + + def invalidate(self) -> None: + self._state_cache = None + + def _state(self, env: int) -> tuple: + if self._state_cache is None: + self._state_cache = {} + if env not in self._state_cache: + engine = self.scene.engine + self._state_cache[env] = self.scene.state.robot_state(engine.viewer, self.robots[env]) + return self._state_cache[env] + + def _link_poses(self) -> tuple[np.ndarray, np.ndarray]: + states = [self._state(env) for env in range(self.scene.n_envs)] + return ( + np.stack([s[1] for s in states]), # (n_envs, n_links, 3) + np.stack([s[2] for s in states]), # (n_envs, n_links, 4) wxyz + ) + + def get_qpos(self) -> np.ndarray: + rows = np.stack([self._state(env)[0] for env in range(self.scene.n_envs)]) + return _squeeze_envs(rows, self.scene.n_envs) + + def get_dofs_velocity(self) -> np.ndarray: + """Generalized velocities, read back from the GPU dof state in the + same order as `get_qpos`.""" + viewer = self.scene.engine.viewer + rows = [ + np.asarray(self.scene.state.robot_qvel(viewer, self.robots[env]), dtype=np.float32) + for env in range(self.scene.n_envs) + ] + return _squeeze_envs(np.stack(rows), self.scene.n_envs) + + def set_qpos(self, qpos, envs_idx=None) -> None: + rows, envs = _broadcast_rows(qpos, self.scene.n_envs, envs_idx) + viewer = self.scene.engine.viewer + for row, env in zip(rows, envs, strict=True): + self.scene.state.set_robot_qpos(viewer, self.robots[env], [float(v) for v in row]) + self.scene.invalidate_poses() + + def zero_all_dofs_velocity(self, envs_idx=None) -> None: + """Re-send the simulated qpos: `set_robot_qpos` zeroes the generalized + velocities and the link body velocities on the GPU, so writing the + current coordinates back is how the robot is brought to rest without + moving it (the bridge calls this on every reset).""" + envs = range(self.scene.n_envs) if envs_idx is None else [int(e) for e in envs_idx] + for env in envs: + qpos = self._state(env)[0] + self.scene.state.set_robot_qpos( + self.scene.engine.viewer, self.robots[env], [float(v) for v in qpos] + ) + self.scene.invalidate_poses() + + def get_pos(self) -> np.ndarray: + return self.links[0].get_pos() + + def get_quat(self) -> np.ndarray: + return self.links[0].get_quat() + + def _root_pose(self, env: int) -> tuple[np.ndarray, np.ndarray]: + """The root pose to write the other half of a re-basing against: the + pending write if there is one, else the simulated pose.""" + pending = self._root_writes.get(env) + if pending is not None: + return pending + pos, quat = self._link_poses() + return np.asarray(pos[env, 0], dtype=np.float64), np.asarray(quat[env, 0], dtype=np.float64) + + def root_write_consumed(self) -> None: + """A step has read the written root out of the body buffer, so the + link readback is authoritative again.""" + self._root_writes.clear() + + def _write_root(self, env: int, pos, quat) -> None: + pos = np.asarray(pos, dtype=np.float64).reshape(-1)[:3] + quat = np.asarray(quat, dtype=np.float64).reshape(-1)[:4] + self.scene.set_body_pose(env, self.link_bodies[env][0], pos, quat) + self._root_writes[env] = (pos, quat) + self.invalidate() + + def set_pos(self, pos, envs_idx=None) -> None: + """Re-base the (fixed-root) robot: the GPU reads a fixed root's pose + from the body buffer every step (SPEC 210 mobile re-basing).""" + rows, envs = _broadcast_rows(pos, self.scene.n_envs, envs_idx) + for row, env in zip(rows, envs, strict=True): + self._write_root(env, row[:3], self._root_pose(env)[1]) + + def set_quat(self, quat, envs_idx=None) -> None: + rows, envs = _broadcast_rows(quat, self.scene.n_envs, envs_idx) + for row, env in zip(rows, envs, strict=True): + self._write_root(env, self._root_pose(env)[0], row[:4]) + + # -- control --------------------------------------------------------- + + def set_dofs_kp(self, kp, dofs_idx_local=None, envs_idx=None) -> None: + dofs = list(range(self.n_dofs)) if dofs_idx_local is None else list(dofs_idx_local) + for robot in self.robots: + robot.set_pd_gains(dofs, kp=[float(v) for v in _per_dof(kp, dofs, "kp")]) + + def set_dofs_kv(self, kv, dofs_idx_local=None, envs_idx=None) -> None: + dofs = list(range(self.n_dofs)) if dofs_idx_local is None else list(dofs_idx_local) + for robot in self.robots: + robot.set_pd_gains(dofs, kv=[float(v) for v in _per_dof(kv, dofs, "kv")]) + + def control_dofs_position(self, target, dofs_idx_local=None, envs_idx=None) -> None: + dofs = list(range(self.n_dofs)) if dofs_idx_local is None else list(dofs_idx_local) + rows, envs = _broadcast_rows(target, self.scene.n_envs, envs_idx) + viewer = self.scene.engine.viewer + for row, env in zip(rows, envs, strict=True): + self.scene.state.set_robot_targets( + viewer, self.robots[env], [float(v) for v in row[: len(dofs)]], dofs + ) + + # -- kinematics ------------------------------------------------------ + + def inverse_kinematics( + self, + link, + pos, + quat, + local_point=None, + init_qpos=None, + max_samples: int = 1, + max_solver_iters: int = 100, + rot_mask=None, + dofs_idx_local=None, + return_error: bool = False, + **_ignored, + ): + """Genesis-compatible IK on environment 0's CPU kinematic model. + + `rot_mask[k]` asks to align the link's k-th axis with the target's; + Nexus constrains world-frame rotation components instead, so the world + axis closest to the one free tool axis is left unconstrained. The + returned rotation error follows Genesis's masked-axis convention: + component k is the angle between the current and target k-th axes, + zero where the mask is False.""" + link_name = link.name if isinstance(link, NexusLink) else str(link) + pos = np.asarray(pos, dtype=np.float64).reshape(-1, 3)[0] + quat = np.asarray(quat, dtype=np.float64).reshape(-1, 4)[0] + mask = [True, True, True] if rot_mask is None else [bool(m) for m in rot_mask] + target_rot = quat_wxyz_to_matrix(quat) + constrained = [True, True, True, True, True, True] + if not all(mask): + free = [k for k, m in enumerate(mask) if not m] + if len(free) == 2: + aligned = mask.index(True) + # aligning one tool axis leaves rotation about it free: free + # the world axis it points along most + axis = target_rot[:, aligned] + constrained[3 + int(np.argmax(np.abs(axis)))] = False + else: + for k in free: + constrained[3 + k] = False + if init_qpos is None: + init = [float(v) for v in self._state(0)[0]] + else: + init = [ + float(v) + for v in np.asarray(init_qpos, dtype=np.float64).reshape(-1, self.n_dofs)[0] + ] + robot = self.robots[0] + qpos, error = self.scene.state.robot_inverse_kinematics( + robot, + link_name, + [float(v) for v in pos], + [float(v) for v in quat], + init, + local_point=None if local_point is None else [float(v) for v in local_point], + constrained_axes=constrained, + dofs=None if dofs_idx_local is None else [int(d) for d in dofs_idx_local], + max_iters=int(max_solver_iters), + ) + qpos = np.asarray(qpos, dtype=np.float32) + if return_error: + # Genesis convention: per masked axis, the angle between axes + fk_pos, fk_quat = self.scene.state.robot_forward_kinematics( + robot, list(map(float, qpos)), link_name + ) + current_rot = quat_wxyz_to_matrix(fk_quat) + point = ( + np.zeros(3) if local_point is None else np.asarray(local_point, dtype=np.float64) + ) + realized = np.asarray(fk_pos) + current_rot @ point + lin_err = pos - realized + rot_err = np.zeros(3) + for k, on in enumerate(mask): + if on: + cosine = float(np.clip(np.dot(current_rot[:, k], target_rot[:, k]), -1.0, 1.0)) + rot_err[k] = math.acos(cosine) + err = np.concatenate([lin_err, rot_err]).astype(np.float32) + if self.scene.n_envs > 1: + return np.tile(qpos, (self.scene.n_envs, 1)), np.tile(err, (self.scene.n_envs, 1)) + return qpos, err + return np.tile(qpos, (self.scene.n_envs, 1)) if self.scene.n_envs > 1 else qpos + + +class NexusCamera: + """An offscreen sensor camera on the viewer, in Genesis's camera terms: + `res` (w, h), vertical `fov` in degrees, a GL-convention `transform`, + `attach(link, offset_T)` and `render(rgb, depth, segmentation)`.""" + + def __init__(self, scene: NexusScene, cam_id: int, res: tuple[int, int], fov: float): + self.scene = scene + self.cam_id = cam_id + self.res = tuple(int(v) for v in res) + self.fov = fov + self._attached_link: NexusLink | None = None + self._attached_offset: np.ndarray | None = None + self._released_pose: tuple | None = None + + def release(self) -> None: + """Free the viewer camera, keeping its last pose readable.""" + self._released_pose = self.scene.engine.viewer.sensor_camera_pose(self.cam_id) + self.scene.engine.viewer.remove_sensor_camera(self.cam_id) + + @property + def transform(self) -> np.ndarray: + pos, quat = self._released_pose or self.scene.engine.viewer.sensor_camera_pose(self.cam_id) + transform = np.eye(4, dtype=np.float64) + transform[:3, :3] = quat_wxyz_to_matrix(quat) + transform[:3, 3] = pos + return transform + + def set_pose(self, transform=None, pos=None, lookat=None, up=(0.0, 0.0, 1.0)) -> None: + self.scene.activate() + if transform is None: + transform = pos_lookat_up_to_transform(pos, lookat, up) + transform = np.asarray(transform, dtype=np.float64) + self.scene.engine.viewer.set_sensor_camera_pose( + self.cam_id, + [float(v) for v in transform[:3, 3]], + [float(v) for v in matrix_to_quat_wxyz(transform[:3, :3])], + ) + + def attach(self, rigid_link: NexusLink, offset_T) -> None: + self._attached_link = rigid_link + self._attached_offset = np.asarray(offset_T, dtype=np.float64) + if self.scene.built: + self._apply_attachment() + + def _apply_attachment(self) -> None: + link, offset = self._attached_link, self._attached_offset + if link is None or offset is None: + return + body = link.robot.link_bodies[0][link.index] + self.scene.activate() + self.scene.engine.viewer.attach_sensor_camera( + self.cam_id, + 0, + body, + [float(v) for v in offset[:3, 3]], + [float(v) for v in matrix_to_quat_wxyz(offset[:3, :3])], + self.scene.state, + ) + + def render(self, rgb=True, depth=False, segmentation=False, **_ignored): + self.scene.sync() + rgb_arr, depth_arr, seg_arr = self.scene.engine.viewer.render_sensor_camera( + self.cam_id, rgb=bool(rgb), depth=bool(depth), segmentation=bool(segmentation) + ) + if depth_arr is not None: + # Genesis reads the cleared depth buffer, so a pixel that hit + # nothing comes back at the FAR plane; the renderer writes 0.0 + # there instead. The difference is not cosmetic: L2 back-projects + # a bounding-box mask, and a 0.0 sample lands at the camera, which + # is above every real surface, so the background captured the + # top-surface quantile and the grasp pose with it. + depth_arr = np.asarray(depth_arr, dtype=np.float32) + depth_arr[depth_arr <= 0.0] = np.float32(self.scene.camera_planes[1]) + if seg_arr is not None: + # Genesis: int64 ids keyed by `segmentation_idx_dict`, -1 for the + # background; the renderer's background is 0 and no scene id is 0 + seg_arr = np.asarray(seg_arr, dtype=np.int64) + seg_arr[seg_arr == 0] = _BACKGROUND_SEG_ID + return rgb_arr, depth_arr, seg_arr, None + + +class NexusScene: + """The Nexus counterpart of a Genesis `Scene`: bodies, robots and cameras + in `n_envs` batched environments, stepped by one compute pipeline.""" + + def __init__( + self, + engine: NexusEngine, + dt: float, + substeps: int, + gravity, + n_envs: int, + ambient, + background_rgba, + camera_planes: tuple[float, float], + ): + nx = engine.nexus3d + self.engine = engine + self.nexus3d = nx + self.n_envs = int(n_envs) + self.dt = float(dt) + self.gravity = [float(g) for g in gravity] + # per-channel, as Genesis's `ambient_light` takes it, so the lighting + # DR draw that `dr_applied` records is the one rendered + self.ambient = [float(c) for c in ambient] + self.background_rgba = [float(c) for c in background_rgba] + self.camera_planes = camera_planes + self.state = nx.NexusState() + self.friction_combine_rule = str( + load_nexus_physics()["sim"].get("friction_combine_rule", "average") + ) + # GPU timestamp queries: harvested by `sync`, read by `perf_stats` + self.timestamps = nx.GpuTimestamps(engine.viewer, 2048) + for _ in range(1, self.n_envs): + self.state.add_environment() + self.state.set_rbd_timestep(self.dt, int(substeps)) + self.entities: list[NexusEntity] = [] + self.robots: list[NexusRobot] = [] + self.cameras: list[NexusCamera] = [] + self.built = False + self.segmentation_idx_dict: dict[int, Any] = {} + self._pose_cache: tuple[np.ndarray, np.ndarray] | None = None + self._vel_cache: tuple[np.ndarray, np.ndarray] | None = None + self._synced = False + engine.scene_count += 1 + # one render generation per scene: a later build supersedes this + # scene's rendering (its nodes leave the shared graph and its cameras + # are freed), physics stays + engine.release_cameras() + self.generation = engine.viewer.begin_scene() + + # -- construction ---------------------------------------------------- + + def add_box( + self, + name: str, + size, + pos, + quat_wxyz=(1.0, 0.0, 0.0, 0.0), + fixed: bool = False, + friction: float = 0.5, + density: float | None = None, + color=(0.7, 0.7, 0.7, 1.0), + label_texture: np.ndarray | None = None, + ) -> NexusEntity: + nx = self.nexus3d + half = [float(s) / 2 for s in size] + w, x, y, z = (float(v) for v in quat_wxyz) + pose = nx.Pose.from_parts(nx.Vec3(*(float(p) for p in pos)), nx.Quat.from_xyzw(x, y, z, w)) + handles = [] + for env in range(self.n_envs): + builder = nx.RigidBodyBuilder.fixed() if fixed else nx.RigidBodyBuilder.dynamic() + body = builder.pose(pose).build() + # Genesis takes the larger friction of a touching pair; Nexus + # defaults to the average, so ask for the same rule + collider = ( + nx.ColliderBuilder.cuboid(*half) + .friction(float(friction)) + .friction_combine_rule(self.friction_combine_rule) + ) + if density is not None: + collider = collider.density(float(density)) + collider = collider.build() + handle = self.state.insert_rigid_body_in(env, body, collider, nx.RbdCoupling.NONE) + handles.append(handle) + if env == 0: + self._register_box_visual(handle, collider, size, color, label_texture, name) + entity = NexusEntity(self, len(self.entities), name, handles, fixed) + self.entities.append(entity) + return entity + + def _register_box_visual(self, handle, collider, size, color, label_texture, name) -> None: + nx = self.nexus3d + viewer = self.engine.viewer + rgba = [float(c) for c in color] + if len(rgba) == 3: + rgba.append(1.0) + if label_texture is None: + viewer.insert_sensor_shape(0, handle, collider.shared_shape(), nx.Pose.IDENTITY, rgba) + return + # T2 labels: the committed UV cube scaled to the med size, textured + # with the label image; collision stays the cuboid + positions, faces, uvs = load_uv_cube() + scaled = [[p[0] * size[0], p[1] * size[1], p[2] * size[2]] for p in positions] + mesh = nx.ColliderBuilder.trimesh(scaled, faces).build().shared_shape() + viewer.insert_sensor_shape( + 0, + handle, + mesh, + nx.Pose.IDENTITY, + [1.0, 1.0, 1.0, rgba[3]], + uvs=uvs, + texture=_png_bytes(label_texture), + texture_name=f"label-{name}", + ) + + def add_ground(self, size, friction: float) -> NexusEntity: + ground_cfg = load_nexus_physics()["ground"] + ground = self.add_box( + "ground", + size, + (0.0, 0.0, -float(size[2]) / 2), + fixed=True, + friction=friction, + color=tuple(ground_cfg["color"]), + label_texture=checkerboard_texture( + int(ground_cfg["checker_squares"]), + ground_cfg["color"], + ground_cfg["checker_color"], + ), + ) + if not ground_cfg.get("casts_shadows", True): + self.engine.viewer.set_body_casts_shadows(0, ground.handles[0], False) + return ground + + def add_urdf_robot(self, path: Path, name: str, convex_hull: bool) -> NexusRobot: + nx = self.nexus3d + options = nx.UrdfLoaderOptions(make_roots_fixed=True, convex_hull=convex_hull) + dynamics = load_nexus_physics()["robot"] + robots = [] + for env in range(self.n_envs): + robot = self.state.load_urdf_robot(env, str(path), options) + # URDF joints declare no armature or damping; without them the + # GPU solver has no stable PD hold on a light arm (read at build) + self.state.set_robot_joint_dynamics( + robot, + armature=[float(dynamics["default_armature"])] * robot.n_dofs, + damping=[float(dynamics["default_damping"])] * robot.n_dofs, + ) + if env == 0: + for body, shape, local_pose in robot.render_shapes: + self.engine.viewer.insert_visual_shape(0, body, shape, local_pose) + robots.append(robot) + wrapper = NexusRobot(self, robots, name) + self.robots.append(wrapper) + return wrapper + + def add_mjcf_robot(self, path: Path, name: str) -> NexusRobot: + robots = [ + self.state.load_mjcf_robot( + self.engine.viewer, env, str(path), register_visuals=env == 0 + ) + for env in range(self.n_envs) + ] + wrapper = NexusRobot(self, robots, name) + self.robots.append(wrapper) + return wrapper + + def add_camera(self, res, fov: float, pos=None, lookat=None) -> NexusCamera: + znear, zfar = self.camera_planes + cam_id = self.engine.viewer.add_sensor_camera( + int(res[0]), int(res[1]), float(fov), znear, zfar + ) + self.engine.viewer.set_sensor_camera_ambient(cam_id, 1.0) + self.engine.viewer.set_sensor_camera_ambient_color(cam_id, self.ambient) + self.engine.viewer.set_sensor_camera_background(cam_id, self.background_rgba) + camera = NexusCamera(self, cam_id, res, fov) + self.engine.live_cameras.append(camera) + if pos is not None and lookat is not None: + camera.set_pose(pos=pos, lookat=lookat) + self.cameras.append(camera) + return camera + + def build(self, n_envs: int | None = None) -> None: + if n_envs is not None and int(n_envs) != self.n_envs: + raise ValueError("n_envs is fixed at scene construction for the nexus backend") + nx = self.nexus3d + viewer = self.engine.viewer + self.state.finalize(viewer) + self.state.set_rbd_gravity(viewer, nx.Vec3(*self.gravity)) + # link poses are only derived from the joint coordinates by the first + # step: write them now so readbacks before any step are consistent + for robot in self.robots: + for per_env in robot.robots: + self.state.set_robot_qpos(viewer, per_env, list(self.state.robot_cpu_qpos(per_env))) + self.built = True + self._assign_segmentation_ids() + self.sync(force=True) + for camera in self.cameras: + camera._apply_attachment() + self.sync(force=True) + + def _assign_segmentation_ids(self) -> None: + """Genesis's default `link` segmentation level: one id per (entity, + link); `segmentation_idx_dict` maps id -> (entity_idx, link_idx) with + -1 for the background (TC-9). Ids start at 1: 0 is the renderer's + background, which `render` maps to -1.""" + viewer = self.engine.viewer + idx_dict: dict[int, Any] = {_BACKGROUND_SEG_ID: _BACKGROUND_SEG_ID} + seg_id = 1 + for entity in self.entities: + viewer.set_body_segmentation_id(0, entity.handles[0], seg_id) + idx_dict[seg_id] = (entity.idx, 0) + seg_id += 1 + for robot in self.robots: + entity_idx = len(self.entities) + self.robots.index(robot) + for link_idx, body in enumerate(robot.link_bodies[0]): + viewer.set_body_segmentation_id(0, body, seg_id) + idx_dict[seg_id] = (entity_idx, link_idx) + seg_id += 1 + self.segmentation_idx_dict = idx_dict + + # -- stepping and readback ------------------------------------------- + + def step(self) -> None: + self.engine.pipeline.simulate(self.engine.viewer, self.state, self.timestamps) + self.invalidate_poses() + for robot in self.robots: + robot.root_write_consumed() + + def perf_stats(self) -> dict: + """Engine-side timing of the last step: `gpu_ms` is the summed GPU + pass time from the timestamp queries (lags a frame or two behind + the step that produced it) and `gpu_passes` the per-pass split. + Empty until the first harvest. The bridge writes these into its + timing sidecar so engines can be compared on the same run.""" + # harvesting happens in the viewer sync, which the render path also + # needs; a step-only tick would otherwise never see fresh timings + self.sync() + stats = self.state.run_stats() + gpu_ms = float(stats.get("gpu_total_time_ms", 0.0)) + if gpu_ms <= 0.0: + return {} + return { + "gpu_ms": gpu_ms, + "gpu_passes": {str(k): float(v) for k, v in stats.get("gpu_pass_times", {}).items()}, + "encoding_ms": float(stats.get("encoding_time_ms", 0.0)), + } + + def activate(self) -> None: + """Refuse to render a scene a newer build superseded: the shared + viewer draws one scene at a time (the latest), and its render nodes + left the graph when the next scene began. Physics readbacks still + work on a superseded scene; only cameras do not.""" + if self.engine.viewer.active_scene() != self.generation: + raise RuntimeError( + "this nexus scene was superseded by a newer build_scene in this process; " + "its cameras can no longer render (one live scene per process)" + ) + + def sync(self, force: bool = False) -> None: + """Pull GPU poses into the render scene graph (and the attached + cameras) once per step, before any render.""" + self.activate() + if force or not self._synced: + self.engine.viewer.sync(self.state, self.timestamps) + self._synced = True + + def invalidate_poses(self) -> None: + self._pose_cache = None + self._vel_cache = None + self._synced = False + for robot in self.robots: + robot.invalidate() + + def body_poses(self) -> tuple[np.ndarray, np.ndarray]: + if self._pose_cache is None: + pos, quat = self.state.read_body_poses(self.engine.viewer) + self._pose_cache = ( + np.asarray(pos, dtype=np.float32), + np.asarray(quat, dtype=np.float32), + ) + return self._pose_cache + + def body_velocities(self) -> tuple[np.ndarray, np.ndarray]: + if self._vel_cache is None: + lin, ang = self.state.read_body_velocities(self.engine.viewer) + self._vel_cache = (np.asarray(lin, dtype=np.float32), np.asarray(ang, dtype=np.float32)) + return self._vel_cache + + def gpu_index(self, env: int, handle) -> int: + index = self.state.body_gpu_index(env, handle) + if index is None: + raise RuntimeError("scene not built (call build() first)") + return int(index) + + def set_body_pose(self, env: int, handle, pos, quat_wxyz) -> None: + self.state.set_body_pose( + self.engine.viewer, env, handle, [float(v) for v in pos], [float(v) for v in quat_wxyz] + ) + # a teleport writes the pose in two halves (set_pos then set_quat) and + # each reads back the other, so dropping the cache here cost two GPU + # readbacks per body on every reset. The written row is known: patch + # it instead, and keep the velocity cache, which the write leaves alone + self._synced = False + if self._pose_cache is not None: + row = self.gpu_index(env, handle) + self._pose_cache[0][row] = pos + self._pose_cache[1][row] = quat_wxyz + for robot in self.robots: + robot.invalidate() + + def set_body_velocity( + self, env: int, handle, linvel=(0.0, 0.0, 0.0), angvel=(0.0, 0.0, 0.0) + ) -> None: + self.state.set_body_velocity( + self.engine.viewer, env, handle, [float(v) for v in linvel], [float(v) for v in angvel] + ) + self._vel_cache = None + + +# --- scene builders (mirror the frozen Genesis builders) -------------------- + + +def _new_scene(engine: NexusEngine, physics: dict, n_envs: int, ambient) -> NexusScene: + nexus_physics = load_nexus_physics() + scene = NexusScene( + engine, + dt=physics["sim"]["dt"], + # the engine's own substep count: the frozen value is Genesis's + substeps=nexus_physics["sim"]["substeps"], + gravity=physics["sim"]["gravity"], + n_envs=n_envs, + ambient=ambient, + background_rgba=nexus_physics["camera"]["background_rgba"], + camera_planes=(nexus_physics["camera"]["znear"], nexus_physics["camera"]["zfar"]), + ) + sim = nexus_physics["sim"] + scene.state.set_rbd_solver_params( + contact_natural_frequency=float(sim["contact_natural_frequency"]), + static_contact_natural_frequency=float(sim["static_contact_natural_frequency"]), + allowed_linear_error=float(sim["allowed_linear_error"]), + internal_pgs_iterations=int(sim["internal_pgs_iterations"]), + friction_in_bias_pass=bool(sim.get("friction_in_bias_pass", False)), + ) + scene.state.set_rbd_implicit_coriolis(engine.viewer, bool(sim.get("implicit_coriolis", True))) + scene.state.set_rbd_substep_refresh( + engine.viewer, + bool(sim.get("substep_refresh", True)), + bool(sim.get("substep_refresh_light", False)), + ) + return scene + + +def _add_robot(scene: NexusScene, embodiment: str, physics: dict) -> NexusRobot: + nexus_physics = load_nexus_physics() + if embodiment in ("franka", "mobile"): + return scene.add_mjcf_robot(franka_mjcf_path(), "franka") + if not SO101_URDF.exists(): + raise FileNotFoundError(f"so101 asset missing: {SO101_URDF} (acquisition pending, ADR-6)") + return scene.add_urdf_robot(SO101_URDF, embodiment, bool(nexus_physics["robot"]["convex_hull"])) + + +def _apply_home_and_gains(robot: NexusRobot, profile: dict, n_envs: int) -> None: + """The frozen builder's post-build robot setup, verbatim in behavior: + home pose in TC-5 wire order mapped by joint name, then the profile's + gripper gains.""" + wire_dof_indices = profile_dof_indices(robot, profile) + if "home_qpos" in profile: + home = np.asarray(profile["home_qpos"], dtype=np.float32) + if wire_dof_indices is not None: + native_home = np.empty(robot.n_dofs, dtype=np.float32) + native_home[list(wire_dof_indices)] = home + home = native_home + robot.set_qpos(home if n_envs == 1 else np.tile(home, (n_envs, 1))) + if "gripper_dofs" in profile and "gripper_kp" in profile: + if wire_dof_indices is None: + count = int(profile["gripper_dofs"]) + finger_dofs = list(range(robot.n_dofs - count, robot.n_dofs)) + else: + count = len(profile["gripper_joint_names"]) + finger_dofs = list(wire_dof_indices[-count:]) + robot.set_dofs_kp( + np.asarray(profile["gripper_kp"], dtype=np.float32), dofs_idx_local=finger_dofs + ) + robot.set_dofs_kv( + np.asarray(profile["gripper_kv"], dtype=np.float32), dofs_idx_local=finger_dofs + ) + + +def build_scene( + seed: int, + embodiment: str = "franka", + n_envs: int = 1, + headless: bool = True, + cfg: SceneCfg | None = None, + sim_backend: str | None = None, +) -> SceneHandle: + """Nexus twin of `aisle.scenes.pharmacy.build_scene` (SPEC 020): the same + seeded layout, DR draws, colors, labels and cameras, realized on Nexus. + Every scene-deciding value comes from the frozen module's pure functions.""" + cfg = cfg or SceneCfg() + engine = _ensure_nexus(sim_backend, headless=headless) + physics = load_physics() + nexus_physics = load_nexus_physics() + layout = resolve_layout(physics, embodiment) + meds = load_meds() + shelf, tray_cfg = layout["shelf"], layout["tray"] + dr_cfg = physics["domain_randomization"] + + for label, target in (("tray", tray_cfg["pos"]), ("shelf", shelf["pos"])): + distance = math.hypot(*target) + assert distance <= layout["reach_m"], ( + f"{label} at {target} outside {embodiment} workspace (SCN-4)" + ) + + lighting_rng = random.Random(cfg.lighting.seed) + textures_rng = random.Random(cfg.textures.seed) + friction_rng = random.Random(cfg.friction_jitter.seed) + camera_rng = random.Random(cfg.camera_jitter.seed) + + ambient = (dr_cfg["ambient_default"],) * 3 + if cfg.lighting.enabled: + ambient = tuple( + min(1.0, dr_cfg["ambient_min"] + lighting_rng.random() * dr_cfg["ambient_range"]) + for _ in range(3) + ) + + scene = _new_scene(engine, physics, n_envs, ambient) + ground = nexus_physics["ground"] + scene.add_ground(ground["size"], ground["friction"]) + + shelf_friction = physics["materials"]["shelf"]["friction"] + width = shelf["level_size"][1] + for level, (level_height, level_depth) in enumerate( + zip(shelf["level_heights"], shelf["level_depths"], strict=True) + ): + x_min, x_max = level_x_span(shelf, level) + scene.add_box( + f"shelf_{level}", + (level_depth, width, shelf["board_thickness"]), + ((x_min + x_max) / 2, shelf["pos"][1], shelf["pos"][2] + level_height), + fixed=True, + friction=shelf_friction, + ) + tray = scene.add_box( + "tray", + tuple(tray_cfg["size"]), + tuple(tray_cfg["pos"]), + fixed=True, + friction=physics["materials"]["tray"]["friction"], + ) + + robot = _add_robot(scene, embodiment, physics) + + box_physics = physics["materials"]["box"] + applied_frictions: dict[str, float] = {} + applied_colors: dict[str, list[float]] = {} + boxes: dict[str, Any] = {} + color_by_med = {name: meds[name]["color"] for name in meds} + if cfg.shuffle_colors: + names = list(meds) + shuffled = list(np.random.default_rng(seed ^ 0x5EED).permutation(names)) + color_by_med = { + name: meds[donor]["color"] for name, donor in zip(names, shuffled, strict=True) + } + placements = sample_placements(seed, list(meds), layout) + if cfg.occlusion: + placements = apply_occlusion(placements, seed, list(meds), layout) + for placement in placements: + friction = box_physics["friction"] + if cfg.friction_jitter.enabled: + friction *= 1.0 + (friction_rng.random() - 0.5) * dr_cfg["friction_jitter_frac"] + applied_frictions[placement.name] = friction + color = list(color_by_med[placement.name]) + if cfg.textures.enabled: + scale_min, scale_range = dr_cfg["texture_scale_min"], dr_cfg["texture_scale_range"] + color = [ + min(1.0, c * (scale_min + textures_rng.random() * scale_range)) for c in color[:3] + ] + [color[3]] + applied_colors[placement.name] = color + label_texture = None + if cfg.labels: + label_texture = label_texture_image( + str(meds[placement.name].get("label", placement.name.upper())), color + ) + boxes[placement.name] = scene.add_box( + placement.name, + tuple(meds[placement.name]["size"]), + (placement.x, placement.y, placement.z), + friction=friction, + density=box_physics["density_kg_m3"], + color=tuple(color), + label_texture=label_texture, + ) + + cam_cfg = physics["cameras"] + overhead_pos = list(cam_cfg["overhead_pos"]) + if cfg.camera_jitter.enabled: + jitter = dr_cfg["camera_jitter_m"] + overhead_pos = [p + (camera_rng.random() - 0.5) * jitter for p in overhead_pos] + cams = { + "overhead": scene.add_camera( + (640, 480), 55, pos=tuple(overhead_pos), lookat=tuple(cam_cfg["overhead_lookat"]) + ), + "wrist": scene.add_camera((320, 240), 70), + } + + scene.build() + + profile = physics["embodiment"][embodiment] + _apply_home_and_gains(robot, profile, n_envs) + + ee_link = robot.get_link(profile.get("ee_link", FRANKA_EE_LINK)) + offset = wrist_mount_transform(cam_cfg, profile) + cams["wrist"].attach(ee_link, offset_T=offset) + + handle = SceneHandle( + scene=scene, + robot=robot, + boxes=boxes, + tray=tray, + cams=cams, + embodiment=embodiment, + seed=seed, + med_sizes={name: list(meds[name]["size"]) for name in meds}, + dr_applied={ + "ambient": ambient, + "overhead_pos": overhead_pos, + "frictions": applied_frictions, + "colors": applied_colors, + }, + ) + _assert_reachable(handle, ee_link, layout, n_envs) + return handle + + +def build_store( + seed: int, + scenario: str, + embodiment: str = "mobile", + n_envs: int = 1, + headless: bool = True, + sim_backend: str | None = None, +): + """Nexus twin of `aisle.scenes.store.build_store` (SPEC 200 RS-2/RS-3).""" + from aisle.scenes.store import ( + StoreHandle, + episode_layout, + full_stock, + generate_episode, + load_planogram, + yaw_quat_wxyz, + ) + + engine = _ensure_nexus(sim_backend, headless=headless) + meds = load_meds() + physics = load_physics() + nexus_physics = load_nexus_physics() + plano = load_planogram() + episode = generate_episode(seed, scenario) + store, geo = plano["store"], plano["store"]["unit_geometry"] + profile = physics["embodiment"][embodiment] + + scene = _new_scene( + engine, physics, n_envs, (physics["domain_randomization"]["ambient_default"],) * 3 + ) + ground = nexus_physics["ground"] + scene.add_ground(ground["size"], ground["friction"]) + + shelf_friction = physics["materials"]["shelf"]["friction"] + for unit_id, unit in plano["units"].items(): + for level, level_height in enumerate(geo["level_heights"]): + scene.add_box( + f"{unit_id}_level_{level}", + (geo["depth"], geo["width"], geo["board_thickness"]), + (unit["pos"][0], unit["pos"][1], level_height), + quat_wxyz=yaw_quat_wxyz(unit["yaw"]), + fixed=True, + friction=shelf_friction, + ) + tray_friction = physics["materials"]["tray"]["friction"] + counter = scene.add_box( + "counter", + tuple(store["counter_size"]), + tuple(store["counter_pos"]), + fixed=True, + friction=tray_friction, + ) + bin_entity = scene.add_box( + "bin", tuple(store["bin_size"]), tuple(store["bin_pos"]), fixed=True, friction=tray_friction + ) + + robot = scene.add_mjcf_robot(franka_mjcf_path(), "franka") + + box_physics = physics["materials"]["box"] + items: dict[str, Any] = {} + categories: dict[str, str] = {} + layout = episode_layout(plano, episode, meds) + for item in full_stock(plano): + x, y, z, yaw = layout[item.item_id] + categories[item.item_id] = item.category + items[item.item_id] = scene.add_box( + item.item_id, + tuple(meds[item.category]["size"]), + (x, y, z), + quat_wxyz=yaw_quat_wxyz(yaw), + friction=box_physics["friction"], + density=box_physics["density_kg_m3"], + color=tuple(meds[item.category]["color"]), + ) + + cam_cfg = physics["cameras"] + cams = { + "overhead": scene.add_camera( + (640, 480), + 70, + pos=tuple(cam_cfg["store_overhead_pos"]), + lookat=tuple(cam_cfg["store_overhead_lookat"]), + ), + "wrist": scene.add_camera((320, 240), 70), + } + + scene.build() + + home = np.asarray(profile["home_qpos"], dtype=np.float32) + robot.set_qpos(home if n_envs == 1 else np.tile(home, (n_envs, 1))) + count = int(profile["gripper_dofs"]) + finger_dofs = list(range(robot.n_dofs - count, robot.n_dofs)) + robot.set_dofs_kp( + np.asarray(profile["gripper_kp"], dtype=np.float32), dofs_idx_local=finger_dofs + ) + robot.set_dofs_kv( + np.asarray(profile["gripper_kv"], dtype=np.float32), dofs_idx_local=finger_dofs + ) + + ee_link = robot.get_link("hand") + offset = np.eye(4, dtype=np.float32) + offset[:3, 3] = cam_cfg["wrist_offset_m"] + cams["wrist"].attach(ee_link, offset_T=offset) + + return StoreHandle( + scene=scene, + robot=robot, + items=items, + categories=categories, + counter=counter, + bin=bin_entity, + cams=cams, + planogram=plano, + episode=episode, + embodiment=embodiment, + seed=seed, + scenario=scenario, + med_sizes={name: list(meds[name]["size"]) for name in meds}, + ) + + +__all__ = [ + "NexusCamera", + "NexusEngine", + "NexusEntity", + "NexusJoint", + "NexusLink", + "NexusRobot", + "NexusScene", + "build_scene", + "build_store", + "franka_mjcf_path", + "load_nexus_physics", + "load_uv_cube", + "matrix_to_quat_wxyz", + "pos_lookat_up_to_transform", + "quat_wxyz_to_matrix", +] diff --git a/src/aisle/sim/nexus_physics.toml b/src/aisle/sim/nexus_physics.toml new file mode 100644 index 00000000..f9a0e1ae --- /dev/null +++ b/src/aisle/sim/nexus_physics.toml @@ -0,0 +1,102 @@ +# Nexus-engine constants (SCN-2 spirit: no physics constants inline in code). +# Kept outside the frozen scene directory: these describe how the Nexus +# backend realizes the frozen scene configuration, not the scene itself. + +[sim] +# Solver substeps per 10 ms step. physics.toml's `substeps = 1` is tuned for +# Genesis. Replaying the recorded T0 pinch (tools/nexus_grasp_replay.py) after +# the Nexus solver fixes of September 2026 (duplicated multibody contacts, +# interleaved PGS sweeps, nearest-point contact warmstarting): the box rides +# the hand with a 2.5 to 3.2 degree tilt at 8, 4, 2 substeps and even at 1 +# with softer contacts; resting boxes drift 0.001 mm in 3 s from 2 substeps +# up. GPU per step: 9 ms at 8, 5 at 4, 3 at 2 substeps. 2 keeps the stiff +# contacts below with a wide margin (4 substeps held with one PGS iteration). +substeps = 2 +# Contact solver. 240 Hz soft contacts with 0.5 mm allowed penetration keep +# the pinch penetration at Genesis's scale (about 1 mm per pad under 60 N); +# Nexus's defaults (30 Hz, 5 mm) also hold the box at one substep but sink the +# pads several millimeters. Eight PGS iterations cost little at 2 substeps +# and leave margin for heavier objects. Frequencies in Hz, lengths in meters. +contact_natural_frequency = 240.0 +static_contact_natural_frequency = 480.0 +allowed_linear_error = 0.0005 +internal_pgs_iterations = 8 +# Solve the friction rows in every biased PGS iteration, not only in the +# once-per-substep stabilization sweep (rapier's default scheduling, under +# which friction got one iteration per substep against eight for the +# normals: the pinched box tilts 5.8 degrees instead of 2.6). +friction_in_bias_pass = true +# Explicit Coriolis and gyroscopic terms for the robots: the multibody mass +# matrix is then refreshed once per step instead of every substep. Nexus's +# default is implicit. +implicit_coriolis = false +# Multibody refresh cadence. Nexus's default (substep_refresh = true) rebuilds +# the robots' constraints, mass matrix and LU factors every substep. The +# light cadence keeps the constraints per substep but the mass matrix and LU +# per step: the replayed T0 pinch holds exactly as with the default (2.5 +# degree tilt) at about 16% less GPU time per step (4.2 ms against 4.9 ms). +# Both off (constraints per step too) blows the robot up within a few steps +# on this scene, so it stays out. +substep_refresh = false +substep_refresh_light = true +# Genesis combines the friction of two touching geoms with max(); Nexus's +# default is the average. Match Genesis so the box/finger pair sees the same +# coefficient on both engines. +friction_combine_rule = "max" + +[ground] +# Nexus has no infinite plane primitive; the ground is a fixed slab whose +# top face sits at z = 0. A slate checkerboard like Genesis's floor: +# `checker_squares` squares across the slab (20 m / 40 = 0.5 m squares) in +# `color` and `checker_color`, drawn as a texture on the collision slab. +size = [20.0, 20.0, 0.1] +friction = 1.0 +color = [0.30, 0.33, 0.38, 1.0] +checker_color = [0.42, 0.46, 0.52, 1.0] +checker_squares = 40 +# The slab receives shadows but casts none (nothing sits below it), which +# keeps it out of the shadow depth passes. +casts_shadows = false + +[robot] +# Genesis's URDF defaults (`default_dofs_kp` / `default_dofs_kv`): joints +# without an explicit profile gain get these. MJCF position actuators keep +# the gains declared in the model, as in Genesis. +default_kp = 100.0 +default_kv = 10.0 +# Per-DoF reflected rotor inertia and viscous damping given to URDF joints, +# which declare neither (MJCF models carry their own). A light arm without +# them has no stable PD hold on the GPU solver; these are the values the +# official SO-101 MJCF uses for its STS3215 servos. +default_armature = 0.01 +default_damping = 0.1 +# Robot mesh colliders: a convex hull per mesh (Genesis's `convexify`). +convex_hull = true + +[camera] +znear = 0.01 +zfar = 50.0 +# Sky color behind the geometry, dark like Genesis's. Depth reads `zfar` +# where nothing was hit, which is what Genesis's rasterizer returns from its +# cleared depth buffer; the renderer writes 0.0 there and the camera remaps it. +background_rgba = [0.07, 0.08, 0.12, 1.0] +# Sensor and debug renders: 4x MSAA (1 disables antialiasing) and kiss3d's +# default PCF penumbra (1.0, about five shadow texels wide, close to the 3x3 +# bilinear PCF Genesis's rasterizer applies; 0.0 gives hard edges). The +# sharpest directional cascade is fit to the camera's first 2.5 m (the desk) +# instead of kiss3d's 12 m, and shadows stop 20 m out, so the penumbra stays +# a few millimeters wide rather than smearing over centimeters. +msaa_samples = 4 +shadow_softness = 1.0 +shadow_first_cascade_m = 2.5 +shadow_distance_m = 20.0 +# Shadow map texels per atlas layer and layers allocated: the scene has one +# directional light, whose four cascades fit four layers (kiss3d's default is +# 16), so 4096 texels cost the same memory as 2048 over the full atlas. +shadow_resolution = 4096 +shadow_atlas_layers = 4 + +[light] +# One directional light on the shared scene graph, pointing down at the +# workspace of the Z-up scene. +direction = [0.3, -0.4, -1.0] diff --git a/src/aisle/sim/rapier_backend.py b/src/aisle/sim/rapier_backend.py new file mode 100644 index 00000000..1baaac4e --- /dev/null +++ b/src/aisle/sim/rapier_backend.py @@ -0,0 +1,1228 @@ +"""rapier engine backend (ADR-68): the frozen scenes, stepped by rapier on the CPU. + +`build_scene` / `build_store` here are the rapier counterparts of the frozen +`aisle.scenes.pharmacy.build_scene` / `aisle.scenes.store.build_store`, and of +their Nexus twins in `nexus_backend`. They call the SAME frozen pure functions +for everything that decides the scene (layout resolution, seeded placements, +occlusion, colors, label textures, camera mounts) and only replace the object +construction, behind the same duck-typed surface the bridge already uses on +Genesis objects. + +The split that makes this engine possible: rapier owns the physics, the Nexus +viewer owns the pixels. Every scene is built twice, into one +`rapier3d.PhysicsWorld` per environment and into one Nexus scene that is built +exactly as the Nexus backend builds it and never stepped. A step advances +rapier; the resulting poses are written into the Nexus state just before the +viewer syncs, so RGB, metric depth and segmentation follow externally computed +poses. Two consequences are worth naming: a rapier run and a Nexus run share +the identical renderer, camera model and segmentation ids, and this backend +needs both wheels. + +The render mirror is reused, not forked: `RapierScene` composes a `NexusScene` +and calls its `add_box`, `add_ground`, `add_mjcf_robot`, `add_urdf_robot`, +`add_camera` and `build`, so textures, shadows, cameras and the segmentation +numbering are literally the Nexus backend's. + +`rapier3d` and `nexus3d` are imported lazily (CON-12): importing this module is +sim-free. +""" + +from __future__ import annotations + +import math +import random +import tomllib +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +import numpy as np + +from aisle.embodiment import profile_dof_indices +from aisle.scenes.pharmacy import ( + FRANKA_EE_LINK, + SO101_URDF, + SceneCfg, + SceneHandle, + _assert_reachable, + apply_occlusion, + label_texture_image, + level_x_span, + load_meds, + load_physics, + resolve_layout, + sample_placements, + wrist_mount_transform, +) +from aisle.sim.nexus_backend import ( + NexusScene, + _broadcast_rows, + _ensure_nexus, + _per_dof, + _squeeze_envs, + franka_mjcf_path, + quat_wxyz_to_matrix, +) + +_SIM_DIR = Path(__file__).parent + + +def load_rapier_physics() -> dict: + with open(_SIM_DIR / "rapier_physics.toml", "rb") as f: + return tomllib.load(f) + + +# --- small conversions between rapier types and the wire's numpy arrays ----- + + +def _xyz(vec) -> np.ndarray: + return np.array([float(vec[0]), float(vec[1]), float(vec[2])], dtype=np.float64) + + +def _wxyz(rot) -> np.ndarray: + return np.array([float(rot.w), float(rot.i), float(rot.j), float(rot.k)], dtype=np.float64) + + +def _quat_xyzw(quat_wxyz) -> tuple[float, float, float, float]: + w, x, y, z = (float(v) for v in quat_wxyz) + return (x, y, z, w) + + +def _combine_rule(name: str): + """The rapier enum for a friction-combine rule named in the toml.""" + import rapier3d as rp + + try: + return getattr(rp.CoefficientCombineRule, str(name).upper()) + except AttributeError as exc: + raise ValueError(f"unknown friction_combine_rule {name!r}") from exc + + +def _isometry(pos, quat_wxyz) -> tuple: + """rapier accepts a (translation, rotation) pair wherever it wants a pose.""" + import rapier3d as rp + + return ( + rp.Vec3(float(pos[0]), float(pos[1]), float(pos[2])), + rp.Quaternion.from_tuple(_quat_xyzw(quat_wxyz)), + ) + + +# --- the render mirror ------------------------------------------------------ + + +class RapierRenderMirror(NexusScene): + """The Nexus scene a rapier scene renders through: built by the Nexus + backend's own constructors, never stepped. `sync` is the one hook the + rapier side needs, because every render path goes through it.""" + + physics: Any = None + + def step(self) -> None: + raise RuntimeError("the rapier backend never steps its render mirror (ADR-68)") + + def sync(self, force: bool = False) -> None: + if self.physics is not None: + self.physics.flush_render() + super().sync(force=force) + + +# --- scene objects (the Genesis duck-typed surface) ------------------------- + + +class RapierEntity: + """A free rigid body per environment (a med box, a board, a tray), with + its render twin in the Nexus mirror.""" + + def __init__(self, scene: RapierScene, name: str, handles: list, mirror, fixed: bool): + self.scene = scene + self.name = name + self.handles = handles # one rapier RigidBodyHandle per env + self.mirror = mirror # the NexusEntity that draws it + self.fixed = fixed + self.idx = mirror.idx + + def _body(self, env: int): + return self.scene.worlds[env].rigid_bodies.get(self.handles[env]) + + def get_pos(self) -> np.ndarray: + rows = [_xyz(self._body(env).translation) for env in range(self.scene.n_envs)] + return _squeeze_envs(np.stack(rows), self.scene.n_envs) + + def get_quat(self) -> np.ndarray: + rows = [_wxyz(self._body(env).rotation) for env in range(self.scene.n_envs)] + return _squeeze_envs(np.stack(rows), self.scene.n_envs) + + def get_dofs_velocity(self) -> np.ndarray: + rows = [] + for env in range(self.scene.n_envs): + body = self._body(env) + rows.append(np.concatenate([_xyz(body.linvel), _xyz(body.angvel)])) + return _squeeze_envs(np.stack(rows), self.scene.n_envs) + + def _write(self, env: int, pos, quat_wxyz) -> None: + import rapier3d as rp + + body = self._body(env) + body.translation = (float(pos[0]), float(pos[1]), float(pos[2])) + body.rotation = rp.Quaternion.from_tuple(_quat_xyzw(quat_wxyz)) + body.wake_up() + self.scene.render_dirty() + + def set_pos(self, pos, envs_idx=None) -> None: + rows, envs = _broadcast_rows(pos, self.scene.n_envs, envs_idx) + for row, env in zip(rows, envs, strict=True): + self._write(env, row[:3], _wxyz(self._body(env).rotation)) + + def set_quat(self, quat, envs_idx=None) -> None: + rows, envs = _broadcast_rows(quat, self.scene.n_envs, envs_idx) + for row, env in zip(rows, envs, strict=True): + self._write(env, _xyz(self._body(env).translation), row[:4]) + + def zero_all_dofs_velocity(self, envs_idx=None) -> None: + envs = range(self.scene.n_envs) if envs_idx is None else envs_idx + for env in envs: + body = self._body(int(env)) + body.linvel = (0.0, 0.0, 0.0) + body.angvel = (0.0, 0.0, 0.0) + + +@dataclass +class RapierJoint: + name: str + n_dofs: int + dofs_idx_local: list[int] + + +class RapierLink: + """One robot link; poses come from the link's rapier rigid body.""" + + def __init__(self, robot: RapierRobot, index: int, name: str): + self.robot = robot + self.index = index + self.name = name + + def get_pos(self) -> np.ndarray: + return _squeeze_envs(self.robot._link_poses()[0][:, self.index], self.robot.scene.n_envs) + + def get_quat(self) -> np.ndarray: + return _squeeze_envs(self.robot._link_poses()[1][:, self.index], self.robot.scene.n_envs) + + +@dataclass +class RapierArticulation: + """One environment's copy of a robot: the world it lives in, a handle that + resolves to its multibody, and its link bodies in link order.""" + + world: Any + joint_handle: Any + link_bodies: list + + def multibody(self): + return self.world.multibody_joints.multibody(self.joint_handle) + + def link_joint_handle(self, link_index: int): + """The multibody-joint handle naming one link, which is what rapier's + IK addresses. A multibody joint is keyed by its child link's rigid + body, so the handle is that body's raw parts.""" + import rapier3d as rp + + body = self.link_bodies[link_index] + return rp.MultibodyJointHandle.from_raw_parts(body.index, body.generation) + + +class RapierRobot: + """One articulated robot as a rapier reduced-coordinate multibody, one per + environment, mirrored by a Nexus robot that only ever draws it.""" + + def __init__( + self, + scene: RapierScene, + arts: list[RapierArticulation], + mirror, + name: str, + joint_links: list[tuple[str, int]], + ): + self.scene = scene + self.arts = arts + self.mirror = mirror + self.name = name + first = arts[0].multibody() + self.n_dofs = int(first.ndofs) + self.n_qs = self.n_dofs + # The renderer is driven by qpos, so the two models must agree on the + # DoF and link layout. They come from the same file, but a loader + # disagreeing silently would corrupt every frame. + link_names = list(mirror.robots[0].link_names) + if len(link_names) != first.num_links: + raise RuntimeError( + f"rapier loaded {first.num_links} links for {name!r} but nexus loaded " + f"{len(link_names)}; the two loaders disagree on the model" + ) + if int(mirror.n_dofs) != self.n_dofs: + raise RuntimeError( + f"rapier loaded {self.n_dofs} DoFs for {name!r} but nexus loaded {mirror.n_dofs}" + ) + self.links = [RapierLink(self, i, n) for i, n in enumerate(link_names)] + self.joints = self._build_joints(first, joint_links) + self.dof_axes = self._build_dof_axes(first) + # Gains from the model. rapier's MJCF binding reports `kp`/`kv` as None + # for MuJoCo "general" actuators (the Franka's kind), so they are read + # off the Nexus loader, which resolves them from the same file: both + # engines then run the identical servo. + self.kp = np.asarray(mirror.robots[0].kp, dtype=np.float64).copy() + self.kv = np.asarray(mirror.robots[0].kv, dtype=np.float64).copy() + self._motorized = False + self._targets = np.zeros((scene.n_envs, self.n_dofs), dtype=np.float64) + + # -- structure ------------------------------------------------------- + + def _build_joints(self, multibody, joint_links) -> list[RapierJoint]: + """The model's joints in the order the loader reports them, each + carrying its DoF indices into the generalized vectors. A multibody + link's `assembly_id` is that index, so the joint that moves a link + owns the DoFs at its child link's assembly id.""" + joints = [] + for name, link_id in joint_links: + link = multibody.get_link(link_id) + n_dofs = int(link.ndofs) + offset = int(link.assembly_id) + joints.append( + RapierJoint( + name=name, n_dofs=n_dofs, dofs_idx_local=list(range(offset, offset + n_dofs)) + ) + ) + return joints + + def _build_dof_axes(self, multibody) -> list[tuple[int, Any]]: + """Per DoF, the link whose joint owns it and that joint's free axis. + + rapier builds a 1-DoF joint with its free axis first in the local + frames, so a revolute joint moves along ANG_X and a prismatic one + along LIN_X. Which of the two a joint is is not exposed by the MJCF + binding, so it is read off the body jacobian: the column of a + revolute DoF is a unit angular velocity, a prismatic one's is a unit + linear velocity. + """ + import rapier3d as rp + + axes: list[tuple[int, Any]] = [(-1, None)] * self.n_dofs + for link_id in range(multibody.num_links): + link = multibody.get_link(link_id) + if link.ndofs == 0: + continue + jacobian = np.asarray(multibody.body_jacobian(link_id), dtype=np.float64) + for k in range(int(link.ndofs)): + dof = int(link.assembly_id) + k + angular = float(np.linalg.norm(jacobian[3:6, dof])) + axes[dof] = (link_id, rp.JointAxis.ANG_X if angular > 0.5 else rp.JointAxis.LIN_X) + missing = [d for d, (link_id, _) in enumerate(axes) if link_id < 0] + if missing: + raise RuntimeError(f"robot {self.name!r} has DoFs owned by no link: {missing}") + return axes + + def get_joint(self, name: str) -> RapierJoint: + for joint in self.joints: + if joint.name == name: + return joint + raise KeyError(f"robot {self.name!r} has no joint {name!r}") + + def get_link(self, name: str) -> RapierLink: + for link in self.links: + if link.name == name: + return link + raise KeyError(f"robot {self.name!r} has no link {name!r}") + + def get_dofs_limit(self) -> tuple[np.ndarray, np.ndarray]: + """rapier exposes no per-DoF limit getter on a multibody link, so the + limits come from the Nexus mirror's read of the same model.""" + first = self.mirror.robots[0] + return ( + np.asarray(first.dof_lower, dtype=np.float32), + np.asarray(first.dof_upper, dtype=np.float32), + ) + + @property + def link_bodies(self) -> list[list]: + """The render bodies, for a sensor camera to attach to (SCN-5).""" + return self.mirror.link_bodies + + # -- state ----------------------------------------------------------- + + def _link_poses(self) -> tuple[np.ndarray, np.ndarray]: + pos = np.empty((self.scene.n_envs, len(self.links), 3), dtype=np.float64) + quat = np.empty((self.scene.n_envs, len(self.links), 4), dtype=np.float64) + for env, art in enumerate(self.arts): + bodies = art.world.rigid_bodies + for index, handle in enumerate(art.link_bodies): + body = bodies.get(handle) + pos[env, index] = _xyz(body.translation) + quat[env, index] = _wxyz(body.rotation) + return pos, quat + + def get_qpos(self) -> np.ndarray: + rows = [np.asarray(art.multibody().generalized_position()) for art in self.arts] + return _squeeze_envs(np.stack(rows), self.scene.n_envs) + + def get_dofs_velocity(self) -> np.ndarray: + rows = [np.asarray(art.multibody().generalized_velocity()) for art in self.arts] + return _squeeze_envs(np.stack(rows), self.scene.n_envs) + + def _write_qpos(self, env: int, qpos) -> None: + """Drive the generalized coordinates to `qpos` and refresh the link + bodies. `apply_displacements` is the only writer rapier exposes, so a + coordinate is set by displacing it from where it is.""" + art = self.arts[env] + multibody = art.multibody() + current = multibody.generalized_position() + multibody.apply_displacements([float(q) - c for q, c in zip(qpos, current, strict=True)]) + multibody.forward_kinematics(art.world.rigid_bodies, False) + multibody.update_rigid_bodies(art.world.rigid_bodies, False) + self._wake(env) + + def _wake(self, env: int) -> None: + """A teleported arm must integrate on the next step: one rapier put to + sleep after settling would otherwise stay at rest (TC-6 reset).""" + bodies = self.arts[env].world.rigid_bodies + for handle in self.arts[env].link_bodies: + bodies.get(handle).wake_up() + + def set_qpos(self, qpos, envs_idx=None) -> None: + """Genesis's `set_qpos`, with the Nexus backend's rest semantics: the + write also brings the arm to rest, which is what the bridge's reset + path (TC-6) expects on every engine.""" + rows, envs = _broadcast_rows(qpos, self.scene.n_envs, envs_idx) + for row, env in zip(rows, envs, strict=True): + self._write_qpos(env, row[: self.n_dofs]) + self._rest(env) + self.scene.render_dirty() + + def _rest(self, env: int) -> None: + art = self.arts[env] + multibody = art.multibody() + multibody.set_generalized_velocity([0.0] * self.n_dofs) + bodies = art.world.rigid_bodies + for handle in art.link_bodies: + body = bodies.get(handle) + body.linvel = (0.0, 0.0, 0.0) + body.angvel = (0.0, 0.0, 0.0) + + def zero_all_dofs_velocity(self, envs_idx=None) -> None: + envs = range(self.scene.n_envs) if envs_idx is None else [int(e) for e in envs_idx] + for env in envs: + self._rest(env) + self.scene.render_dirty() + + def get_pos(self) -> np.ndarray: + return self.links[0].get_pos() + + def get_quat(self) -> np.ndarray: + return self.links[0].get_quat() + + def _write_root(self, env: int, pos, quat_wxyz) -> None: + """Re-base the (fixed-root) arm: write the root body's pose, then let + the multibody re-read it and carry every link along (SPEC 210).""" + import rapier3d as rp + + art = self.arts[env] + root = art.world.rigid_bodies.get(art.link_bodies[0]) + root.translation = (float(pos[0]), float(pos[1]), float(pos[2])) + root.rotation = rp.Quaternion.from_tuple(_quat_xyzw(quat_wxyz)) + multibody = art.multibody() + multibody.forward_kinematics(art.world.rigid_bodies, True) + multibody.update_rigid_bodies(art.world.rigid_bodies, False) + self._wake(env) + self.scene.render_dirty() + + def set_pos(self, pos, envs_idx=None) -> None: + rows, envs = _broadcast_rows(pos, self.scene.n_envs, envs_idx) + for row, env in zip(rows, envs, strict=True): + root = self.arts[env].world.rigid_bodies.get(self.arts[env].link_bodies[0]) + self._write_root(env, row[:3], _wxyz(root.rotation)) + + def set_quat(self, quat, envs_idx=None) -> None: + rows, envs = _broadcast_rows(quat, self.scene.n_envs, envs_idx) + for row, env in zip(rows, envs, strict=True): + root = self.arts[env].world.rigid_bodies.get(self.arts[env].link_bodies[0]) + self._write_root(env, _xyz(root.translation), row[:4]) + + # -- control --------------------------------------------------------- + + def set_dofs_kp(self, kp, dofs_idx_local=None, envs_idx=None) -> None: + dofs = list(range(self.n_dofs)) if dofs_idx_local is None else list(dofs_idx_local) + self.kp[dofs] = _per_dof(kp, dofs, "kp") + if self._motorized: + self._push_motors(dofs) + + def set_dofs_kv(self, kv, dofs_idx_local=None, envs_idx=None) -> None: + dofs = list(range(self.n_dofs)) if dofs_idx_local is None else list(dofs_idx_local) + self.kv[dofs] = _per_dof(kv, dofs, "kv") + if self._motorized: + self._push_motors(dofs) + + def control_dofs_position(self, target, dofs_idx_local=None, envs_idx=None) -> None: + """Genesis position control: one PD servo per commanded DoF. The + motors are installed on the first command and not before, so an + uncommanded arm sags under gravity exactly as it does on the other + two engines.""" + dofs = list(range(self.n_dofs)) if dofs_idx_local is None else list(dofs_idx_local) + rows, envs = _broadcast_rows(target, self.scene.n_envs, envs_idx) + for row, env in zip(rows, envs, strict=True): + self._targets[env, dofs] = row[: len(dofs)] + self._motorized = True + self._push_motors(dofs, envs) + + def _push_motors(self, dofs, envs=None) -> None: + import rapier3d as rp + + envs = range(self.scene.n_envs) if envs is None else envs + for env in envs: + multibody = self.arts[int(env)].multibody() + for dof in dofs: + link_id, axis = self.dof_axes[dof] + multibody.set_link_motor( + link_id, + axis, + float(self._targets[int(env), dof]), + 0.0, + float(self.kp[dof]), + float(self.kv[dof]), + ) + multibody.set_link_motor_model(link_id, axis, rp.MotorModel.FORCE_BASED) + + # -- kinematics ------------------------------------------------------ + + def inverse_kinematics( + self, + link, + pos, + quat, + local_point=None, + init_qpos=None, + max_samples: int = 1, + max_solver_iters: int = 100, + rot_mask=None, + dofs_idx_local=None, + return_error: bool = False, + **_ignored, + ): + """Genesis-compatible IK on environment 0's multibody. + + rapier's `inverse_kinematics_for_link` returns the displacement that + drives a link to a world pose, over a chosen subset of DoFs and a + chosen set of constrained axes. Genesis's `rot_mask[k]` instead asks + to align the link's k-th axis with the target's, so a single aligned + tool axis is mapped to the world axis it points along most, as the + Nexus backend does. The solve runs on the live multibody and restores + the configuration it found afterwards. + """ + import rapier3d as rp + + link_name = link.name if isinstance(link, RapierLink) else str(link) + link_index = self.get_link(link_name).index + pos = np.asarray(pos, dtype=np.float64).reshape(-1, 3)[0] + quat = np.asarray(quat, dtype=np.float64).reshape(-1, 4)[0] + mask = [True, True, True] if rot_mask is None else [bool(m) for m in rot_mask] + target_rot = quat_wxyz_to_matrix(quat) + constrained = [True] * 6 + if not all(mask): + free = [k for k, m in enumerate(mask) if not m] + if len(free) == 2: + aligned = mask.index(True) + axis = target_rot[:, aligned] + constrained[3 + int(np.argmax(np.abs(axis)))] = False + else: + for k in free: + constrained[3 + k] = False + axes_mask = rp.JointAxesMask.empty() + for bit, on in zip( + ( + rp.JointAxesMask.LIN_X, + rp.JointAxesMask.LIN_Y, + rp.JointAxesMask.LIN_Z, + rp.JointAxesMask.ANG_X, + rp.JointAxesMask.ANG_Y, + rp.JointAxesMask.ANG_Z, + ), + constrained, + strict=True, + ): + if on: + axes_mask = axes_mask | bit + + point = np.zeros(3) if local_point is None else np.asarray(local_point, dtype=np.float64) + # rapier drives the link origin; Genesis drives a point in the link + # frame, so the target is shifted back along the target rotation + target_pos = pos - target_rot @ point + + art = self.arts[0] + saved = list(art.multibody().generalized_position()) + if init_qpos is not None: + init = np.asarray(init_qpos, dtype=np.float64).reshape(-1, self.n_dofs)[0] + self._write_qpos(0, init) + joint_can_move = None + if dofs_idx_local is not None: + allowed = {int(d) for d in dofs_idx_local} + + def joint_can_move(link) -> bool: + # rapier frees whole joints, so a link moves only when all its DoFs may + start = link.assembly_id + return all(i in allowed for i in range(start, start + link.ndofs)) + + ik_cfg = self.scene.cfg["ik"] + option = rp.InverseKinematicsOption( + max_iters=int(max_solver_iters), + constrained_axes=axes_mask, + damping=float(ik_cfg["damping"]), + epsilon_linear=float(ik_cfg["epsilon_linear"]), + epsilon_angular=float(ik_cfg["epsilon_angular"]), + ) + displacement = art.world.multibody_joints.inverse_kinematics_for_link( + art.world.rigid_bodies, + art.link_joint_handle(link_index), + _isometry(target_pos, quat), + option, + joint_can_move=joint_can_move, + ) + multibody = art.multibody() + multibody.apply_displacements([float(v) for v in displacement]) + multibody.forward_kinematics(art.world.rigid_bodies, False) + multibody.update_rigid_bodies(art.world.rigid_bodies, False) + qpos = np.asarray(multibody.generalized_position(), dtype=np.float32) + + error = None + if return_error: + body = art.world.rigid_bodies.get(art.link_bodies[link_index]) + current_rot = quat_wxyz_to_matrix(_wxyz(body.rotation)) + realized = _xyz(body.translation) + current_rot @ point + rot_error = np.zeros(3) + for k, on in enumerate(mask): + if on: + cosine = float(np.clip(np.dot(current_rot[:, k], target_rot[:, k]), -1.0, 1.0)) + rot_error[k] = math.acos(cosine) + error = np.concatenate([pos - realized, rot_error]).astype(np.float32) + + self._write_qpos(0, saved) + self.scene.render_dirty() + batched = self.scene.n_envs > 1 + if return_error: + if batched: + return np.tile(qpos, (self.scene.n_envs, 1)), np.tile(error, (self.scene.n_envs, 1)) + return qpos, error + return np.tile(qpos, (self.scene.n_envs, 1)) if batched else qpos + + +# --- the scene -------------------------------------------------------------- + + +class RapierScene: + """The rapier counterpart of a Genesis `Scene`: one `PhysicsWorld` per + environment, and one Nexus scene that draws environment 0.""" + + def __init__(self, engine, physics: dict, n_envs: int, ambient) -> None: + import rapier3d as rp + + rapier_physics = load_rapier_physics() + self.rp = rp + self.n_envs = int(n_envs) + self.dt = float(physics["sim"]["dt"]) + self.gravity = [float(g) for g in physics["sim"]["gravity"]] + self.cfg = rapier_physics + camera = rapier_physics["camera"] + self.render = RapierRenderMirror( + engine, + dt=self.dt, + substeps=1, + gravity=self.gravity, + n_envs=self.n_envs, + ambient=ambient, + background_rgba=camera["background_rgba"], + camera_planes=(camera["znear"], camera["zfar"]), + ) + self.render.physics = self + self.worlds = [self._new_world() for _ in range(self.n_envs)] + self.entities: list[RapierEntity] = [] + self.robots: list[RapierRobot] = [] + self.friction_combine_rule = _combine_rule(rapier_physics["sim"]["friction_combine_rule"]) + self._dynamic_count = 0 + self._render_dirty = True + + def _new_world(self): + rp, sim = self.rp, self.cfg["sim"] + world = rp.PhysicsWorld(gravity=tuple(self.gravity)) + # Single-threaded by construction: the island solver's work split is + # rapier's only run-to-run ordering variation, and CON-5 wants the + # reproducible answer more than the parallel one. + world.set_num_threads(int(sim["num_threads"])) + params = world.integration_parameters + params.dt = self.dt + params.length_unit = float(sim["length_unit"]) + params.num_solver_iterations = int(sim["num_solver_iterations"]) + params.num_internal_pgs_iterations = int(sim["num_internal_pgs_iterations"]) + params.warmstart_coefficient = float(sim["warmstart_coefficient"]) + params.normalized_allowed_linear_error = float(sim["allowed_linear_error"]) / float( + sim["length_unit"] + ) + params.contact_softness = rp.SpringCoefficients( + float(sim["contact_natural_frequency"]), float(sim["contact_damping_ratio"]) + ) + params.static_contact_softness = rp.SpringCoefficients( + float(sim["static_contact_natural_frequency"]), float(sim["contact_damping_ratio"]) + ) + world.physics_pipeline.counters.enable() + return world + + @property + def segmentation_idx_dict(self) -> dict: + return self.render.segmentation_idx_dict + + @property + def built(self) -> bool: + return self.render.built + + # -- construction ---------------------------------------------------- + + def add_box( + self, + name: str, + size, + pos, + quat_wxyz=(1.0, 0.0, 0.0, 0.0), + fixed: bool = False, + friction: float = 0.5, + density: float | None = None, + color=(0.7, 0.7, 0.7, 1.0), + label_texture: np.ndarray | None = None, + ) -> RapierEntity: + rp = self.rp + mirror = self.render.add_box( + name, + size, + pos, + quat_wxyz=quat_wxyz, + fixed=fixed, + friction=friction, + density=density, + color=color, + label_texture=label_texture, + ) + half = [float(s) / 2 for s in size] + body_type = rp.RigidBodyType.FIXED if fixed else rp.RigidBodyType.DYNAMIC + handles = [] + for world in self.worlds: + collider = ( + rp.Collider.cuboid(*half) + .friction(float(friction)) + .friction_combine_rule(self.friction_combine_rule) + ) + if density is not None: + collider = collider.density(float(density)) + builder = rp.RigidBodyBuilder(body_type).position(_isometry(pos, quat_wxyz)) + handles.append(world.add_body(builder, [collider])) + if not fixed: + self._dynamic_count += 1 + entity = RapierEntity(self, name, handles, mirror, fixed) + self.entities.append(entity) + return entity + + def add_ground(self, size, friction: float) -> RapierEntity: + """The floor slab. Its checkerboard texture and shadow flag are the + Nexus backend's own, so the mirror is built by `NexusScene.add_ground` + and only the collision slab is added here.""" + mirror = self.render.add_ground(size, friction) + pos = (0.0, 0.0, -float(size[2]) / 2) + handles = [self._add_fixed_slab(world, size, pos, friction) for world in self.worlds] + entity = RapierEntity(self, "ground", handles, mirror, True) + self.entities.append(entity) + return entity + + def _add_fixed_slab(self, world, size, pos, friction: float): + rp = self.rp + collider = ( + rp.Collider.cuboid(*[float(s) / 2 for s in size]) + .friction(float(friction)) + .friction_combine_rule(self.friction_combine_rule) + ) + builder = rp.RigidBodyBuilder(rp.RigidBodyType.FIXED).position( + _isometry(pos, (1.0, 0.0, 0.0, 0.0)) + ) + return world.add_body(builder, [collider]) + + def _warm_up(self, world) -> None: + """A multibody is built with a 6-DoF free root that the engine + collapses only during the first step, so `ndofs` reads six too many + until one step has run. Take it under zero gravity, with only fixed + scenery in the world, so nothing else moves.""" + if self._dynamic_count: + raise RuntimeError( + "a rapier robot must be inserted before any dynamic body: its warm-up step " + f"would advance {self._dynamic_count} free bodies" + ) + gravity = world.gravity + world.gravity = (0.0, 0.0, 0.0) + world.step() + world.gravity = gravity + + def _articulate(self, world, joint_handles, link_bodies) -> RapierArticulation: + handle = next(h for h in joint_handles if h is not None) + return RapierArticulation(world=world, joint_handle=handle, link_bodies=link_bodies) + + def add_mjcf_robot(self, path: Path, name: str) -> RapierRobot: + from rapier3d.loaders import mjcf + + mirror = self.render.add_mjcf_robot(path, name) + arts, joint_links = [], None + for world in self.worlds: + options = mjcf.MjcfLoaderOptions() + # the Franka's base is welded to the world in MuJoCo; a dynamic + # root would insert a 6-DoF free joint ahead of joint1 + options.make_roots_fixed = True + robot, _model = mjcf.MjcfRobot.from_file(str(path), options) + joint_names = list(robot.joint_names) + handles = robot.insert_using_multibody_joints( + world.rigid_bodies, world.colliders, world.multibody_joints, world.impulse_joints + ) + self._warm_up(world) + # body_names[0] is the MJCF worldbody, which is not a link + link_bodies = [h.body for h in handles.bodies[1:]] + if joint_links is None: + joint_links = self._name_links(joint_names, handles.joints, link_bodies) + arts.append(self._articulate(world, [j.joint for j in handles.joints], link_bodies)) + robot = RapierRobot(self, arts, mirror, name, joint_links) + self.robots.append(robot) + return robot + + def add_urdf_robot(self, path: Path, name: str, convex_hull: bool) -> RapierRobot: + from rapier3d.loaders import urdf + + mirror = self.render.add_urdf_robot(path, name, convex_hull) + dynamics = self.cfg["robot"] + arts, joint_links = [], None + for world in self.worlds: + options = urdf.UrdfLoaderOptions() + options.make_roots_fixed = True + robot, description = urdf.UrdfRobot.from_file(str(path), options) + joint_names = [j.name for j in description.joints] + handles = robot.insert_using_multibody_joints( + world.rigid_bodies, world.colliders, world.multibody_joints + ) + self._warm_up(world) + link_bodies = [h.body for h in handles.links] + if joint_links is None: + joint_links = self._name_links(joint_names, handles.joints, link_bodies) + art = self._articulate(world, [j.joint for j in handles.joints], link_bodies) + # URDF joints declare no armature or damping; a light arm has no + # stable PD hold without them, as on the Nexus backend + multibody = art.multibody() + multibody.set_armature([float(dynamics["default_armature"])] * multibody.ndofs) + multibody.set_damping([float(dynamics["default_damping"])] * multibody.ndofs) + arts.append(art) + robot = RapierRobot(self, arts, mirror, name, joint_links) + self.robots.append(robot) + return robot + + @staticmethod + def _name_links(joint_names, joint_handles, link_bodies) -> list[tuple[str, int]]: + """Pair every named joint with the multibody link it moves: a joint's + DoFs live on its child link, which `link2` names by rigid body.""" + index_of = {handle: i for i, handle in enumerate(link_bodies)} + pairs = [] + for name, handle in zip(joint_names, joint_handles, strict=True): + if name is None or handle.link2 not in index_of: + continue + pairs.append((str(name), index_of[handle.link2])) + return pairs + + def add_camera(self, res, fov: float, pos=None, lookat=None): + return self.render.add_camera(res, fov, pos=pos, lookat=lookat) + + def build(self, n_envs: int | None = None) -> None: + if n_envs is not None and int(n_envs) != self.n_envs: + raise ValueError("n_envs is fixed at scene construction for the rapier backend") + self.render.build() + + # -- stepping and readback ------------------------------------------- + + def step(self) -> None: + for world in self.worlds: + world.step() + self.render_dirty() + + def render_dirty(self) -> None: + self._render_dirty = True + + def flush_render(self) -> None: + """Push rapier's poses into the Nexus state. Called from the render + mirror's `sync`, so nothing is copied on a step that is never drawn.""" + if not self._render_dirty or not self.render.built: + return + self._render_dirty = False + for entity in self.entities: + if entity.fixed: + continue + for env in range(self.n_envs): + body = entity._body(env) + self.render.set_body_pose( + env, entity.mirror.handles[env], _xyz(body.translation), _wxyz(body.rotation) + ) + for robot in self.robots: + for env, art in enumerate(robot.arts): + # every link's world pose, written straight into the mirror's + # body buffer. Pushing the joint coordinates instead would run + # the mirror's own forward kinematics from ITS root, which the + # mirror never learns about, so a re-based robot (the mobile + # store embodiment) rendered frozen at the origin while its + # boxes moved. + mirrored = robot.mirror.link_bodies[env] + for index, body_handle in enumerate(art.link_bodies): + body = art.world.rigid_bodies.get(body_handle) + self.render.set_body_pose( + env, mirrored[index], _xyz(body.translation), _wxyz(body.rotation) + ) + self.render.invalidate_poses() + self.render._synced = False + + def perf_stats(self) -> dict: + """Engine-side timing of the last step, from rapier's own counters. + No `gpu_ms`: rapier's step is a synchronous CPU call, so the bridge's + wall-clock step time already is the engine time, and claiming a GPU + number here would put a render cost in a physics column.""" + counters = self.worlds[0].physics_pipeline.counters + if not counters.enabled: + return {} + return { + "cpu_ms": float(counters.step_time_ms), + "collision_detection_ms": float(counters.stages.collision_detection_time_ms), + "solver_ms": float(counters.stages.solver_time_ms), + } + + +# --- scene builders (mirror the frozen Genesis builders) -------------------- + + +def _resolve_backend(sim_backend: str | None) -> None: + """rapier steps on the CPU and takes no other backend. The renderer's + adapter is chosen by the Nexus engine, not by this name (ADR-68).""" + if sim_backend not in (None, "cpu"): + raise ValueError(f"the rapier engine is CPU only; got backend {sim_backend!r}") + + +def _add_robot(scene: RapierScene, embodiment: str) -> RapierRobot: + if embodiment in ("franka", "mobile"): + return scene.add_mjcf_robot(franka_mjcf_path(), "franka") + if not SO101_URDF.exists(): + raise FileNotFoundError(f"so101 asset missing: {SO101_URDF} (acquisition pending, ADR-6)") + return scene.add_urdf_robot( + SO101_URDF, embodiment, bool(load_rapier_physics()["robot"]["convex_hull"]) + ) + + +def _apply_home_and_gains(robot: RapierRobot, profile: dict, n_envs: int) -> None: + """The frozen builder's post-build robot setup, verbatim in behavior: + home pose in TC-5 wire order mapped by joint name, then the profile's + gripper gains.""" + wire_dof_indices = profile_dof_indices(robot, profile) + if "home_qpos" in profile: + home = np.asarray(profile["home_qpos"], dtype=np.float32) + if wire_dof_indices is not None: + native_home = np.empty(robot.n_dofs, dtype=np.float32) + native_home[list(wire_dof_indices)] = home + home = native_home + robot.set_qpos(home if n_envs == 1 else np.tile(home, (n_envs, 1))) + if "gripper_dofs" in profile and "gripper_kp" in profile: + if wire_dof_indices is None: + count = int(profile["gripper_dofs"]) + finger_dofs = list(range(robot.n_dofs - count, robot.n_dofs)) + else: + count = len(profile["gripper_joint_names"]) + finger_dofs = list(wire_dof_indices[-count:]) + robot.set_dofs_kp( + np.asarray(profile["gripper_kp"], dtype=np.float32), dofs_idx_local=finger_dofs + ) + robot.set_dofs_kv( + np.asarray(profile["gripper_kv"], dtype=np.float32), dofs_idx_local=finger_dofs + ) + + +def build_scene( + seed: int, + embodiment: str = "franka", + n_envs: int = 1, + headless: bool = True, + cfg: SceneCfg | None = None, + sim_backend: str | None = None, +) -> SceneHandle: + """rapier twin of `aisle.scenes.pharmacy.build_scene` (SPEC 020): the same + seeded layout, DR draws, colors, labels and cameras, stepped by rapier and + rendered through the Nexus viewer (ADR-68).""" + cfg = cfg or SceneCfg() + _resolve_backend(sim_backend) + engine = _ensure_nexus(None, headless=headless) + physics = load_physics() + rapier_physics = load_rapier_physics() + layout = resolve_layout(physics, embodiment) + meds = load_meds() + shelf, tray_cfg = layout["shelf"], layout["tray"] + dr_cfg = physics["domain_randomization"] + + for label, target in (("tray", tray_cfg["pos"]), ("shelf", shelf["pos"])): + distance = math.hypot(*target) + assert distance <= layout["reach_m"], ( + f"{label} at {target} outside {embodiment} workspace (SCN-4)" + ) + + lighting_rng = random.Random(cfg.lighting.seed) + textures_rng = random.Random(cfg.textures.seed) + friction_rng = random.Random(cfg.friction_jitter.seed) + camera_rng = random.Random(cfg.camera_jitter.seed) + + ambient = (dr_cfg["ambient_default"],) * 3 + if cfg.lighting.enabled: + ambient = tuple( + min(1.0, dr_cfg["ambient_min"] + lighting_rng.random() * dr_cfg["ambient_range"]) + for _ in range(3) + ) + + scene = RapierScene(engine, physics, n_envs, ambient) + ground = rapier_physics["ground"] + scene.add_ground(ground["size"], ground["friction"]) + + shelf_friction = physics["materials"]["shelf"]["friction"] + width = shelf["level_size"][1] + for level, (level_height, level_depth) in enumerate( + zip(shelf["level_heights"], shelf["level_depths"], strict=True) + ): + x_min, x_max = level_x_span(shelf, level) + scene.add_box( + f"shelf_{level}", + (level_depth, width, shelf["board_thickness"]), + ((x_min + x_max) / 2, shelf["pos"][1], shelf["pos"][2] + level_height), + fixed=True, + friction=shelf_friction, + ) + tray = scene.add_box( + "tray", + tuple(tray_cfg["size"]), + tuple(tray_cfg["pos"]), + fixed=True, + friction=physics["materials"]["tray"]["friction"], + ) + + robot = _add_robot(scene, embodiment) + + box_physics = physics["materials"]["box"] + applied_frictions: dict[str, float] = {} + applied_colors: dict[str, list[float]] = {} + boxes: dict[str, Any] = {} + color_by_med = {name: meds[name]["color"] for name in meds} + if cfg.shuffle_colors: + names = list(meds) + shuffled = list(np.random.default_rng(seed ^ 0x5EED).permutation(names)) + color_by_med = { + name: meds[donor]["color"] for name, donor in zip(names, shuffled, strict=True) + } + placements = sample_placements(seed, list(meds), layout) + if cfg.occlusion: + placements = apply_occlusion(placements, seed, list(meds), layout) + for placement in placements: + friction = box_physics["friction"] + if cfg.friction_jitter.enabled: + friction *= 1.0 + (friction_rng.random() - 0.5) * dr_cfg["friction_jitter_frac"] + applied_frictions[placement.name] = friction + color = list(color_by_med[placement.name]) + if cfg.textures.enabled: + scale_min, scale_range = dr_cfg["texture_scale_min"], dr_cfg["texture_scale_range"] + color = [ + min(1.0, c * (scale_min + textures_rng.random() * scale_range)) for c in color[:3] + ] + [color[3]] + applied_colors[placement.name] = color + label_texture = None + if cfg.labels: + label_texture = label_texture_image( + str(meds[placement.name].get("label", placement.name.upper())), color + ) + boxes[placement.name] = scene.add_box( + placement.name, + tuple(meds[placement.name]["size"]), + (placement.x, placement.y, placement.z), + friction=friction, + density=box_physics["density_kg_m3"], + color=tuple(color), + label_texture=label_texture, + ) + + cam_cfg = physics["cameras"] + overhead_pos = list(cam_cfg["overhead_pos"]) + if cfg.camera_jitter.enabled: + jitter = dr_cfg["camera_jitter_m"] + overhead_pos = [p + (camera_rng.random() - 0.5) * jitter for p in overhead_pos] + cams = { + "overhead": scene.add_camera( + (640, 480), 55, pos=tuple(overhead_pos), lookat=tuple(cam_cfg["overhead_lookat"]) + ), + "wrist": scene.add_camera((320, 240), 70), + } + + scene.build() + + profile = physics["embodiment"][embodiment] + _apply_home_and_gains(robot, profile, n_envs) + + ee_link = robot.get_link(profile.get("ee_link", FRANKA_EE_LINK)) + offset = wrist_mount_transform(cam_cfg, profile) + cams["wrist"].attach(ee_link, offset_T=offset) + + handle = SceneHandle( + scene=scene, + robot=robot, + boxes=boxes, + tray=tray, + cams=cams, + embodiment=embodiment, + seed=seed, + med_sizes={name: list(meds[name]["size"]) for name in meds}, + dr_applied={ + "ambient": ambient, + "overhead_pos": overhead_pos, + "frictions": applied_frictions, + "colors": applied_colors, + }, + ) + _assert_reachable(handle, ee_link, layout, n_envs) + return handle + + +def build_store( + seed: int, + scenario: str, + embodiment: str = "mobile", + n_envs: int = 1, + headless: bool = True, + sim_backend: str | None = None, +): + """rapier twin of `aisle.scenes.store.build_store` (SPEC 200 RS-2/RS-3).""" + from aisle.scenes.store import ( + StoreHandle, + episode_layout, + full_stock, + generate_episode, + load_planogram, + yaw_quat_wxyz, + ) + + _resolve_backend(sim_backend) + engine = _ensure_nexus(None, headless=headless) + meds = load_meds() + physics = load_physics() + rapier_physics = load_rapier_physics() + plano = load_planogram() + episode = generate_episode(seed, scenario) + store, geo = plano["store"], plano["store"]["unit_geometry"] + profile = physics["embodiment"][embodiment] + + scene = RapierScene( + engine, physics, n_envs, (physics["domain_randomization"]["ambient_default"],) * 3 + ) + ground = rapier_physics["ground"] + scene.add_ground(ground["size"], ground["friction"]) + + shelf_friction = physics["materials"]["shelf"]["friction"] + for unit_id, unit in plano["units"].items(): + for level, level_height in enumerate(geo["level_heights"]): + scene.add_box( + f"{unit_id}_level_{level}", + (geo["depth"], geo["width"], geo["board_thickness"]), + (unit["pos"][0], unit["pos"][1], level_height), + quat_wxyz=yaw_quat_wxyz(unit["yaw"]), + fixed=True, + friction=shelf_friction, + ) + tray_friction = physics["materials"]["tray"]["friction"] + counter = scene.add_box( + "counter", + tuple(store["counter_size"]), + tuple(store["counter_pos"]), + fixed=True, + friction=tray_friction, + ) + bin_entity = scene.add_box( + "bin", tuple(store["bin_size"]), tuple(store["bin_pos"]), fixed=True, friction=tray_friction + ) + + robot = scene.add_mjcf_robot(franka_mjcf_path(), "franka") + + box_physics = physics["materials"]["box"] + items: dict[str, Any] = {} + categories: dict[str, str] = {} + layout = episode_layout(plano, episode, meds) + for item in full_stock(plano): + x, y, z, yaw = layout[item.item_id] + categories[item.item_id] = item.category + items[item.item_id] = scene.add_box( + item.item_id, + tuple(meds[item.category]["size"]), + (x, y, z), + quat_wxyz=yaw_quat_wxyz(yaw), + friction=box_physics["friction"], + density=box_physics["density_kg_m3"], + color=tuple(meds[item.category]["color"]), + ) + + cam_cfg = physics["cameras"] + cams = { + "overhead": scene.add_camera( + (640, 480), + 70, + pos=tuple(cam_cfg["store_overhead_pos"]), + lookat=tuple(cam_cfg["store_overhead_lookat"]), + ), + "wrist": scene.add_camera((320, 240), 70), + } + + scene.build() + + home = np.asarray(profile["home_qpos"], dtype=np.float32) + robot.set_qpos(home if n_envs == 1 else np.tile(home, (n_envs, 1))) + count = int(profile["gripper_dofs"]) + finger_dofs = list(range(robot.n_dofs - count, robot.n_dofs)) + robot.set_dofs_kp( + np.asarray(profile["gripper_kp"], dtype=np.float32), dofs_idx_local=finger_dofs + ) + robot.set_dofs_kv( + np.asarray(profile["gripper_kv"], dtype=np.float32), dofs_idx_local=finger_dofs + ) + + ee_link = robot.get_link("hand") + offset = np.eye(4, dtype=np.float32) + offset[:3, 3] = cam_cfg["wrist_offset_m"] + cams["wrist"].attach(ee_link, offset_T=offset) + + return StoreHandle( + scene=scene, + robot=robot, + items=items, + categories=categories, + counter=counter, + bin=bin_entity, + cams=cams, + planogram=plano, + episode=episode, + embodiment=embodiment, + seed=seed, + scenario=scenario, + med_sizes={name: list(meds[name]["size"]) for name in meds}, + ) + + +__all__ = [ + "RapierArticulation", + "RapierEntity", + "RapierJoint", + "RapierLink", + "RapierRenderMirror", + "RapierRobot", + "RapierScene", + "build_scene", + "build_store", + "load_rapier_physics", +] diff --git a/src/aisle/sim/rapier_physics.toml b/src/aisle/sim/rapier_physics.toml new file mode 100644 index 00000000..724d63a2 --- /dev/null +++ b/src/aisle/sim/rapier_physics.toml @@ -0,0 +1,87 @@ +# rapier-engine constants (ADR-68, SCN-2 spirit: no physics constants inline +# in code). Kept outside the frozen scene directory for the same reason +# nexus_physics.toml is: these describe how the rapier backend realizes the +# frozen scene configuration, not the scene itself. +# +# The sections mirror nexus_physics.toml so a diff between the two engines is +# readable. The Nexus-only scheduling knobs (friction_in_bias_pass, +# implicit_coriolis, substep_refresh) have no rapier counterpart and are +# absent; the camera and light blocks are copied verbatim, because rapier +# renders through the Nexus viewer and must produce the same pixels. + +[sim] +# rapier has no substep count; the analogous knob is the number of solver +# iterations, each of which runs the internal PGS sweeps below. Two, the +# like-for-like match to the Nexus configuration's substeps, lifts the box but +# loses it in transport (T0 seed 0 closes `dropped` at 17.9 s, 80 degrees of +# tilt); eight holds the pinch to 2.6 degrees and closes the episode at the +# same 20.86 s as both other engines. At 0.2 ms per step the extra iterations +# cost nothing worth saving. +num_solver_iterations = 8 +num_internal_pgs_iterations = 8 +# Contact solver, the same targets as the Nexus backend: 240 Hz soft contacts +# with 0.5 mm allowed penetration, so the pinch penetration stays at Genesis's +# scale. Frequencies in Hz, lengths in meters. +contact_natural_frequency = 240.0 +contact_damping_ratio = 5.0 +static_contact_natural_frequency = 480.0 +allowed_linear_error = 0.0005 +length_unit = 1.0 +warmstart_coefficient = 1.0 +# Genesis combines the friction of two touching geoms with max(); rapier's +# default is the average. Match Genesis, as the Nexus backend does. +friction_combine_rule = "max" +# Single-threaded stepping: the island solver's work distribution is the only +# run-to-run ordering variation rapier has, and CON-5 wants the reproducible +# answer more than the parallel one on a scene this small. +num_threads = 1 + +[ground] +# rapier has a half-space primitive, but the ground is built as the same fixed +# slab the Nexus backend uses so both engines render the identical floor. +size = [20.0, 20.0, 0.1] +friction = 1.0 +color = [0.30, 0.33, 0.38, 1.0] +checker_color = [0.42, 0.46, 0.52, 1.0] +checker_squares = 40 +casts_shadows = false + +[robot] +# Genesis's URDF defaults: joints without an explicit profile gain get these. +# MJCF position actuators keep the gains declared in the model. +default_kp = 100.0 +default_kv = 10.0 +# Per-DoF reflected rotor inertia and viscous damping for URDF joints, which +# declare neither. The values the official SO-101 MJCF uses for its STS3215 +# servos, as on the Nexus backend. +default_armature = 0.01 +default_damping = 0.1 +convex_hull = true + +[ik] +# rapier solves IK by damped least squares, Genesis by Gauss-Newton, so +# physics.toml's `max_solver_iters` does not mean the same thing on the two +# engines; it stays the per-call iteration budget and these tune the step. +# At rapier's default damping of 1.0, seed 3's omeprazole pre-grasp is still +# 16 mm out after 100 iterations, just past the frozen 15 mm `pos_tol_m`; +# 0.1 lands every desk seed well inside it at the same budget. The frozen +# tolerances remain the acceptance gate (SCN-3). +damping = 0.1 +# Residual at which the solver stops early, an order of magnitude under the +# frozen tolerances so the gate, not the solver, decides. +epsilon_linear = 0.00001 +epsilon_angular = 0.0001 + +[camera] +znear = 0.01 +zfar = 50.0 +background_rgba = [0.07, 0.08, 0.12, 1.0] +msaa_samples = 4 +shadow_softness = 1.0 +shadow_first_cascade_m = 2.5 +shadow_distance_m = 20.0 +shadow_resolution = 4096 +shadow_atlas_layers = 4 + +[light] +direction = [0.3, -0.4, -1.0] diff --git a/tests/sim/test_nexus_scene.py b/tests/sim/test_nexus_scene.py new file mode 100644 index 00000000..8742b9c3 --- /dev/null +++ b/tests/sim/test_nexus_scene.py @@ -0,0 +1,728 @@ +"""Sim acceptance tests for the Nexus engine backend (ADR-67): the frozen +pharmacy scene (SPEC 020 SCN-1, SCN-3..5, SCN-7) and the bridge's object +surface (SPEC 030), realized on Nexus. + +Marker `sim`: imports nexus3d (and genesis for the Franka asset), runs +headless. Skipped when the nexus3d wheel is not installed. + +Scene order: only the newest scene can render (a later build frees the +previous scene's cameras), so tests that only read a built scene share one +module-scoped build, each group that steps, teleports or renders gets exactly +one build, and the groups that render come last. +""" + +import importlib.util + +import numpy as np +import pytest + +from aisle.scenes.pharmacy import MED_NAMES, load_physics, oracle_state, to_numpy + +pytestmark = [ + pytest.mark.sim, + pytest.mark.skipif( + importlib.util.find_spec("nexus3d") is None or importlib.util.find_spec("genesis") is None, + reason="nexus3d wheel or the sim extra not installed", + ), +] + + +# Function-scoped on purpose: the shared Nexus viewer renders the LATEST +# scene only (a later build supersedes the previous one's cameras), so a test +# that renders or mutates gets a fresh scene (about a second each). +@pytest.fixture +def handle(): + from aisle.sim import build_scene + + return build_scene("nexus", seed=7, embodiment="franka", n_envs=1, headless=True) + + +# Module-scoped: these tests only read the built scene (no step, no teleport, +# no render), so one build serves them all. +@pytest.fixture(scope="module") +def pristine(): + from aisle.sim import build_scene + + return build_scene("nexus", seed=7, embodiment="franka", n_envs=1, headless=True) + + +@pytest.fixture(scope="module") +def so101_handle(): + from aisle.sim import build_scene + + return build_scene("nexus", seed=3, embodiment="so101", n_envs=1, headless=True) + + +def test_build_determinism(pristine): + """SCN-1, SCN-7 on Nexus: the initial oracle_state is a pure function of + the seed (bitwise) and differs across seeds.""" + from aisle.sim import build_scene + + first = oracle_state(pristine) + again = oracle_state(build_scene("nexus", seed=7, embodiment="franka", headless=True)) + other = oracle_state(build_scene("nexus", seed=11, embodiment="franka", headless=True)) + assert first.dtype == np.float32 + assert first.shape == (len(MED_NAMES) * 7,) + assert np.array_equal(first, again) + assert not np.array_equal(first, other) + # a superseded scene keeps its physics readbacks but refuses to render + assert np.array_equal(oracle_state(pristine), first) + with pytest.raises(RuntimeError, match="superseded"): + pristine.cams["overhead"].render() + + +def test_placements_match_frozen_sampler(pristine): + """ADR-67: the Nexus builder places the boxes exactly where the frozen + sampler says (same pure function, same seed), before any step.""" + from aisle.scenes.pharmacy import resolve_layout, sample_placements + + placements = { + p.name: p for p in sample_placements(7, MED_NAMES, resolve_layout(load_physics(), "franka")) + } + for name, entity in pristine.boxes.items(): + pos = to_numpy(entity.get_pos()).reshape(-1)[:3] + expected = placements[name] + assert np.allclose(pos, [expected.x, expected.y, expected.z], atol=1e-5), name + + +def test_reachability_and_home(pristine): + """SCN-3/SCN-4 (Nexus IK) and SCN-1: every placement admits an IK + solution and the robot rests at its home pose.""" + assert pristine.reachability_errors == [] + home = np.asarray(load_physics()["embodiment"]["franka"]["home_qpos"], dtype=np.float32) + actual = to_numpy(pristine.robot.get_qpos()).reshape(-1)[: home.shape[0]] + assert np.allclose(actual, home, atol=1e-4) + + +def test_realized_calibration_matches_nominal(pristine): + """BRG-8 / VER-8: the realized overhead transform is the Genesis look-at + convention, so the published v1 block passes stage 0.""" + from aisle.nodes.dora_genesis import realized_calibration + from aisle.verifier.calibration import build_calibration_v1 + + physics = load_physics() + realized = realized_calibration(pristine, physics, is_store=False) + from aisle.scenes.pharmacy import wrist_mount_transform + + wrist = wrist_mount_transform(physics["cameras"], physics["embodiment"]["franka"]) + nominal = build_calibration_v1( + overhead_pos=physics["cameras"]["overhead_pos"], + overhead_lookat=physics["cameras"]["overhead_lookat"], + overhead_resolution=(640, 480), + overhead_fov_deg=55, + wrist_offset_m=wrist[:3, 3].tolist(), + wrist_resolution=(320, 240), + wrist_fov_deg=70, + wrist_mount_rotation_gl=wrist[:3, :3], + ) + assert realized["overhead"]["intrinsics"] == nominal["overhead"]["intrinsics"] + assert np.allclose( + realized["overhead"]["cam_to_base"]["pos"], + nominal["overhead"]["cam_to_base"]["pos"], + atol=1e-5, + ) + assert np.allclose( + realized["overhead"]["cam_to_base"]["quat_xyzw"], + nominal["overhead"]["cam_to_base"]["quat_xyzw"], + atol=1e-4, + ) + + +def test_step_teleport_and_control(handle): + """BRG-2/BRG-4: stepping advances physics, a teleport reset lands a box + exactly where asked with zero velocity, and position control holds the + home pose under gravity.""" + robot = handle.robot + home = to_numpy(robot.get_qpos()).reshape(-1).copy() + robot.control_dofs_position(home) + box = next(iter(handle.boxes.values())) + robot_before = to_numpy(robot.get_link("hand").get_pos()).reshape(-1) + for _ in range(50): + handle.scene.step() + after = to_numpy(robot.get_qpos()).reshape(-1) + assert np.allclose(after[:7], home[:7], atol=0.02) + assert np.allclose( + to_numpy(robot.get_link("hand").get_pos()).reshape(-1), robot_before, atol=0.02 + ) + box.set_pos(np.array([0.3, -0.45, 0.3], dtype=np.float32)) + box.set_quat(np.array([1.0, 0.0, 0.0, 0.0], dtype=np.float32)) + box.zero_all_dofs_velocity() + assert np.allclose(to_numpy(box.get_pos()).reshape(-1), [0.3, -0.45, 0.3], atol=1e-5) + handle.scene.step() + pos = to_numpy(box.get_pos()).reshape(-1) + assert pos[2] < 0.3 and pos[2] > 0.29 # falling under gravity, one step later + + +def test_resting_boxes_hold_still(handle): + """SCN-3 on Nexus: boxes at rest on the boards must not creep past the + oracle's knock threshold over an episode. The substep count in + nexus_physics.toml is chosen for this; one second of settling must stay + well under a millimeter per box.""" + robot = handle.robot + robot.control_dofs_position(to_numpy(robot.get_qpos()).reshape(-1)) + start = oracle_state(handle).reshape(-1, 7)[:, :3] + for _ in range(100): + handle.scene.step() + drift = np.linalg.norm(oracle_state(handle).reshape(-1, 7)[:, :3] - start, axis=1) + assert drift.max() < 1.0e-3, drift + + +def test_so101_urdf_matches_frozen_chain(so101_handle): + """ADR-67: the imported SO-101 kinematics agree with the frozen URDF + chain (`aisle.kinematics`) at several configurations, so grasp planning + and the guard see the same arm the physics does.""" + from aisle.kinematics import so101_chain + + chain = so101_chain() + robot = so101_handle.robot + rng = np.random.default_rng(1) + for _ in range(3): + q = rng.uniform(-1.0, 1.0, size=5) + expected, _ = chain.forward(q) + pos, _ = robot.scene.state.robot_forward_kinematics( + robot.robots[0], [float(v) for v in q] + [0.0], "gripper_frame_link" + ) + assert np.allclose(pos, expected, atol=2e-4), (q, pos, expected) + + +def test_so101_home_and_gripper_gains(so101_handle): + """SCN-1 / TC-5: the SO-101 rests at its profile home in wire order and + carries the profile's gripper gains; batched envs share the layout.""" + from aisle.embodiment import profile_dof_indices + + profile = load_physics()["embodiment"]["so101"] + robot = so101_handle.robot + wire = profile_dof_indices(robot, profile) + qpos = to_numpy(robot.get_qpos()).reshape(-1) + assert np.allclose(qpos[list(wire)], profile["home_qpos"], atol=1e-4) + gripper = robot.robots[0] + assert np.isclose(gripper.kp[wire[-1]], profile["gripper_kp"][0]) + assert np.isclose(gripper.kv[wire[-1]], profile["gripper_kv"][0]) + assert so101_handle.reachability_errors == [] + + +def test_batched_build_oracle_covers_all_envs(): + """SCN-1 on Nexus: a batched build reports every env and identical + initial placements.""" + from aisle.sim import build_scene + + batched = build_scene("nexus", seed=7, embodiment="franka", n_envs=2, headless=True) + state = oracle_state(batched) + assert state.shape == (2, len(MED_NAMES) * 7) + assert np.array_equal(state[0], state[1]) + qpos = to_numpy(batched.robot.get_qpos()) + assert qpos.shape == (2, batched.robot.n_dofs) + + +def test_no_interpenetration(pristine): + """SCN-3 on Nexus: the initial placements the frozen sampler produced are + realized free of interpenetration (pairwise AABBs of the meds).""" + aabbs = [] + for name, entity in pristine.boxes.items(): + pos = to_numpy(entity.get_pos()).reshape(-1)[:3] + half = np.asarray(pristine.med_sizes[name]) / 2.0 + aabbs.append((name, pos - half, pos + half)) + for i, (name_a, lo_a, hi_a) in enumerate(aabbs): + for name_b, lo_b, hi_b in aabbs[i + 1 :]: + overlap = np.all(lo_a < hi_b) and np.all(lo_b < hi_a) + assert not overlap, (name_a, name_b) + + +def test_oracle_quaternions_are_xyzw(pristine): + """TC-1 on Nexus: oracle_state quaternions are (x, y, z, w) wire order. + The Nexus readback is w-first like Genesis's, so a missing roll here + would publish the same wrong wire order the Genesis test pins.""" + state = oracle_state(pristine) + for i in range(len(MED_NAMES)): + quat = state[i * 7 + 3 : i * 7 + 7] + assert abs(quat[3]) > 0.99, quat # w last + assert np.all(np.abs(quat[:3]) < 0.1), quat + + +def test_boxes_follow_oracle_order(pristine): + """SCN-1 on Nexus: the boxes dict is in the fixed meds.toml order, which + is the oracle_state layout (TC table).""" + assert list(pristine.boxes) == MED_NAMES + + +@pytest.fixture(scope="module") +def mobile_scene(): + """One mobile-profile build for the re-basing tests (they move the base, + never the arm, so they compose on one scene).""" + from aisle.sim import build_scene + + return build_scene("nexus", seed=0, embodiment="mobile", n_envs=1, headless=True) + + +def test_mobile_builds_franka_arm_and_rebases(mobile_scene): + """MOB-4 and ADR-13 on Nexus: the mobile profile builds the same franka + arm (it rests at the frozen franka home, as the fixed-base build does), + and re-basing through `NexusRobot.set_pos`/`set_quat` moves the whole + arm's world mount while the arm stays base-relative (joint FK unchanged). + + Nexus roots the arm at a fixed body whose pose the GPU re-reads every + step, so the links must ride the base rather than the root sliding out + from under them.""" + import math + + from aisle.mobility.base import integrate_base_pose + from aisle.nodes.ik_trajectory import fk_tcp + + mobile = mobile_scene + robot = mobile.robot + home = np.asarray(load_physics()["embodiment"]["franka"]["home_qpos"], dtype=np.float32) + qpos = to_numpy(robot.get_qpos()).reshape(-1) + assert np.allclose(qpos[: home.shape[0]], home, atol=1e-4) # MOB-4: the same arm + tcp_before = fk_tcp(qpos[:7]) + + hand_before = to_numpy(robot.get_link("hand").get_pos()).reshape(-1)[:3] + + pose = [0.0, 0.0, 0.0] + for _ in range(10): # 1 m forward at 1 m/s over 0.1 s ticks + pose = integrate_base_pose(pose, [1.0, 0.0], dt=0.1) + robot.set_pos(np.array([pose[0], pose[1], 0.0], dtype=np.float32)) + mobile.scene.step() + + assert to_numpy(robot.get_pos()).reshape(-1)[:2] == pytest.approx([1.0, 0.0], abs=1e-3) + # the links rode the base: the hand moved by the same world translation + hand_after = to_numpy(robot.get_link("hand").get_pos()).reshape(-1)[:3] + assert hand_after - hand_before == pytest.approx([1.0, 0.0, 0.0], abs=1e-2) + # and the arm-frame FK is unchanged: the arm rode the base, it did not move + assert fk_tcp(to_numpy(robot.get_qpos()).reshape(-1)[:7]) == pytest.approx(tcp_before, abs=1e-2) + + # a yaw re-base rotates the arm about the base origin, which stays put + robot.set_quat( + np.array([math.cos(math.pi / 4), 0.0, 0.0, math.sin(math.pi / 4)], dtype=np.float32) + ) + mobile.scene.step() + root_now = to_numpy(robot.get_pos()).reshape(-1)[:3] + assert root_now[:2] == pytest.approx([1.0, 0.0], abs=1e-3) + hand_yawed = to_numpy(robot.get_link("hand").get_pos()).reshape(-1)[:3] + offset = hand_after - np.asarray([1.0, 0.0, 0.0]) # base-frame hand offset + expected = root_now + np.asarray([-offset[1], offset[0], offset[2]]) # +90 deg about z + assert hand_yawed == pytest.approx(expected, abs=1e-2), (hand_yawed, expected) + + +def test_mobile_rebase_survives_set_pos_then_set_quat(mobile_scene): + """ADR-13/MOB-4 on Nexus: the bridge re-bases with `set_pos` immediately + followed by `set_quat`, with no step between, so the pair must compose. + + Regression: `set_quat` used to read the root position back from + `robot_state`, which still reports the pre-step root, so it wrote the old + position back and cancelled the `set_pos` before it. On Nexus a mobile base + then only ever turned in place, its translation silently dropped. The root + write is now remembered until the next step consumes it.""" + import math + + mobile = mobile_scene + robot = mobile.robot + yaw = math.pi / 3 + robot.set_pos(np.array([2.0, 0.5, 0.0], dtype=np.float32)) + robot.set_quat(np.array([math.cos(yaw / 2), 0.0, 0.0, math.sin(yaw / 2)], dtype=np.float32)) + mobile.scene.step() + assert to_numpy(robot.get_pos()).reshape(-1)[:2] == pytest.approx([2.0, 0.5], abs=1e-3) + + +# --- the store scene on Nexus (SPEC 200), ported from test_store_scene.py --- + + +def _xyz(entity) -> list[float]: + return [float(v) for v in to_numpy(entity.get_pos()).reshape(-1)[:3]] + + +def _assert_yaw(entity, yaw: float, label) -> None: + """The entity's physical orientation matches the composed world yaw: + quaternion equal up to sign, and the rotated +x (front face) points where + the slot faces.""" + import math + + from aisle.scenes.store import yaw_quat_wxyz + + got = [float(v) for v in to_numpy(entity.get_quat()).reshape(-1)[:4]] + want = list(yaw_quat_wxyz(yaw)) + if got[0] * want[0] + got[3] * want[3] < 0: # q and -q are the same rotation + want = [-c for c in want] + assert got == pytest.approx(want, abs=1e-5), (label, got, want) + w, _, _, z = got + front = (1 - 2 * z * z, 2 * w * z) + assert front[0] == pytest.approx(math.cos(yaw), abs=1e-5), label + assert front[1] == pytest.approx(math.sin(yaw), abs=1e-5), label + + +@pytest.fixture(scope="module") +def store_handle(): + """One store build, shared in file order: the planogram check reads it, + then the oracle/teleport check disturbs and resets it.""" + from aisle.sim import build_store + + return build_store("nexus", seed=1, scenario="S1") + + +def test_store_build_realizes_the_planogram(store_handle): + """RS-1/RS-2 on Nexus (ADR-67, T16/ADR-19): the Nexus store is generated + from the same planogram.toml as the Genesis one: the full stock spawns + at its slots' world template poses with the composed unit yaw, the shelf + boards carry that yaw too, the bin holds one item per category, counter + and bin sit where the planogram puts them, and the mobile-profile robot + starts at the store-frame origin.""" + import math + + from aisle.scenes.store import full_stock, slot_world_pose + + handle = store_handle + plano = handle.planogram + + assert list(handle.items) == [item.item_id for item in full_stock(plano)] + for slot_id, slot in plano["slots"].items(): + item_id = f"{slot_id}#0" + assert item_id in handle.items, f"slot {slot_id} not stocked" + world, yaw = slot_world_pose(plano, slot_id) + size_z = handle.med_sizes[slot["category"]][2] + pos = _xyz(handle.items[item_id]) + assert pos[0] == pytest.approx(world[0], abs=1e-4), slot_id + assert pos[1] == pytest.approx(world[1], abs=1e-4), slot_id + assert pos[2] == pytest.approx(world[2] + size_z / 2, abs=1e-4), slot_id + _assert_yaw(handle.items[item_id], yaw, slot_id) + + # the shelf boards the items rest on carry the unit's yaw as well: the + # Nexus builder composes it into the board's spawn quaternion (RS-2) + boards = {entity.name: entity for entity in handle.scene.entities} + for unit_id, unit in plano["units"].items(): + for level in range(len(plano["store"]["unit_geometry"]["level_heights"])): + _assert_yaw(boards[f"{unit_id}_level_{level}"], unit["yaw"], unit_id) + + store = plano["store"] + bin_top = store["bin_pos"][2] + store["bin_size"][2] / 2 + bin_items = [i for i in handle.items if i.startswith("bin#")] + assert len(bin_items) == len(handle.med_sizes) + for item_id in bin_items: + pos = _xyz(handle.items[item_id]) + size_z = handle.med_sizes[handle.categories[item_id]][2] + assert pos[2] == pytest.approx(bin_top + size_z / 2, abs=1e-4), item_id + + assert _xyz(handle.counter)[:2] == pytest.approx(store["counter_pos"][:2], abs=1e-4) + assert _xyz(handle.bin)[:2] == pytest.approx(store["bin_pos"][:2], abs=1e-4) + + assert len(plano["units"]) >= 3 + assert len({u["zone"] for u in plano["units"].values()}) >= 2 + + base = _xyz(handle.robot) + assert math.hypot(base[0], base[1]) < 0.05 + assert handle.embodiment == "mobile" + + +def test_store_oracle_and_teleport_reset(store_handle): + """T15 Stage A / T16 (ADR-18, ADR-19) on Nexus: store_oracle_state is + n_items*7 in stock order with TC-1 (x, y, z, w) quats matching the spawn + poses, and `teleport_store_reset` restores a disturbed item exactly (the + reset path the store bridge runs on this engine).""" + import math + + from aisle.scenes.store import ( + full_stock, + generate_episode, + spawn_pose, + store_oracle_state, + teleport_store_reset, + ) + + handle = store_handle + stock = full_stock(handle.planogram) + state = store_oracle_state(handle) + assert state.dtype == np.float32 + assert state.shape == (len(stock) * 7,) + x, _y, z, yaw = spawn_pose(handle.planogram, stock[0]) + assert state[0] == pytest.approx(x, abs=1e-5) + assert state[2] == pytest.approx(z, abs=1e-5) + assert abs(float(state[5])) == pytest.approx(abs(math.sin(yaw / 2)), abs=1e-5) # qz + assert abs(float(state[6])) == pytest.approx(abs(math.cos(yaw / 2)), abs=1e-5) # w last + + item = handle.items[stock[0].item_id] + item.set_pos(np.array([0.0, 0.0, 0.5], dtype=np.float32)) + assert store_oracle_state(handle)[0] != pytest.approx(x, abs=1e-3) + teleport_store_reset(handle, handle.episode) + assert np.allclose(store_oracle_state(handle), state, atol=1e-5) + + # T16: the one built scene realizes another scenario by teleport alone: + # S2's emptied slots go to the stash line, the entity set never changes + from aisle.scenes.store import STASH_Y + + s2 = generate_episode(7, "S2") + teleport_store_reset(handle, s2) + emptied = [entry["slot"] for entry in s2["restock"]] + for item_spec in stock: + stashed = _xyz(handle.items[item_spec.item_id])[1] == pytest.approx(STASH_Y, abs=1e-4) + assert stashed == any(item_spec.item_id.startswith(f"{s}#") for s in emptied), item_spec + + +@pytest.mark.xfail( + reason="issue #109 again, in the store path and on both engines: the store " + "builder attaches the wrist camera with a translation-only offset while " + "realized_calibration publishes a cam_to_ee built from wrist_mount_transform, " + "whose wrist_rotation_xyzw is a half turn about x. Measured on nexus: the " + "attached camera looks along the EE link's -Z (back up the arm) while the " + "published block says +Z, elementwise difference 2.0. aisle.scenes.store " + "builds the same identity offset, so Genesis publishes the same wrong block.", + strict=True, +) +def test_store_wrist_cam_to_ee_matches_the_attached_camera(store_handle): + """VER-8/SCN-5 for the store scene on Nexus: the published `cam_to_ee` + must equal the mount the engine realized, exactly as the desk scene's + twin requires. Nothing checks this for a store run, and an L1 or L2 + policy reading the store wrist frame is looking the other way.""" + from aisle.nodes.dora_genesis import realized_calibration + from aisle.verifier.calibration import GL_TO_CV, rotation_from_quat_xyzw + + handle = store_handle + handle.scene.step() + handle.cams["wrist"].render(rgb=True) # an attached camera syncs on render + + link = handle.robot.get_link("hand") + link_rot = rotation_from_quat_xyzw(np.roll(to_numpy(link.get_quat()).reshape(-1)[:4], -1)) + cam_gl = np.asarray(to_numpy(handle.cams["wrist"].transform)).reshape(4, 4) + realized_cv = (link_rot.T @ cam_gl[:3, :3]) @ GL_TO_CV + published = rotation_from_quat_xyzw( + realized_calibration(handle, load_physics(), is_store=True)["wrist"]["cam_to_ee"][ + "quat_xyzw" + ] + ) + assert np.allclose(published, realized_cv, atol=1e-5), ( + "published store cam_to_ee disagrees with the attached camera " + f"(maxdiff {np.abs(published - realized_cv).max():.3e})" + ) + + +def test_lighting_dr_renders_the_per_channel_ambient_it_records(): + """SCN-6 on Nexus: the lighting toggle draws one ambient value per channel + and `dr_applied` records the triple, so the camera must render the triple, + not its mean. Re-rendering with the mean shifts each channel toward it.""" + from aisle.scenes.pharmacy import DRToggle, SceneCfg + from aisle.sim import build_scene + + lit = build_scene( + "nexus", + seed=7, + embodiment="franka", + n_envs=1, + headless=True, + cfg=SceneCfg(lighting=DRToggle(enabled=True, seed=5)), + ) + ambient = np.asarray(lit.dr_applied["ambient"], dtype=np.float64) + assert np.ptp(ambient) > 0.05 and lit.scene.ambient == [float(c) for c in ambient] + camera = lit.cams["overhead"] + drawn = camera.render()[0].reshape(-1, 3).mean(axis=0) + lit.scene.engine.viewer.set_sensor_camera_ambient_color(camera.cam_id, [ambient.mean()] * 3) + lit.scene.sync(force=True) + averaged = camera.render()[0].reshape(-1, 3).mean(axis=0) + brighter = ambient > ambient.mean() + assert np.all((drawn - averaged)[brighter] > 0) and np.all((drawn - averaged)[~brighter] < 0) + + +def test_rebuilding_frees_the_superseded_scene_cameras(): + """A superseded scene's cameras cannot render (one live scene per + process), so the next build frees them: each holds render targets and a + shadow atlas, and a wgpu panic after about ten builds was the result of + keeping them. The superseded camera still refuses with a Python error.""" + from aisle.sim import build_scene + + first = build_scene("nexus", seed=0, embodiment="franka", n_envs=1, headless=True) + for seed in range(1, 16): + latest = build_scene("nexus", seed=seed, embodiment="franka", n_envs=1, headless=True) + assert latest.scene.engine.viewer.num_sensor_cameras() == len(latest.cams) + assert latest.cams["overhead"].render()[0].shape == (480, 640, 3) + with pytest.raises(RuntimeError, match="superseded"): + first.cams["overhead"].render() + + +# --- the live scene: the last build in the file, so nothing supersedes it --- + + +@pytest.fixture(scope="module") +def live(): + """One franka build for the tests that render or step, in file order: + L1 first (it wants the settled build), then the wrist conformance, the + debug camera, and finally the reset-to-rest check, which is the only one + that leaves the arm somewhere else.""" + from aisle.sim import build_scene + + return build_scene("nexus", seed=3, embodiment="franka", n_envs=1, headless=True) + + +def test_cameras_render_all_passes(live): + """SCN-5 / TC-9: overhead 640x480 fov 55, wrist 320x240 fov 70 attached + to the EE link; one pass yields rgb (uint8), metric depth (float32) and + a segmentation map whose ids are the scene's own map (background -1).""" + handle = live + overhead, wrist = handle.cams["overhead"], handle.cams["wrist"] + assert tuple(overhead.res) == (640, 480) and overhead.fov == 55 + assert tuple(wrist.res) == (320, 240) and wrist.fov == 70 + assert wrist._attached_link is not None and wrist._attached_link.name == "hand" + rgb, depth, seg, _ = overhead.render(rgb=True, depth=True, segmentation=True) + assert rgb.shape == (480, 640, 3) and rgb.dtype == np.uint8 + assert depth.shape == (480, 640) and depth.dtype == np.float32 + assert seg.shape == (480, 640) and seg.dtype == np.int64 + zfar = float(handle.scene.camera_planes[1]) + hit = depth[depth < zfar] + assert hit.size > 0 and 0.3 < hit.min() < hit.max() < 2.0 # meters, camera at z=1.2 + # Genesis reads its cleared depth buffer, so a pixel that hit nothing comes + # back at the far plane. Nexus must agree: L2 back-projects a bounding box, + # and a 0.0 there would sit above every real surface (TC-9). + sky = seg == -1 + if sky.any(): + assert np.all(depth[sky] >= zfar) + ids = {int(i) for i in np.unique(seg)} - {-1} + idx_dict = handle.scene.segmentation_idx_dict + assert ids and ids <= set(idx_dict) + # every box is visible from the overhead camera under its own id + entity_idx = {name: entity.idx for name, entity in handle.boxes.items()} + box_ids = {sid for sid, ref in idx_dict.items() if ref != -1 and ref[0] in entity_idx.values()} + assert box_ids <= ids + wrist_rgb = wrist.render()[0] + assert wrist_rgb.shape == (240, 320, 3) + + +def test_l1_estimate_matches_nexus_ground_truth(live): + """TC-9 on Nexus: seg + depth from one render pass, ids from the map the + bridge publishes, pose ESTIMATED with the same estimator and the same + 10 mm gate the Genesis twin uses (tests/sim/test_l1_perception.py). If + the estimate does not land on the ground truth L0 would have given away, + an L1 number on this engine measures the estimator, not the policy.""" + from aisle.nodes.dora_genesis import realized_calibration, segmentation_id_map + from aisle.nodes.segmented_pose import estimate_pose + from aisle.scenes.pharmacy import load_meds + from aisle.verifier.stages import backproject_overhead + + # the Genesis gate (tests/sim/test_l1_perception.py MAX_XY_ERROR_M): a + # regression bound with room for settling wobble, not an accuracy claim + max_xy_error_m = 0.010 + + handle = live + meds = load_meds() + calibration = realized_calibration(handle, load_physics(), is_store=False) + + _rgb, depth, seg, _ = handle.cams["overhead"].render(rgb=True, depth=True, segmentation=True) + seg = np.asarray(seg, dtype=np.int32) # TC-1: the wire type is the contract + depth = np.asarray(depth, dtype=np.float32) + id_map = segmentation_id_map( + handle.scene.segmentation_idx_dict, + {name: entity.idx for name, entity in handle.boxes.items()}, + ) + # the ids are the scene's own segmentation numbering, not entity indices + assert set(id_map) == set(handle.boxes) + assert all( + np.iinfo(np.int32).min < i < np.iinfo(np.int32).max for ids in id_map.values() for i in ids + ) + + def project(d, px): + return backproject_overhead(d, calibration, px) + + errors = {} + for name, entity in handle.boxes.items(): + size = meds[name]["size"] + estimate = estimate_pose( + seg, depth, id_map[name], float(size[2]), project, footprint_m=tuple(size[:2]) + ) + truth = np.asarray(to_numpy(entity.get_pos())).reshape(-1)[:3] + errors[name] = float(np.linalg.norm(np.asarray(estimate["pos"][:2]) - truth[:2])) + assert abs(estimate["pos"][2] - truth[2]) < max_xy_error_m, (name, estimate, truth) + + assert errors, "no meds in the scene to estimate" + assert max(errors.values()) < max_xy_error_m, errors + + +def test_wrist_cam_to_ee_matches_the_attached_nexus_camera(live): + """VER-8/SCN-5 on Nexus, the issue-#109 regression ported: the published + `cam_to_ee` must equal the mount the engine actually realized, in OpenCV. + + The overhead-only check cannot see a wrist camera attached to the wrong + body or composed with the offset in the wrong order: both halves stay + self-consistent while the camera points back up the arm, which is how no + recorded wrist frame ever contained the workspace.""" + from aisle.nodes.dora_genesis import realized_calibration + from aisle.scenes.pharmacy import FRANKA_EE_LINK + from aisle.verifier.calibration import GL_TO_CV, rotation_from_quat_xyzw + + handle = live + handle.scene.step() + # an attached camera's transform is stale until the scene syncs, which + # rendering does (the Genesis twin hit the same trap while diagnosing #109) + handle.cams["wrist"].render(rgb=True) + + link = handle.robot.get_link(FRANKA_EE_LINK) + link_quat_wxyz = to_numpy(link.get_quat()).reshape(-1)[:4] + link_rot = rotation_from_quat_xyzw(np.roll(link_quat_wxyz, -1)) + cam_gl = np.asarray(to_numpy(handle.cams["wrist"].transform)).reshape(4, 4) + + realized_cv = (link_rot.T @ cam_gl[:3, :3]) @ GL_TO_CV + calibration = realized_calibration(handle, load_physics(), is_store=False) + published = rotation_from_quat_xyzw(calibration["wrist"]["cam_to_ee"]["quat_xyzw"]) + assert np.allclose(published, realized_cv, atol=1e-5), ( + "published cam_to_ee disagrees with the attached camera " + f"(maxdiff {np.abs(published - realized_cv).max():.3e})" + ) + + # the realized translation is the published offset, in the link frame + link_pos = to_numpy(link.get_pos()).reshape(-1)[:3] + realized_offset = link_rot.T @ (cam_gl[:3, 3] - link_pos) + assert np.allclose(realized_offset, calibration["wrist"]["cam_to_ee"]["pos"], atol=1e-5), ( + realized_offset, + calibration["wrist"]["cam_to_ee"]["pos"], + ) + + # and the camera must look where the gripper goes: the CV optical axis + # (+Z) is the link's approach axis, not its reverse + optical_axis_in_link = realized_cv[:, 2] + assert optical_axis_in_link[2] == pytest.approx(1.0, abs=1e-5), ( + f"wrist camera looks along {optical_axis_in_link} of the EE link, not its +Z approach axis" + ) + + +def test_debug_camera_attaches_after_build(live): + """ADR-67 operator tooling: the Nexus branch of `add_debug_camera` + (dora_genesis.py) joins a built scene, since Nexus accepts cameras at any + time, renders at the debug resolution and leaves the frozen wire cameras + untouched (SCN-5). The Genesis twin passes the literal "genesis", so this + branch had no coverage at all.""" + from aisle.nodes.dora_genesis import DEBUG_CAMERA_RES, add_debug_camera, parse_debug_view + from aisle.scenes.pharmacy import resolve_layout + + handle = live + eye, lookat = parse_debug_view( + {"AISLE_DEBUG_VIEW": "side"}, resolve_layout(load_physics(), "franka") + ) + camera = add_debug_camera(handle.scene, "nexus", eye, lookat) + frame = np.asarray(camera.render(rgb=True)[0]) + assert frame.shape == (DEBUG_CAMERA_RES[1], DEBUG_CAMERA_RES[0], 3) + assert frame.std() > 0 + assert set(handle.cams) == {"overhead", "wrist"} + assert np.asarray(handle.cams["overhead"].render(rgb=True)[0]).shape == (480, 640, 3) + + +def test_zeroing_velocities_brings_the_arm_to_rest(live): + """BRG-4/TC-6 on Nexus: the bridge calls `zero_all_dofs_velocity` on + every reset, so it must leave the arm actually at rest (every link's + linear and angular velocity zero) without moving it. The implementation + brings the robot to rest by writing its own qpos back, so a coordinate + drift here would teleport the arm on each reset. + + Runs last in the file: it is the one test that leaves the arm sagging.""" + handle = live + robot = handle.robot + for _ in range(20): # no position targets: the arm sags under gravity + handle.scene.step() + moving = max(float(np.abs(np.asarray(robot._state(0)[k])).max()) for k in (3, 4)) + assert moving > 1e-4, moving # there is motion to undo + + # the generalized velocities say the same thing as the link velocities, + # and they are read back rather than reported as zeros + assert np.abs(to_numpy(robot.get_dofs_velocity())).max() > 1e-4 + + before = to_numpy(robot.get_qpos()).reshape(-1).copy() + robot.zero_all_dofs_velocity() + lin, ang = np.asarray(robot._state(0)[3]), np.asarray(robot._state(0)[4]) + assert lin.shape[0] == ang.shape[0] == len(robot.links) + assert np.abs(lin).max() == 0.0, lin + assert np.abs(ang).max() == 0.0, ang + qvel = to_numpy(robot.get_dofs_velocity()).reshape(-1) + assert qvel.shape == (robot.n_dofs,) and np.abs(qvel).max() == 0.0, qvel + assert np.allclose(to_numpy(robot.get_qpos()).reshape(-1), before, atol=1e-6) diff --git a/tests/sim/test_rapier_scene.py b/tests/sim/test_rapier_scene.py new file mode 100644 index 00000000..10c9a1a1 --- /dev/null +++ b/tests/sim/test_rapier_scene.py @@ -0,0 +1,806 @@ +"""Sim acceptance tests for the rapier engine backend (ADR-68): the frozen +pharmacy scene (SPEC 020 SCN-1, SCN-3..5, SCN-7) and the bridge's object +surface (SPEC 030), stepped by rapier on the CPU and rendered through the +Nexus viewer. + +Marker `sim`: imports rapier3d and nexus3d (and genesis for the Franka +asset), runs headless. Skipped when either wheel is missing. + +Scene order: the renderer is the Nexus viewer, where only the newest scene +can render (a later build frees the previous scene's cameras). Tests that +only read a built scene share one module-scoped build; each group that steps, +teleports or renders gets exactly one; the groups that render come last. +""" + +import importlib.util +import math + +import numpy as np +import pytest + +from aisle.scenes.pharmacy import MED_NAMES, load_physics, oracle_state, to_numpy + +pytestmark = [ + pytest.mark.sim, + pytest.mark.skipif( + importlib.util.find_spec("rapier3d") is None + or importlib.util.find_spec("nexus3d") is None + or importlib.util.find_spec("genesis") is None, + reason="the rapier3d or nexus3d wheel, or the sim extra, is not installed", + ), +] + + +# Module-scoped: these tests only read the built scene (no step, no teleport, +# no render), so one build serves them all. +@pytest.fixture(scope="module") +def pristine(): + from aisle.sim import build_scene + + return build_scene("rapier", seed=7, embodiment="franka", n_envs=1, headless=True) + + +def test_engine_render_config_matches_the_nexus_backend(): + """ADR-68: the rapier engine renders through the Nexus viewer, which is + process-wide and configured from nexus_physics.toml. rapier_physics.toml + copies the camera, light and ground blocks so a reader can diff the two + engines in one place; if the copies ever drift, a rapier run would claim a + render configuration it does not have.""" + from aisle.sim.nexus_backend import load_nexus_physics + from aisle.sim.rapier_backend import load_rapier_physics + + nexus, rapier = load_nexus_physics(), load_rapier_physics() + for section in ("camera", "light", "ground"): + assert rapier[section] == nexus[section], section + + +def test_build_determinism(pristine): + """SCN-1, SCN-7 on rapier: the initial oracle_state is a pure function of + the seed (bitwise) and differs across seeds.""" + from aisle.sim import build_scene + + first = oracle_state(pristine) + again = oracle_state(build_scene("rapier", seed=7, embodiment="franka", headless=True)) + other = oracle_state(build_scene("rapier", seed=11, embodiment="franka", headless=True)) + assert first.dtype == np.float32 + assert first.shape == (len(MED_NAMES) * 7,) + assert np.array_equal(first, again) + assert not np.array_equal(first, other) + # rapier owns the physics per scene, so a superseded scene still reads + # back; only its cameras, which live on the shared viewer, are gone + assert np.array_equal(oracle_state(pristine), first) + with pytest.raises(RuntimeError, match="superseded"): + pristine.cams["overhead"].render() + + +def test_placements_match_frozen_sampler(pristine): + """ADR-68: the rapier builder places the boxes exactly where the frozen + sampler says (same pure function, same seed), before any step.""" + from aisle.scenes.pharmacy import resolve_layout, sample_placements + + placements = { + p.name: p for p in sample_placements(7, MED_NAMES, resolve_layout(load_physics(), "franka")) + } + for name, entity in pristine.boxes.items(): + pos = to_numpy(entity.get_pos()).reshape(-1)[:3] + expected = placements[name] + assert np.allclose(pos, [expected.x, expected.y, expected.z], atol=1e-5), name + + +def test_reachability_and_home(pristine): + """SCN-3/SCN-4 (rapier IK) and SCN-1: every placement admits an IK + solution and the robot rests at its home pose. The IK solve runs on the + live multibody, so it must also leave the arm where it found it.""" + assert pristine.reachability_errors == [] + home = np.asarray(load_physics()["embodiment"]["franka"]["home_qpos"], dtype=np.float32) + actual = to_numpy(pristine.robot.get_qpos()).reshape(-1)[: home.shape[0]] + assert np.allclose(actual, home, atol=1e-4) + + +def test_joint_layout_matches_the_render_mirror(pristine): + """ADR-68 / TC-5: the renderer is driven by generalized coordinates, so + the rapier multibody and the Nexus robot that draws it must agree on the + DoF order, the joint names and the link names. They load the same MJCF + through two different importers; a silent disagreement would corrupt + every frame while the physics stayed plausible.""" + robot = pristine.robot + mirror = robot.mirror.robots[0] + actuated = [j for j in robot.joints if j.n_dofs > 0] + assert [j.name for j in actuated] == list(mirror.joint_names) + assert [j.dofs_idx_local[0] for j in actuated] == list(mirror.joint_dof_offsets) + assert [link.name for link in robot.links] == list(mirror.link_names) + assert robot.n_dofs == int(mirror.n_dofs) + + +def test_realized_calibration_matches_nominal(pristine): + """BRG-8 / VER-8: the realized overhead transform is the Genesis look-at + convention, so the published v1 block passes stage 0.""" + from aisle.nodes.dora_genesis import realized_calibration + from aisle.scenes.pharmacy import wrist_mount_transform + from aisle.verifier.calibration import build_calibration_v1 + + physics = load_physics() + realized = realized_calibration(pristine, physics, is_store=False) + wrist = wrist_mount_transform(physics["cameras"], physics["embodiment"]["franka"]) + nominal = build_calibration_v1( + overhead_pos=physics["cameras"]["overhead_pos"], + overhead_lookat=physics["cameras"]["overhead_lookat"], + overhead_resolution=(640, 480), + overhead_fov_deg=55, + wrist_offset_m=wrist[:3, 3].tolist(), + wrist_resolution=(320, 240), + wrist_fov_deg=70, + wrist_mount_rotation_gl=wrist[:3, :3], + ) + assert realized["overhead"]["intrinsics"] == nominal["overhead"]["intrinsics"] + assert np.allclose( + realized["overhead"]["cam_to_base"]["pos"], + nominal["overhead"]["cam_to_base"]["pos"], + atol=1e-5, + ) + assert np.allclose( + realized["overhead"]["cam_to_base"]["quat_xyzw"], + nominal["overhead"]["cam_to_base"]["quat_xyzw"], + atol=1e-4, + ) + + +def test_no_interpenetration(pristine): + """SCN-3 on rapier: the initial placements the frozen sampler produced + are realized free of interpenetration (pairwise AABBs of the meds).""" + aabbs = [] + for name, entity in pristine.boxes.items(): + pos = to_numpy(entity.get_pos()).reshape(-1)[:3] + half = np.asarray(pristine.med_sizes[name]) / 2.0 + aabbs.append((name, pos - half, pos + half)) + for i, (name_a, lo_a, hi_a) in enumerate(aabbs): + for name_b, lo_b, hi_b in aabbs[i + 1 :]: + overlap = np.all(lo_a < hi_b) and np.all(lo_b < hi_a) + assert not overlap, (name_a, name_b) + + +def test_oracle_quaternions_are_xyzw(pristine): + """TC-1 on rapier: oracle_state quaternions are (x, y, z, w) wire order. + The rapier readback is w-first like Genesis's, so a missing roll here + would publish the same wrong wire order the Genesis test pins.""" + state = oracle_state(pristine) + for i in range(len(MED_NAMES)): + quat = state[i * 7 + 3 : i * 7 + 7] + assert abs(quat[3]) > 0.99, quat # w last + assert np.all(np.abs(quat[:3]) < 0.1), quat + + +def test_boxes_follow_oracle_order(pristine): + """SCN-1 on rapier: the boxes dict is in the fixed meds.toml order, which + is the oracle_state layout (TC table).""" + assert list(pristine.boxes) == MED_NAMES + + +# --- stepping: one build, disturbed in place -------------------------------- + + +@pytest.fixture(scope="module") +def stepping(): + """One franka build for the tests that step but never render, in file + order: the settling check, which leaves the scene as it found it, then + the teleport and control checks, which do not.""" + from aisle.sim import build_scene + + return build_scene("rapier", seed=7, embodiment="franka", n_envs=1, headless=True) + + +def test_resting_boxes_hold_still(stepping): + """SCN-3 on rapier: boxes at rest on the boards must not creep past the + oracle's knock threshold over an episode. One second of settling must + stay well under a millimeter per box, as on the other two engines.""" + handle = stepping + robot = handle.robot + robot.control_dofs_position(to_numpy(robot.get_qpos()).reshape(-1)) + start = oracle_state(handle).reshape(-1, 7)[:, :3] + for _ in range(100): + handle.scene.step() + drift = np.linalg.norm(oracle_state(handle).reshape(-1, 7)[:, :3] - start, axis=1) + assert drift.max() < 1.0e-3, drift + + +def test_step_teleport_and_control(stepping): + """BRG-2/BRG-4: stepping advances physics, a teleport reset lands a box + exactly where asked with zero velocity, and position control holds the + home pose under gravity.""" + handle = stepping + robot = handle.robot + home = to_numpy(robot.get_qpos()).reshape(-1).copy() + robot.control_dofs_position(home) + box = next(iter(handle.boxes.values())) + robot_before = to_numpy(robot.get_link("hand").get_pos()).reshape(-1) + for _ in range(50): + handle.scene.step() + after = to_numpy(robot.get_qpos()).reshape(-1) + assert np.allclose(after[:7], home[:7], atol=0.02) + assert np.allclose( + to_numpy(robot.get_link("hand").get_pos()).reshape(-1), robot_before, atol=0.02 + ) + box.set_pos(np.array([0.3, -0.45, 0.3], dtype=np.float32)) + box.set_quat(np.array([1.0, 0.0, 0.0, 0.0], dtype=np.float32)) + box.zero_all_dofs_velocity() + assert np.allclose(to_numpy(box.get_pos()).reshape(-1), [0.3, -0.45, 0.3], atol=1e-5) + handle.scene.step() + pos = to_numpy(box.get_pos()).reshape(-1) + assert pos[2] < 0.3 and pos[2] > 0.29 # falling under gravity, one step later + + +def test_a_teleported_arm_wakes_up(stepping): + """TC-6 on rapier: an arm rapier put to sleep after settling must move on + the step after a reset writes its joints or re-bases its root.""" + handle = stepping + robot = handle.robot + art = robot.arts[0] + links = [art.world.rigid_bodies.get(h) for h in art.link_bodies] + for write in ( + lambda: robot.set_qpos(to_numpy(robot.get_qpos()).reshape(-1) + 0.05), + lambda: robot.set_pos(to_numpy(robot.get_pos()).reshape(-1)), + ): + for body in links: + body.sleep() + assert all(body.is_sleeping for body in links) + write() + assert not any(body.is_sleeping for body in links) + + +def test_a_malformed_gain_profile_is_refused(stepping): + """A gains list that is neither one value nor one per DoF is refused, not + silently broadcast from its first entry.""" + robot = stepping.robot + robot.set_dofs_kp([100.0], dofs_idx_local=[0, 1, 2]) + assert np.all(robot.kp[[0, 1, 2]] == 100.0) + with pytest.raises(ValueError, match="2 entries for 3 DoFs"): + robot.set_dofs_kp([1.0, 2.0], dofs_idx_local=[0, 1, 2]) + with pytest.raises(ValueError, match="kv has 2 entries"): + robot.set_dofs_kv([1.0, 2.0], dofs_idx_local=[0, 1, 2]) + + +def test_inverse_kinematics_moves_only_the_listed_dofs(stepping): + """Genesis `dofs_idx_local` on rapier: an IK solve over the first arm + joints moves them toward the target and leaves every other DoF alone.""" + robot = stepping.robot + hand = robot.get_link("hand") + start = to_numpy(robot.get_qpos()).reshape(-1).copy() + target = to_numpy(hand.get_pos()).reshape(-1) + np.array([0.03, 0.0, -0.03]) + quat = to_numpy(hand.get_quat()).reshape(-1) + qpos = to_numpy( + robot.inverse_kinematics(link=hand, pos=target, quat=quat, dofs_idx_local=[0, 1, 2, 3]) + ).reshape(-1) + assert not np.allclose(qpos[:4], start[:4], atol=1e-4) + assert np.allclose(qpos[4:], start[4:], atol=1e-6) + assert np.allclose(to_numpy(robot.get_qpos()).reshape(-1), start, atol=1e-6) + + +def test_perf_stats_reads_the_step_counters(stepping): + """The bridge's timing sidecar gets rapier's own step time, and no GPU + number, since the step is a synchronous CPU call.""" + stepping.scene.step() + stats = stepping.scene.perf_stats() + assert stats["cpu_ms"] > 0.0 and stats["solver_ms"] > 0.0 + assert "gpu_ms" not in stats + + +def test_lighting_dr_renders_the_per_channel_ambient_it_records(): + """SCN-6 on rapier: the render mirror takes the recorded ambient triple, + not its mean (the Nexus test checks the pixels).""" + from aisle.scenes.pharmacy import DRToggle, SceneCfg + from aisle.sim import build_scene + + lit = build_scene( + "rapier", + seed=7, + embodiment="franka", + n_envs=1, + headless=True, + cfg=SceneCfg(lighting=DRToggle(enabled=True, seed=5)), + ) + ambient = [float(c) for c in lit.dr_applied["ambient"]] + assert len(set(ambient)) == 3 and lit.scene.render.ambient == ambient + + +# --- what this engine exists for: a reproducible step ----------------------- + + +@pytest.fixture(scope="module") +def twin_worlds(): + """A batched build: rapier has no batching, so `n_envs=2` is literally two + identically constructed `PhysicsWorld`s, which is exactly the comparison + the determinism test wants. Used in file order: the batch check, then the + bitwise step check, then the snapshot round trip.""" + from aisle.sim import build_scene + + return build_scene("rapier", seed=7, embodiment="franka", n_envs=2, headless=True) + + +def test_batched_build_oracle_covers_all_envs(twin_worlds): + """SCN-1 on rapier: a batched build reports every env and identical + initial placements.""" + state = oracle_state(twin_worlds) + assert state.shape == (2, len(MED_NAMES) * 7) + assert np.array_equal(state[0], state[1]) + qpos = to_numpy(twin_worlds.robot.get_qpos()) + assert qpos.shape == (2, twin_worlds.robot.n_dofs) + + +def test_bitwise_step_determinism(twin_worlds): + """CON-5 and the point of ADR-68: two identically seeded worlds stepped + the same number of times must produce bitwise identical state. Genesis + reduces on Metal and Nexus colours constraints with GPU atomics, so + neither can promise this; rapier stepped single-threaded can, and a + regression that made it merely near-identical would remove the only + reproducible engine AISLE has. + + The run is not a trivial one: a box is dropped so the contact solver, not + just free flight, is part of what has to agree.""" + handle = twin_worlds + robot = handle.robot + robot.control_dofs_position(to_numpy(robot.get_qpos())) + box = next(iter(handle.boxes.values())) + box.set_pos(np.tile(np.array([0.3, -0.45, 0.4], dtype=np.float32), (2, 1))) + box.zero_all_dofs_velocity() + for _ in range(200): + handle.scene.step() + state = oracle_state(handle) + assert state.shape[0] == 2 + assert np.array_equal(state[0], state[1]), np.abs(state[0] - state[1]).max() + qpos = to_numpy(robot.get_qpos()) + assert np.array_equal(qpos[0], qpos[1]), np.abs(qpos[0] - qpos[1]).max() + # the box really did land, so the contact solver was exercised + assert float(state[0][2]) < 0.4 + + +def test_snapshot_restore_round_trip(twin_worlds): + """CON-5 on rapier (ADR-68): a world serializes to bytes and restores to + a world that steps to the same state. That is the engine-level state + capture Genesis and Nexus have no equivalent of, and the foundation a + replay or a mid-episode fork would stand on.""" + import rapier3d + + handle = twin_worlds + world = handle.scene.worlds[0] + box = next(iter(handle.boxes.values())) + body = box.handles[0] + + blob = world.snapshot() + assert isinstance(blob, bytes) and len(blob) > 0 + restored = rapier3d.PhysicsWorld.restore(blob) + + def pose(w): + b = w.rigid_bodies.get(body) + return (tuple(b.translation), (b.rotation.w, b.rotation.i, b.rotation.j, b.rotation.k)) + + assert pose(restored) == pose(world) + for _ in range(50): + world.step() + restored.step() + assert pose(restored) == pose(world) + + +# --- the SO-101 URDF -------------------------------------------------------- + + +@pytest.fixture(scope="module") +def so101_handle(): + from aisle.sim import build_scene + + return build_scene("rapier", seed=3, embodiment="so101", n_envs=1, headless=True) + + +def test_so101_urdf_matches_frozen_chain(so101_handle): + """ADR-68: the imported SO-101 kinematics agree with the frozen URDF + chain (`aisle.kinematics`) at several configurations, so grasp planning + and the guard see the same arm the physics does.""" + from aisle.embodiment import profile_dof_indices + from aisle.kinematics import so101_chain + + robot = so101_handle.robot + chain = so101_chain() + wire = profile_dof_indices(robot, load_physics()["embodiment"]["so101"]) + rng = np.random.default_rng(1) + for _ in range(3): + q = rng.uniform(-1.0, 1.0, size=5) + expected, _ = chain.forward(q) + native = np.zeros(robot.n_dofs, dtype=np.float64) + native[list(wire[:5])] = q + robot.set_qpos(native) + pos = to_numpy(robot.get_link("gripper_frame_link").get_pos()).reshape(-1)[:3] + assert np.allclose(pos, expected, atol=2e-4), (q, pos, expected) + + +def test_so101_home_and_gripper_gains(so101_handle): + """SCN-1 / TC-5: the SO-101 rests at its profile home in wire order and + carries the profile's gripper gains. The gains are the ones the servo + actually runs, since rapier installs them as multibody motors.""" + from aisle.embodiment import profile_dof_indices + + profile = load_physics()["embodiment"]["so101"] + robot = so101_handle.robot + wire = profile_dof_indices(robot, profile) + robot.set_qpos(np.asarray(profile["home_qpos"], dtype=np.float64)[list(np.argsort(wire))]) + qpos = to_numpy(robot.get_qpos()).reshape(-1) + assert np.allclose(qpos[list(wire)], profile["home_qpos"], atol=1e-4) + assert np.isclose(robot.kp[wire[-1]], profile["gripper_kp"][0]) + assert np.isclose(robot.kv[wire[-1]], profile["gripper_kv"][0]) + assert so101_handle.reachability_errors == [] + + +# --- the mobile profile (SPEC 210) ------------------------------------------ + + +@pytest.fixture(scope="module") +def mobile_scene(): + """One mobile-profile build for the re-basing tests (they move the base, + never the arm, so they compose on one scene).""" + from aisle.sim import build_scene + + return build_scene("rapier", seed=0, embodiment="mobile", n_envs=1, headless=True) + + +def test_mobile_builds_franka_arm_and_rebases(mobile_scene): + """MOB-4 and ADR-13 on rapier: the mobile profile builds the same franka + arm (it rests at the frozen franka home, as the fixed-base build does), + and re-basing through `set_pos`/`set_quat` moves the whole arm's world + mount while the arm stays base-relative (joint FK unchanged). + + rapier roots the arm at a fixed link whose pose the multibody owns, so + a re-base has to be read back into the articulation and pushed down the + chain; otherwise the links stay behind while the root slides away.""" + from aisle.mobility.base import integrate_base_pose + from aisle.nodes.ik_trajectory import fk_tcp + + mobile = mobile_scene + robot = mobile.robot + home = np.asarray(load_physics()["embodiment"]["franka"]["home_qpos"], dtype=np.float32) + qpos = to_numpy(robot.get_qpos()).reshape(-1) + assert np.allclose(qpos[: home.shape[0]], home, atol=1e-4) # MOB-4: the same arm + tcp_before = fk_tcp(qpos[:7]) + hand_before = to_numpy(robot.get_link("hand").get_pos()).reshape(-1)[:3] + # hold the arm as the bridge does every tick: this test is about the base + # moving, and an uncommanded rapier arm sags a centimetre in one step + robot.control_dofs_position(qpos) + + pose = [0.0, 0.0, 0.0] + for _ in range(10): # 1 m forward at 1 m/s over 0.1 s ticks + pose = integrate_base_pose(pose, [1.0, 0.0], dt=0.1) + robot.set_pos(np.array([pose[0], pose[1], 0.0], dtype=np.float32)) + mobile.scene.step() + + assert to_numpy(robot.get_pos()).reshape(-1)[:2] == pytest.approx([1.0, 0.0], abs=1e-3) + hand_after = to_numpy(robot.get_link("hand").get_pos()).reshape(-1)[:3] + assert hand_after - hand_before == pytest.approx([1.0, 0.0, 0.0], abs=1e-2) + assert fk_tcp(to_numpy(robot.get_qpos()).reshape(-1)[:7]) == pytest.approx(tcp_before, abs=1e-2) + + # a yaw re-base rotates the arm about the base origin, which stays put + robot.set_quat( + np.array([math.cos(math.pi / 4), 0.0, 0.0, math.sin(math.pi / 4)], dtype=np.float32) + ) + mobile.scene.step() + root_now = to_numpy(robot.get_pos()).reshape(-1)[:3] + assert root_now[:2] == pytest.approx([1.0, 0.0], abs=1e-3) + hand_yawed = to_numpy(robot.get_link("hand").get_pos()).reshape(-1)[:3] + offset = hand_after - np.asarray([1.0, 0.0, 0.0]) # base-frame hand offset + expected = root_now + np.asarray([-offset[1], offset[0], offset[2]]) # +90 deg about z + assert hand_yawed == pytest.approx(expected, abs=1e-2), (hand_yawed, expected) + + +def test_mobile_rebase_survives_set_pos_then_set_quat(mobile_scene): + """ADR-13/MOB-4 on rapier: the bridge re-bases with `set_pos` immediately + followed by `set_quat`, with no step between, so the pair must compose. + The Nexus twin regressed here because each half read the other back from + a buffer the step had not refreshed yet.""" + mobile = mobile_scene + robot = mobile.robot + yaw = math.pi / 3 + robot.set_pos(np.array([2.0, 0.5, 0.0], dtype=np.float32)) + robot.set_quat(np.array([math.cos(yaw / 2), 0.0, 0.0, math.sin(yaw / 2)], dtype=np.float32)) + mobile.scene.step() + assert to_numpy(robot.get_pos()).reshape(-1)[:2] == pytest.approx([2.0, 0.5], abs=1e-3) + + +# --- the store scene on rapier (SPEC 200) ----------------------------------- + + +def _xyz(entity) -> list[float]: + return [float(v) for v in to_numpy(entity.get_pos()).reshape(-1)[:3]] + + +def _assert_yaw(entity, yaw: float, label) -> None: + """The entity's physical orientation matches the composed world yaw: + quaternion equal up to sign, and the rotated +x (front face) points where + the slot faces.""" + from aisle.scenes.store import yaw_quat_wxyz + + got = [float(v) for v in to_numpy(entity.get_quat()).reshape(-1)[:4]] + want = list(yaw_quat_wxyz(yaw)) + if got[0] * want[0] + got[3] * want[3] < 0: # q and -q are the same rotation + want = [-c for c in want] + assert got == pytest.approx(want, abs=1e-5), (label, got, want) + w, _, _, z = got + front = (1 - 2 * z * z, 2 * w * z) + assert front[0] == pytest.approx(math.cos(yaw), abs=1e-5), label + assert front[1] == pytest.approx(math.sin(yaw), abs=1e-5), label + + +@pytest.fixture(scope="module") +def store_handle(): + """One store build, shared in file order: the planogram check reads it, + then the oracle/teleport check disturbs and resets it.""" + from aisle.sim import build_store + + return build_store("rapier", seed=1, scenario="S1") + + +def test_store_build_realizes_the_planogram(store_handle): + """RS-1/RS-2 on rapier (ADR-68, T16/ADR-19): the rapier store is + generated from the same planogram.toml as the Genesis one: the full stock + spawns at its slots' world template poses with the composed unit yaw, the + shelf boards carry that yaw too, the bin holds one item per category, + counter and bin sit where the planogram puts them, and the mobile-profile + robot starts at the store-frame origin.""" + from aisle.scenes.store import full_stock, slot_world_pose + + handle = store_handle + plano = handle.planogram + + assert list(handle.items) == [item.item_id for item in full_stock(plano)] + for slot_id, slot in plano["slots"].items(): + item_id = f"{slot_id}#0" + assert item_id in handle.items, f"slot {slot_id} not stocked" + world, yaw = slot_world_pose(plano, slot_id) + size_z = handle.med_sizes[slot["category"]][2] + pos = _xyz(handle.items[item_id]) + assert pos[0] == pytest.approx(world[0], abs=1e-4), slot_id + assert pos[1] == pytest.approx(world[1], abs=1e-4), slot_id + assert pos[2] == pytest.approx(world[2] + size_z / 2, abs=1e-4), slot_id + _assert_yaw(handle.items[item_id], yaw, slot_id) + + boards = {entity.name: entity for entity in handle.scene.entities} + for unit_id, unit in plano["units"].items(): + for level in range(len(plano["store"]["unit_geometry"]["level_heights"])): + _assert_yaw(boards[f"{unit_id}_level_{level}"], unit["yaw"], unit_id) + + store = plano["store"] + bin_top = store["bin_pos"][2] + store["bin_size"][2] / 2 + bin_items = [i for i in handle.items if i.startswith("bin#")] + assert len(bin_items) == len(handle.med_sizes) + for item_id in bin_items: + pos = _xyz(handle.items[item_id]) + size_z = handle.med_sizes[handle.categories[item_id]][2] + assert pos[2] == pytest.approx(bin_top + size_z / 2, abs=1e-4), item_id + + assert _xyz(handle.counter)[:2] == pytest.approx(store["counter_pos"][:2], abs=1e-4) + assert _xyz(handle.bin)[:2] == pytest.approx(store["bin_pos"][:2], abs=1e-4) + + base = _xyz(handle.robot) + assert math.hypot(base[0], base[1]) < 0.05 + assert handle.embodiment == "mobile" + + +def test_store_oracle_and_teleport_reset(store_handle): + """T15 Stage A / T16 (ADR-18, ADR-19) on rapier: store_oracle_state is + n_items*7 in stock order with TC-1 (x, y, z, w) quats matching the spawn + poses, and `teleport_store_reset` restores a disturbed item exactly (the + reset path the store bridge runs on this engine).""" + from aisle.scenes.store import ( + STASH_Y, + full_stock, + generate_episode, + spawn_pose, + store_oracle_state, + teleport_store_reset, + ) + + handle = store_handle + stock = full_stock(handle.planogram) + state = store_oracle_state(handle) + assert state.dtype == np.float32 + assert state.shape == (len(stock) * 7,) + x, _y, z, yaw = spawn_pose(handle.planogram, stock[0]) + assert state[0] == pytest.approx(x, abs=1e-5) + assert state[2] == pytest.approx(z, abs=1e-5) + assert abs(float(state[5])) == pytest.approx(abs(math.sin(yaw / 2)), abs=1e-5) # qz + assert abs(float(state[6])) == pytest.approx(abs(math.cos(yaw / 2)), abs=1e-5) # w last + + item = handle.items[stock[0].item_id] + item.set_pos(np.array([0.0, 0.0, 0.5], dtype=np.float32)) + assert store_oracle_state(handle)[0] != pytest.approx(x, abs=1e-3) + teleport_store_reset(handle, handle.episode) + assert np.allclose(store_oracle_state(handle), state, atol=1e-5) + + # T16: the one built scene realizes another scenario by teleport alone + s2 = generate_episode(7, "S2") + teleport_store_reset(handle, s2) + emptied = [entry["slot"] for entry in s2["restock"]] + for item_spec in stock: + stashed = _xyz(handle.items[item_spec.item_id])[1] == pytest.approx(STASH_Y, abs=1e-4) + assert stashed == any(item_spec.item_id.startswith(f"{s}#") for s in emptied), item_spec + + +# --- the live scene: the last build in the file, so nothing supersedes it --- + + +@pytest.fixture(scope="module") +def live(): + """One franka build for the tests that render, in file order: the camera + passes, the L1 estimate, the wrist conformance, the debug camera, and + finally the reset-to-rest check, which is the only one that leaves the + arm somewhere else.""" + from aisle.sim import build_scene + + return build_scene("rapier", seed=3, embodiment="franka", n_envs=1, headless=True) + + +def test_cameras_render_all_passes(live): + """SCN-5 / TC-9: overhead 640x480 fov 55, wrist 320x240 fov 70 attached + to the EE link; one pass yields rgb (uint8), metric depth (float32) and + a segmentation map whose ids are the scene's own map (background -1). + The poses behind those pixels were computed by rapier, not by the + renderer's solver, which is the whole bet of ADR-68.""" + handle = live + overhead, wrist = handle.cams["overhead"], handle.cams["wrist"] + assert tuple(overhead.res) == (640, 480) and overhead.fov == 55 + assert tuple(wrist.res) == (320, 240) and wrist.fov == 70 + assert wrist._attached_link is not None and wrist._attached_link.name == "hand" + rgb, depth, seg, _ = overhead.render(rgb=True, depth=True, segmentation=True) + assert rgb.shape == (480, 640, 3) and rgb.dtype == np.uint8 + assert depth.shape == (480, 640) and depth.dtype == np.float32 + assert seg.shape == (480, 640) and seg.dtype == np.int64 + hit = depth[depth > 0] + assert hit.size > 0 and 0.3 < hit.min() < hit.max() < 2.0 # meters, camera at z=1.2 + ids = {int(i) for i in np.unique(seg)} - {-1} + idx_dict = handle.scene.segmentation_idx_dict + assert ids and ids <= set(idx_dict) + entity_idx = {name: entity.idx for name, entity in handle.boxes.items()} + box_ids = {sid for sid, ref in idx_dict.items() if ref != -1 and ref[0] in entity_idx.values()} + assert box_ids <= ids + wrist_rgb = wrist.render()[0] + assert wrist_rgb.shape == (240, 320, 3) + + # a pose written with no solver step must reach the pixels: that is the + # only thing keeping the render honest on this engine + box = next(iter(handle.boxes.values())) + start = to_numpy(box.get_pos()).reshape(-1)[:3].copy() + before = overhead.render(rgb=True)[0].copy() + box.set_pos(start + np.array([0.0, 0.0, 0.15], dtype=np.float32)) + after = overhead.render(rgb=True)[0] + assert int(np.count_nonzero(np.any(before != after, axis=-1))) > 100 + box.set_pos(start) + box.zero_all_dofs_velocity() + + +def test_l1_estimate_matches_rapier_ground_truth(live): + """TC-9 on rapier: seg + depth from one render pass, ids from the map the + bridge publishes, pose ESTIMATED with the same estimator and the same + 10 mm gate the Genesis twin uses (tests/sim/test_l1_perception.py). If + the estimate does not land on the ground truth L0 would have given away, + an L1 number on this engine measures the estimator, not the policy.""" + from aisle.nodes.dora_genesis import realized_calibration, segmentation_id_map + from aisle.nodes.segmented_pose import estimate_pose + from aisle.scenes.pharmacy import load_meds + from aisle.verifier.stages import backproject_overhead + + max_xy_error_m = 0.010 + + handle = live + meds = load_meds() + calibration = realized_calibration(handle, load_physics(), is_store=False) + + _rgb, depth, seg, _ = handle.cams["overhead"].render(rgb=True, depth=True, segmentation=True) + seg = np.asarray(seg, dtype=np.int32) # TC-1: the wire type is the contract + depth = np.asarray(depth, dtype=np.float32) + id_map = segmentation_id_map( + handle.scene.segmentation_idx_dict, + {name: entity.idx for name, entity in handle.boxes.items()}, + ) + assert set(id_map) == set(handle.boxes) + + def project(d, px): + return backproject_overhead(d, calibration, px) + + errors = {} + for name, entity in handle.boxes.items(): + size = meds[name]["size"] + estimate = estimate_pose( + seg, depth, id_map[name], float(size[2]), project, footprint_m=tuple(size[:2]) + ) + truth = np.asarray(to_numpy(entity.get_pos())).reshape(-1)[:3] + errors[name] = float(np.linalg.norm(np.asarray(estimate["pos"][:2]) - truth[:2])) + assert abs(estimate["pos"][2] - truth[2]) < max_xy_error_m, (name, estimate, truth) + + assert errors, "no meds in the scene to estimate" + assert max(errors.values()) < max_xy_error_m, errors + + +def test_wrist_cam_to_ee_matches_the_attached_camera(live): + """VER-8/SCN-5 on rapier, the issue-#109 regression ported: the published + `cam_to_ee` must equal the mount the engine actually realized, in OpenCV. + + The camera rides a render body whose pose rapier writes, so this also + pins that the attachment follows externally computed link poses.""" + from aisle.nodes.dora_genesis import realized_calibration + from aisle.scenes.pharmacy import FRANKA_EE_LINK + from aisle.verifier.calibration import GL_TO_CV, rotation_from_quat_xyzw + + handle = live + handle.scene.step() + handle.cams["wrist"].render(rgb=True) # an attached camera syncs on render + + link = handle.robot.get_link(FRANKA_EE_LINK) + link_quat_wxyz = to_numpy(link.get_quat()).reshape(-1)[:4] + link_rot = rotation_from_quat_xyzw(np.roll(link_quat_wxyz, -1)) + cam_gl = np.asarray(to_numpy(handle.cams["wrist"].transform)).reshape(4, 4) + + realized_cv = (link_rot.T @ cam_gl[:3, :3]) @ GL_TO_CV + calibration = realized_calibration(handle, load_physics(), is_store=False) + published = rotation_from_quat_xyzw(calibration["wrist"]["cam_to_ee"]["quat_xyzw"]) + assert np.allclose(published, realized_cv, atol=1e-5), ( + "published cam_to_ee disagrees with the attached camera " + f"(maxdiff {np.abs(published - realized_cv).max():.3e})" + ) + + link_pos = to_numpy(link.get_pos()).reshape(-1)[:3] + realized_offset = link_rot.T @ (cam_gl[:3, 3] - link_pos) + assert np.allclose(realized_offset, calibration["wrist"]["cam_to_ee"]["pos"], atol=1e-5), ( + realized_offset, + calibration["wrist"]["cam_to_ee"]["pos"], + ) + + optical_axis_in_link = realized_cv[:, 2] + assert optical_axis_in_link[2] == pytest.approx(1.0, abs=1e-5), ( + f"wrist camera looks along {optical_axis_in_link} of the EE link, not its +Z approach axis" + ) + + +def test_debug_camera_attaches_after_build(live): + """ADR-68 operator tooling: the rapier branch of `add_debug_camera` + (dora_genesis.py) joins a built scene, since it renders through the same + Nexus viewer, renders at the debug resolution and leaves the frozen wire + cameras untouched (SCN-5).""" + from aisle.nodes.dora_genesis import DEBUG_CAMERA_RES, add_debug_camera, parse_debug_view + from aisle.scenes.pharmacy import resolve_layout + + handle = live + eye, lookat = parse_debug_view( + {"AISLE_DEBUG_VIEW": "side"}, resolve_layout(load_physics(), "franka") + ) + camera = add_debug_camera(handle.scene, "rapier", eye, lookat) + frame = np.asarray(camera.render(rgb=True)[0]) + assert frame.shape == (DEBUG_CAMERA_RES[1], DEBUG_CAMERA_RES[0], 3) + assert frame.std() > 0 + assert set(handle.cams) == {"overhead", "wrist"} + assert np.asarray(handle.cams["overhead"].render(rgb=True)[0]).shape == (480, 640, 3) + + +def test_zeroing_velocities_brings_the_arm_to_rest(live): + """BRG-4/TC-6 on rapier: the bridge calls `zero_all_dofs_velocity` on + every reset, so it must leave the arm actually at rest (generalized and + per-link velocities zero) without moving it. + + Runs last in the file: it is the one test that leaves the arm sagging.""" + handle = live + robot = handle.robot + art = robot.arts[0] + for _ in range(20): # no position targets: the arm sags under gravity + handle.scene.step() + + def link_velocities(): + bodies = art.world.rigid_bodies + rows = [] + for h in art.link_bodies: + body = bodies.get(h) + rows.append([*body.linvel, *body.angvel]) + return np.asarray(rows, dtype=np.float64) + + assert np.abs(link_velocities()).max() > 1e-4 # there is motion to undo + + before = to_numpy(robot.get_qpos()).reshape(-1).copy() + robot.zero_all_dofs_velocity() + assert np.abs(np.asarray(robot.get_dofs_velocity())).max() == 0.0 + velocities = link_velocities() + assert velocities.shape[0] == len(robot.links) + assert np.abs(velocities).max() == 0.0, velocities + assert np.allclose(to_numpy(robot.get_qpos()).reshape(-1), before, atol=1e-6) diff --git a/tests/sim/test_scene.py b/tests/sim/test_scene.py index 836b105b..077c2742 100644 --- a/tests/sim/test_scene.py +++ b/tests/sim/test_scene.py @@ -195,3 +195,21 @@ def test_physics_stability_at_substeps_one(): assert abs(z - initial[name]) < 0.02, (name, initial[name], z) vel = to_numpy(entity.get_dofs_velocity()).reshape(-1) assert float(abs(vel).max()) < 0.5, (name, vel) + + +def test_debug_camera_attaches_after_build(handle): + """ADR-67 operator tooling: AISLE_DEBUG_VIEW's camera joins a built + Genesis scene through the visualizer's debug path (the frozen builder + fixes the wire cameras at build, SCN-5), renders at the debug resolution + and leaves the overhead camera untouched.""" + from aisle.nodes.dora_genesis import DEBUG_CAMERA_RES, add_debug_camera, parse_debug_view + + eye, lookat = parse_debug_view( + {"AISLE_DEBUG_VIEW": "side"}, resolve_layout(load_physics(), "franka") + ) + camera = add_debug_camera(handle.scene, "genesis", eye, lookat) + frame = np.asarray(camera.render(rgb=True)[0]) + assert frame.shape == (DEBUG_CAMERA_RES[1], DEBUG_CAMERA_RES[0], 3) + assert frame.std() > 0 + assert set(handle.cams) == {"overhead", "wrist"} + assert np.asarray(handle.cams["overhead"].render(rgb=True)[0]).shape == (480, 640, 3) diff --git a/tests/unit/test_cmd_coalescing.py b/tests/unit/test_cmd_coalescing.py index b89733fe..8b8233fb 100644 --- a/tests/unit/test_cmd_coalescing.py +++ b/tests/unit/test_cmd_coalescing.py @@ -158,6 +158,7 @@ def test_bridge_info_shape(): perception="L0", segmentation_ids={}, sim_backend="metal", + sim_engine="genesis", ) ) assert info == { @@ -176,6 +177,8 @@ def test_bridge_info_shape(): "perception": "L0", "segmentation_ids": {}, "sim_backend": "metal", + # ADR-67: the engine that realized the scene + "sim_engine": "genesis", } assert info["platform"] assert info["calibration"]["calibration_version"] == 1 @@ -198,6 +201,16 @@ def test_bridge_config_from_env(): with pytest.raises(ValueError, match="simulation backend"): parse_bridge_config({"AISLE_SIM_BACKEND": "auto"}) + # ADR-67: the engine is graph-declared like the rung; unknown names and + # backends the engine cannot run are refused, never defaulted + assert parse_bridge_config({}).sim_engine == "genesis" + assert parse_bridge_config({"AISLE_SIM_ENGINE": "nexus"}).sim_engine == "nexus" + nexus_cfg = parse_bridge_config({"AISLE_SIM_ENGINE": "nexus", "AISLE_SIM_BACKEND": "webgpu"}) + assert (nexus_cfg.sim_engine, nexus_cfg.sim_backend) == ("nexus", "webgpu") + with pytest.raises(ValueError, match="simulation engine"): + parse_bridge_config({"AISLE_SIM_ENGINE": "bullet"}) + with pytest.raises(ValueError, match="simulation backend"): + parse_bridge_config({"AISLE_SIM_ENGINE": "genesis", "AISLE_SIM_BACKEND": "webgpu"}) def test_step_without_reset_defaults_off(): @@ -459,6 +472,7 @@ def test_bridge_info_carries_the_l1_id_map(): perception="L1", segmentation_ids={"amoxicillin": [16], "ibuprofen": [17]}, sim_backend="cuda", + sim_engine="genesis", ) ) assert info["perception"] == "L1" @@ -526,15 +540,17 @@ def test_publish_is_wired_to_the_gate_not_to_an_inline_check(): def test_bridge_info_requires_the_rung_rather_than_defaulting_it(): - """TC-9/BRG-8: `perception` and `segmentation_ids` are REQUIRED arguments. - A defaulted rung would attest "L0" in the trace for a run that executed L1 - — the recorded-vs-actual divergence the rung refusal and the env scrub - exist to prevent, and one no test can catch because the default is a valid - value. Same discipline the docstring already argues for calibration.""" + """TC-9/BRG-8/ADR-67: `perception`, `segmentation_ids` and `sim_engine` are + REQUIRED arguments. A defaulted rung would attest "L0" in the trace for a + run that executed L1 — the recorded-vs-actual divergence the rung refusal + and the env scrub exist to prevent, and one no test can catch because the + default is a valid value. A defaulted engine attests "genesis" for a run + that executed on Nexus for exactly the same reason. Same discipline the + docstring already argues for calibration.""" import inspect sig = inspect.signature(make_bridge_info) - for name in ("perception", "segmentation_ids"): + for name in ("perception", "segmentation_ids", "sim_engine"): assert sig.parameters[name].default is inspect.Parameter.empty, name @@ -594,3 +610,68 @@ def test_l1_refuses_an_unusable_segmentation_id_map(): require_usable_segmentation_ids({"amoxicillin": [], "ibuprofen": [17]}, "L1") # below L1 there is no map to require: L0 publishes ground truth instead require_usable_segmentation_ids({}, "L0") + + +# -- ADR-67: timing sidecar --------------------------------------------------- + + +def test_step_timer_flushes_windows_with_engine_gpu_time(tmp_path): + """ADR-67 / BRG-6 (recorded-vs-actual): the bridge's timing sidecar + carries per-window step wall time, render wall time, sim seconds and the + engine's GPU time when reported, so a run's performance is comparable + across engines without a new topic (SPEC 010 untouched).""" + from pathlib import Path + + from aisle.nodes.dora_genesis import StepTimer, timing_sidecar_path + + path = tmp_path / "sim_timing.jsonl" + timer = StepTimer("nexus", path, flush_every=3, clock=lambda: 0.0) + timer.record_step(0.004, 10_000_000, {"gpu_ms": 3.0}, tick_s=0.010) + timer.record_render(0.002) + timer.record_step(0.006, 20_000_000, None, tick_s=0.011) + timer.record_step(0.008, 30_000_000, {"gpu_ms": 5.0}) + rows = [json.loads(line) for line in path.read_text().splitlines()] + assert len(rows) == 1 + row = rows[0] + assert row["engine"] == "nexus" and row["steps"] == 3 and row["frames"] == 1 + assert row["step_ms_mean"] == pytest.approx(6.0) and row["step_ms_max"] == pytest.approx(8.0) + # the third step gave no tick time, so its step time stands in for it + assert row["tick_ms_mean"] == pytest.approx((10.0 + 11.0 + 8.0) / 3) + assert row["render_ms_mean"] == pytest.approx(2.0) + assert row["gpu_ms_mean"] == pytest.approx(4.0) # over the steps that reported gpu time + assert row["sim_window_s"] == pytest.approx(0.02) and row["sim_time_s"] == pytest.approx(0.03) + # a window without renders or gpu reports leaves those null, never zero + timer.record_step(0.001, 40_000_000) + assert timer.flush(40_000_000)["render_ms_mean"] is None + assert timer.rows[-1]["gpu_ms_mean"] is None + # no results path (hand-launched graph): no sidecar + assert timing_sidecar_path({}) is None + assert timing_sidecar_path({"AISLE_RESULTS": "/runs/x/episodes.jsonl"}) == Path( + "/runs/x/sim_timing.jsonl" + ) + + +def test_debug_view_is_off_by_default_and_parses_presets(): + """ADR-67 operator tooling: AISLE_DEBUG_VIEW adds a camera that feeds + debug_view.mp4 only (no topic, SPEC 010 untouched): unset means none, + `side` looks at the shelf front from the tray side, an explicit + `eye;lookat` pair is honored, and a malformed value is refused.""" + from aisle.nodes.dora_genesis import parse_debug_view + from aisle.scenes.pharmacy import load_physics, resolve_layout + + layout = resolve_layout(load_physics(), "franka") + assert parse_debug_view({}, layout) is None + assert parse_debug_view({"AISLE_DEBUG_VIEW": ""}, layout) is None + eye, lookat = parse_debug_view({"AISLE_DEBUG_VIEW": "side"}, layout) + shelf_front_x = layout["shelf"]["pos"][0] - layout["shelf"]["level_size"][0] / 2 + assert eye[0] < shelf_front_x < lookat[0] # outside the shelf, looking into it + assert eye[1] < layout["tray"]["pos"][1] # from beyond the tray side + assert eye[2] > lookat[2] > 0.0 + assert parse_debug_view({"AISLE_DEBUG_VIEW": "1,-1,0.5;0.5,0,0.3"}, layout) == ( + [1.0, -1.0, 0.5], + [0.5, 0.0, 0.3], + ) + with pytest.raises(ValueError, match="AISLE_DEBUG_VIEW"): + parse_debug_view({"AISLE_DEBUG_VIEW": "sideways"}, layout) + with pytest.raises(ValueError, match="AISLE_DEBUG_VIEW"): + parse_debug_view({"AISLE_DEBUG_VIEW": "1,2;3,4"}, layout) diff --git a/tests/unit/test_docs_inventory.py b/tests/unit/test_docs_inventory.py index 67e5fb04..0d8400c6 100644 --- a/tests/unit/test_docs_inventory.py +++ b/tests/unit/test_docs_inventory.py @@ -340,6 +340,42 @@ def test_bridge_is_found_by_module_path_not_node_id(tmp_path: Path): assert rows["bridgeless.yaml"]["scene"] == "—" +def test_graph_table_names_the_physics_engine(tmp_path: Path): + """ADR-67, CON-5: a graph declaring `AISLE_SIM_ENGINE: nexus` runs + different physics, which is a different environment. Without the engine + column the generated inventory documented every graph as Genesis, and + the Nexus package did not exist anywhere in the layout map.""" + graphs = tmp_path / "graphs" + graphs.mkdir(parents=True) + (graphs / "nexus.yaml").write_text( + "nodes:\n" + " - id: sim-bridge\n" + " path: ../src/aisle/nodes/dora_genesis.py\n" + " env:\n" + " AISLE_SIM_ENGINE: nexus\n" + " outputs: [poses]\n", + encoding="utf-8", + ) + (graphs / "default.yaml").write_text( + "nodes:\n - id: sim-bridge\n path: ../src/aisle/nodes/dora_genesis.py\n" + " outputs: [poses]\n", + encoding="utf-8", + ) + (graphs / "bridgeless.yaml").write_text( + "nodes:\n - id: analysis-only\n outputs: [report]\n", encoding="utf-8" + ) + + rows = {row["path"].name: row for row in inventory_module()._graph_inventory(tmp_path, None)} + assert rows["nexus.yaml"]["engine"] == "nexus" + assert rows["default.yaml"]["engine"] == "genesis (default)" + # no bridge => no engine declaration to report + assert rows["bridgeless.yaml"]["engine"] == "—" + + text, _ = inventory_module().render_inventory(REPO_ROOT) + assert "| Engine |" in text + assert "sim engines" in text # the layout map names the package + + def test_cli_section_refuses_a_root_it_cannot_introspect(tmp_path: Path): """--root is honoured for graphs/manifests/ADRs/tests but the CLI table is built from the IMPORTED package, which --root cannot redirect. diff --git a/tests/unit/test_dynamic_typed_journal.py b/tests/unit/test_dynamic_typed_journal.py index a8589eba..34cbce7b 100644 --- a/tests/unit/test_dynamic_typed_journal.py +++ b/tests/unit/test_dynamic_typed_journal.py @@ -37,6 +37,11 @@ def test_ordinary_typed_child_retains_preparation_and_gate_refusal(tmp_path): dirs_exist_ok=True, ignore=shutil.ignore_patterns("__pycache__"), ) + # This fixture exercises the retained env-hash refusal. Give the rollout + # subprocess an importable engine sentinel so the earlier availability + # gate does not replace the refusal under test when the optional wheel is + # absent. Keep it out of the sealed interpreter runtime below. + (controller.root / "src/genesis.py").write_text("__version__ = 'test-sentinel'\n") packages = tmp_path / "runtime-packages" packages.mkdir() for name in ( diff --git a/tests/unit/test_engine_sources.py b/tests/unit/test_engine_sources.py new file mode 100644 index 00000000..980d3309 --- /dev/null +++ b/tests/unit/test_engine_sources.py @@ -0,0 +1,63 @@ +"""The engine sources are pinned by commit, not tracked by branch (ADR-67, +ADR-68, CON-5). + +A wheel built from "whatever is checked out" cannot be traced back to a +revision, which is the same reason `dora-runtime.json` pins the Dora CLI. The +pin file is validated rather than trusted: a prefix instead of a full commit, +or a missing repository, would silently widen what a run could have been +built from. +""" + +from __future__ import annotations + +import json +import sys + +import pytest +from cli_helpers import REPO_ROOT + +sys.path.insert(0, str(REPO_ROOT / "tools")) + +from engine_sources import NAMES, load_pins # noqa: E402 + +pytestmark = pytest.mark.unit + + +def test_committed_pins_name_a_full_commit_for_every_built_engine(): + """CON-5: the pins AISLE ships resolve to exact revisions.""" + pins = load_pins() + assert set(pins["sources"]) == set(NAMES) + for name in NAMES: + source = pins["sources"][name] + assert source["repository"].startswith("https://"), name + assert len(source["commit"]) == 40, name + assert int(source["commit"], 16) >= 0, name + assert source["branch"], name + + +def test_the_linked_crates_are_pinned_once_in_the_engine_manifest(): + """ADR-68: kiss3d and the rapier crates the engine links against are + resolved by cargo from nexus's own manifest (rapier from its crates.io + release, kiss3d by git revision). Pinning them here too would + be a second source of truth that drifts silently.""" + pins = load_pins() + assert "kiss3d" not in pins["sources"] + + +@pytest.mark.parametrize( + "mangle,expected", + [ + (lambda p: p["sources"].pop("rapier"), "miss"), + (lambda p: p["sources"]["nexus"].__setitem__("commit", "abc1234"), "full commit"), + (lambda p: p["sources"]["nexus"].__setitem__("repository", ""), "no repository"), + ], +) +def test_a_loose_pin_is_refused(tmp_path, mangle, expected): + """CON-8: the loader refuses rather than defaulting, so a half-specified + pin cannot reach a build.""" + pins = json.loads((REPO_ROOT / "engine-runtime.json").read_text()) + mangle(pins) + path = tmp_path / "engine-runtime.json" + path.write_text(json.dumps(pins)) + with pytest.raises(ValueError, match=expected): + load_pins(path) diff --git a/tests/unit/test_env_hash.py b/tests/unit/test_env_hash.py index 752ee739..1399a12b 100644 --- a/tests/unit/test_env_hash.py +++ b/tests/unit/test_env_hash.py @@ -524,6 +524,149 @@ def test_committed_hash_matches_this_tree(): assert computed["n_files"] == committed["n_files"] +def _sim_root(tmp_path: Path) -> Path: + """make_root plus the ADR-67 engine realization package.""" + root = make_root(tmp_path) + sim = root / "src" / "aisle" / "sim" + sim.mkdir(parents=True) + (sim / "__init__.py").write_text("ENGINES = ('genesis', 'nexus')\n") + (sim / "nexus_backend.py").write_text("def build_scene():\n return None\n") + (sim / "nexus_physics.toml").write_text("[sim]\nsubsteps = 2\n") + return root + + +def test_adr55_sim_engine_hash_covers_the_realization_the_fence_does_not(tmp_path): + """CON-5, CON-7, ADR-67: the frozen set is engine neutral by decision + (widening it would move every Genesis env_hash), so the engine + realization gets its OWN digest. Two trees differing only in + nexus_physics.toml's `substeps` share an env_hash — the exact hole the + audit found — and must differ in sim_engine_hash.""" + import sys as _sys + + _sys.path.insert(0, str(REPO_ROOT / "tools")) + from env_hash import compute_env_hash, sim_engine_hash + + root = _sim_root(tmp_path) + before_env, _ = compute_env_hash(root) + before = sim_engine_hash(root, "nexus") + assert before["engine"] == "nexus" and before["n_files"] == 3 + assert sim_engine_hash(root, "nexus") == before # deterministic + + (root / "src" / "aisle" / "sim" / "nexus_physics.toml").write_text("[sim]\nsubsteps = 8\n") + after_env, _ = compute_env_hash(root) + assert after_env == before_env, "the solver settings are outside the fence by decision" + assert sim_engine_hash(root, "nexus")["sim_engine_hash"] != before["sim_engine_hash"] + + +def test_adr55_sim_engine_hash_names_the_engine_and_its_build(tmp_path): + """CON-5, ADR-67: the digest binds the engine name and the engine build + provenance, so the same realization built from different Nexus sources + is a different environment identity — and a run manifest can carry the + receipt the gitignored file would otherwise keep local.""" + import sys as _sys + + _sys.path.insert(0, str(REPO_ROOT / "tools")) + from env_hash import sim_engine_hash + + root = _sim_root(tmp_path) + genesis = sim_engine_hash(root, "genesis") + nexus = sim_engine_hash(root, "nexus") + assert genesis["sim_engine_hash"] != nexus["sim_engine_hash"] + assert genesis["build"] is None + + receipt = {"version": "0.1.0", "sources": {"nexus": {"commit": "a" * 40, "dirty": False}}} + built = sim_engine_hash(root, "nexus", receipt) + assert built["build"] == receipt + assert built["sim_engine_hash"] != nexus["sim_engine_hash"] + moved = {"version": "0.1.0", "sources": {"nexus": {"commit": "b" * 40, "dirty": False}}} + assert sim_engine_hash(root, "nexus", moved)["sim_engine_hash"] != built["sim_engine_hash"] + + +def test_adr55_attested_set_names_the_running_engines_distribution(tmp_path): + """ADR-24 as amended by ADR-67: the attested sim core listed + genesis-world only, so a Nexus run's provenance check never looked at + the wheel whose solver produced the trajectories. The engine-neutral + core stays; the running engine's dist is added to it.""" + import sys as _sys + + _sys.path.insert(0, str(REPO_ROOT / "tools")) + from env_hash import attested_set + + assert "dimforge-nexus3d" not in attested_set(tmp_path) + assert "dimforge-nexus3d" not in attested_set(tmp_path, "genesis") + nexus = attested_set(tmp_path, "nexus") + assert "dimforge-nexus3d" in nexus + assert {"genesis-world", "torch"} <= set(nexus) # the neutral core is kept + + +def test_adr67_cli_refuses_an_out_of_lock_engine_without_its_only_provenance(tmp_path): + """CON-5, CON-8, ADR-67: Nexus and rapier live outside the lock, so their + build receipt is the only source provenance a run can retain. Missing + provenance must fail before a run can record an unidentified wheel.""" + root = _sim_root(tmp_path) + plain = json.loads(run_env_hash("--root", str(root)).stdout) + proc = run_env_hash("--root", str(root), "--sim-engine", "nexus") + assert proc.returncode == 1, proc.stdout + proc.stderr + report = json.loads(proc.stdout) + assert report["env_hash"] == plain["env_hash"] and "sim" not in plain + assert report["sim"]["engine"] == "nexus" + assert len(report["sim"]["sim_engine_hash"]) == 64 + assert report["sim"]["build"] is None + assert "build receipt" in report["error"] + + +def test_adr67_engine_receipt_gate_requires_clean_solver_and_renderer_sources(): + """CON-5, ADR-67/ADR-68: a receipt is provenance only when it identifies + clean source bytes; rapier also needs the Nexus renderer's receipt.""" + import sys as _sys + + _sys.path.insert(0, str(REPO_ROOT / "tools")) + from env_hash import engine_build_problem + + source = {"commit": "a" * 40, "dirty": False} + nexus = { + "schema_version": 1, + "wheel": "dimforge_nexus3d.whl", + "version": "0.1.0", + "platform": "test", + "sources": {"nexus": source}, + } + rapier = { + "schema_version": 1, + "wheel": "rapier3d.whl", + "version": "0.1.0", + "platform": "test", + "sources": {"rapier": source}, + "renderer": nexus, + } + assert engine_build_problem("genesis", None) is None + assert engine_build_problem("nexus", nexus) is None + assert engine_build_problem("rapier", rapier) is None + assert "missing" in engine_build_problem("nexus", None) + assert "renderer" in engine_build_problem("rapier", {**rapier, "renderer": None}) + dirty = {**nexus, "sources": {"nexus": {**source, "dirty": True}}} + assert "dirty" in engine_build_problem("nexus", dirty) + + +def test_adr67_cli_records_a_clean_out_of_lock_engine_receipt(tmp_path): + """CON-5, CON-8, ADR-67: a clean receipt passes and becomes part of the + recorded engine digest rather than merely serving as a presence flag.""" + root = _sim_root(tmp_path) + receipt = { + "schema_version": 1, + "wheel": "dimforge_nexus3d.whl", + "feature": "metal", + "version": "0.1.0", + "platform": "test", + "sources": {"nexus": {"commit": "a" * 40, "dirty": False}}, + } + (root / ".nexus-runtime-receipt.json").write_text(json.dumps(receipt)) + proc = run_env_hash("--root", str(root), "--sim-engine", "nexus") + assert proc.returncode == 0, proc.stdout + proc.stderr + report = json.loads(proc.stdout) + assert report["ok"] is True and report["sim"]["build"] == receipt + + def _frozen_set(): """The fence, read from the checker itself rather than restated.""" import sys diff --git a/tests/unit/test_freeze_registry.py b/tests/unit/test_freeze_registry.py index 7d22718c..6a9403b6 100644 --- a/tests/unit/test_freeze_registry.py +++ b/tests/unit/test_freeze_registry.py @@ -409,7 +409,7 @@ def test_committed_registrations_check_clean_with_withheld_seeds(): registration names it in `superseded`; drift with no successor is the refusal the registry promises (analysis/freeze/README.md).""" manifests = _committed_manifests() - assert len(manifests) == 65 + assert len(manifests) == 69 superseded_ids: set[str] = set() for path in manifests: declaration = json.loads(path.with_name("declaration.json").read_text()) @@ -451,7 +451,7 @@ def test_confirmatory_registrations_carry_a_refused_freeze(): assert "independent statistical review" in refusal["errors"][0] -@pytest.mark.parametrize("previous_version,current_version", [(6, 7), (7, 8)]) +@pytest.mark.parametrize("previous_version,current_version", [(6, 7), (7, 8), (8, 9), (9, 10)]) def test_hardened_perception_registration_requires_a_new_audit(previous_version, current_version): """BND-5/BND-12: a changed auditor cannot inherit a passed historical audit.""" root = REPO_ROOT / "analysis/freeze" @@ -470,8 +470,8 @@ def test_hardened_perception_registration_requires_a_new_audit(previous_version, def test_shared_cli_successor_preserves_bnd_protocol_and_review_gates(): """BND-12/BND-13: shared CLI changes need a successor without changing calibration rules.""" root = REPO_ROOT / "analysis/freeze" - previous = json.loads((root / "bnd-task-band-calibration-v12/freeze-manifest.json").read_text()) - current = json.loads((root / "bnd-task-band-calibration-v13/freeze-manifest.json").read_text()) + previous = json.loads((root / "bnd-task-band-calibration-v14/freeze-manifest.json").read_text()) + current = json.loads((root / "bnd-task-band-calibration-v15/freeze-manifest.json").read_text()) assert current["seed_commitment"] == previous["seed_commitment"] for key in ( "analysis", diff --git a/tests/unit/test_harness_sim_engine.py b/tests/unit/test_harness_sim_engine.py new file mode 100644 index 00000000..94edc990 --- /dev/null +++ b/tests/unit/test_harness_sim_engine.py @@ -0,0 +1,618 @@ +"""Harness-side engine selection (ADR-67). + +The bridge learned `AISLE_SIM_ENGINE` in ADR-67; these tests cover the +harness half: the runner owns the choice (never the operator's shell), the +graph declares it where the graph hash attests it, every sim-launching entry +point can select it, and the attested identity and budgets follow the engine +actually resolved rather than a hardcoded Genesis default. + +Unit-marked: nothing here imports a simulator. +""" + +from pathlib import Path + +import pytest +import yaml + +pytestmark = pytest.mark.unit + +REPO_ROOT = Path(__file__).resolve().parents[2] +BRIDGE_MANIFESTS = {"dora-genesis": {"provides": ["sim_bridge"]}} + + +def _graph_with_bridge_env(tmp_path: Path, env: dict, name: str = "engine.yaml") -> Path: + """The real T0 graph with extra bridge env, node paths absolutized.""" + doc = yaml.safe_load((REPO_ROOT / "graphs" / "expert_t0.yaml").read_text()) + for node in doc["nodes"]: + node["path"] = str((REPO_ROOT / "graphs" / node["path"]).resolve()) + if node["id"] == "dora-genesis": + node["env"] = {**(node.get("env") or {}), **env} + path = tmp_path / name + path.write_text(yaml.safe_dump(doc, sort_keys=False)) + return path + + +# -- 1. the engine never leaks in from the ambient environment ------------ + + +def test_engine_and_backend_are_scrubbed_from_the_ambient_environment(): + """CON-5, ADR-67: `harness fleet` builds its child env from + scrub_bringup_env(os.environ), so an ambient AISLE_SIM_ENGINE=nexus in an + operator's shell would silently swap the physics of a run whose git_sha, + env_hash and graph_hash all attest clean. Both the engine and its backend + are graph- or runner-owned, exactly like AISLE_PERCEPTION.""" + from aisle.harness.rollout import SCRUBBED_ENV, scrub_bringup_env + + assert "AISLE_SIM_ENGINE" in SCRUBBED_ENV + assert "AISLE_SIM_BACKEND" in SCRUBBED_ENV + kept = scrub_bringup_env( + {"AISLE_SIM_ENGINE": "nexus", "AISLE_SIM_BACKEND": "webgpu", "AISLE_SEEDS": "0,1"} + ) + assert kept == {"AISLE_SEEDS": "0,1"} + + +# -- 2. the graph declares the engine; the flag asserts it ---------------- + + +def test_engine_check_takes_the_request_when_the_graph_declares_nothing(tmp_path): + """ADR-67: an undeclared graph runs what the caller asked for, and a + caller who asks for nothing gets the default engine.""" + from aisle.harness.rollout import engine_check + + graph = _graph_with_bridge_env(tmp_path, {}) + assert engine_check(REPO_ROOT, graph, None) == { + "ok": True, + "engine": "genesis", + "declared": None, + } + assert engine_check(REPO_ROOT, graph, "nexus")["engine"] == "nexus" + + +def test_engine_check_honours_a_graph_declaration_the_cli_does_not_contradict(tmp_path): + """ADR-67: the engine is declared on the bridge node like the perception + rung, so a graph that names one is NOT silently overwritten with genesis + when the runner is invoked without --sim-engine.""" + from aisle.harness.rollout import engine_check + + graph = _graph_with_bridge_env(tmp_path, {"AISLE_SIM_ENGINE": "nexus"}) + assert engine_check(REPO_ROOT, graph, None) == { + "ok": True, + "engine": "nexus", + "declared": "nexus", + } + assert engine_check(REPO_ROOT, graph, "nexus")["engine"] == "nexus" + + +def test_engine_check_refuses_a_conflicting_request(tmp_path): + """ADR-67, HAR-2: mirroring TC-9's perception assertion, a graph + declaring one engine and a flag demanding another is refused at the + `sim_engine` gate — overwriting would measure physics the graph hash does + not attest.""" + from aisle.harness.rollout import engine_check + + graph = _graph_with_bridge_env(tmp_path, {"AISLE_SIM_ENGINE": "nexus"}) + refusal = engine_check(REPO_ROOT, graph, "genesis") + assert refusal["ok"] is False and refusal["gate"] == "sim_engine" + assert "nexus" in refusal["detail"] and "genesis" in refusal["detail"] + assert "AISLE_SIM_ENGINE" in refusal["hint"] + + +def test_engine_check_refuses_an_unknown_declaration(tmp_path): + """ADR-67 (TC-9's refuse-don't-guess rule): a typo must fail the gate + rather than fall back to the default engine.""" + from aisle.harness.rollout import engine_check + + graph = _graph_with_bridge_env(tmp_path, {"AISLE_SIM_ENGINE": "bullet"}) + refusal = engine_check(REPO_ROOT, graph, None) + assert refusal["ok"] is False and refusal["gate"] == "sim_engine" + assert "unknown simulation engine" in refusal["detail"] + blank_graph = _graph_with_bridge_env(tmp_path, {"AISLE_SIM_ENGINE": " "}, "blank.yaml") + blank = engine_check(REPO_ROOT, blank_graph, None) + assert blank["ok"] is False + + +def test_instrumented_graph_refuses_rather_than_overwriting_a_declaration(tmp_path): + """ADR-67: the last writer obeys the same rule as the gate. Before this, + instrumentation stamped the requested engine over the bridge's own + declaration, so a nexus graph ran on genesis without a word.""" + from aisle.harness.rollout import EngineConflict, instrumented_graph + + graph = _graph_with_bridge_env(tmp_path, {"AISLE_SIM_ENGINE": "nexus"}) + run_dir = tmp_path / "run" + run_dir.mkdir() + with pytest.raises(EngineConflict, match="nexus"): + instrumented_graph(graph, REPO_ROOT, run_dir, sim_engine="genesis") + out = instrumented_graph(graph, REPO_ROOT, run_dir, sim_engine="nexus") + doc = yaml.safe_load(out.read_text()) + bridge = next(n for n in doc["nodes"] if n["id"] == "dora-genesis") + assert bridge["env"]["AISLE_SIM_ENGINE"] == "nexus" + + +# -- 3. attested identity follows the resolved (engine, backend) pair ----- + + +@pytest.mark.parametrize( + ("engine", "backend", "device"), + [ + ("genesis", "cpu", "cpu"), + ("genesis", "metal", "mps"), + ("nexus", "cpu", "cpu"), + ("nexus", "metal", "metal"), + ("nexus", "webgpu", "webgpu"), + ], +) +def test_sim_device_follows_the_engine_and_backend(engine, backend, device): + """CON-5: the attested device is derived from the resolved pair, not from + the host OS. A Nexus webgpu run on Linux used to be attested `cpu`, which + made GPU and CPU physics evidence indistinguishable in the manifest.""" + from aisle.harness.rollout import sim_device_for + + assert sim_device_for(engine, backend) == device + + +def test_resolve_sim_identity_refuses_an_engine_that_is_not_installed(monkeypatch): + """CON-8, ADR-67: an engine whose package is missing refuses at the gate, + before anything launches, instead of dying inside the bridge.""" + from aisle.harness import rollout as rollout_module + + monkeypatch.setattr("aisle.sim.engine_available", lambda engine: engine != "rapier") + refused = rollout_module.resolve_sim_identity("sim", "rapier") + assert refused["ok"] is False and refused["gate"] == "sim_engine" + assert "'rapier' is not installed" in refused["detail"] + assert rollout_module.resolve_sim_identity("sim", "genesis")["ok"] is True + + +def test_resolve_sim_identity_attests_the_nexus_gpu_device_on_linux(monkeypatch): + """CON-5, ADR-67: the same rule end-to-end through the gate's resolver.""" + from aisle.harness import rollout as rollout_module + + monkeypatch.setattr(rollout_module.platform_module, "system", lambda: "Linux") + monkeypatch.setattr("aisle.sim.engine_available", lambda engine: True) + identity = rollout_module.resolve_sim_identity("sim", "nexus") + assert identity["ok"] is True + assert (identity["sim_backend"], identity["sim_device"]) == ("webgpu", "webgpu") + genesis = rollout_module.resolve_sim_identity("sim", "genesis") + assert (genesis["sim_backend"], genesis["sim_device"]) == ("cpu", "cpu") + + +# -- 4. budgets are engine-derived ---------------------------------------- + + +def test_build_grace_and_pre_data_stall_are_engine_derived(): + """HAR-1, ADR-67: the 420 s build grace and the 600 s pre-data stall were + both justified by the minutes-long Genesis build. Nexus builds a scene in + seconds, so a wedged Nexus launch must clamp far sooner.""" + from aisle.harness.rollout import ( + GENESIS_BUILD_BUDGET_S, + PRE_DATA_STALL_S, + build_budget_s, + pre_data_stall_s, + ) + + assert build_budget_s("genesis") == GENESIS_BUILD_BUDGET_S + assert pre_data_stall_s("genesis") == PRE_DATA_STALL_S + assert build_budget_s("nexus") < GENESIS_BUILD_BUDGET_S + assert pre_data_stall_s("nexus") < PRE_DATA_STALL_S + assert build_budget_s("nexus") > 0 and pre_data_stall_s("nexus") > 0 + + +# -- 5. every sim-launching entry point can select the engine ------------- + + +def _sim_engine_action(subparser_path: list[str]): + """The --sim-engine action of a subcommand, found by walking argparse.""" + from aisle.harness.cli import build_parser + + parser = build_parser() + for name in subparser_path: + choices = next( + action.choices + for action in parser._actions + if getattr(action, "choices", None) and isinstance(action.choices, dict) + ) + parser = choices[name] + return next( + (action for action in parser._actions if action.option_strings == ["--sim-engine"]), None + ) + + +@pytest.mark.parametrize( + "command", + [ + ["rollout"], + ["monolith", "run"], + ["fault", "calibrate"], + ["fleet"], + ["skill", "register"], + ], +) +def test_every_sim_launching_entry_point_shares_one_engine_flag(command): + """ADR-67, CON-8: `harness rollout` was the only path that could select + an engine, so every other launcher forced genesis. They now share one + flag definition, which is also why no path can drift to a different + engine set.""" + from aisle.sim import ENGINES + + action = _sim_engine_action(command) + assert action is not None, f"{' '.join(command)} cannot select an engine" + assert tuple(action.choices) == ENGINES + assert action.default is None # no assertion: the graph's declaration wins + + +def test_monolith_run_forwards_the_engine_to_the_runner(tmp_path, monkeypatch): + """MON-3, ADR-67: the monolithic launcher is a sim-launching entry point; + it must not pin its module to Genesis.""" + from aisle.harness import monolith as mono + from aisle.harness import rollout + + observed = {} + monkeypatch.setattr(mono, "check_module", lambda *args: {"ok": True}) + monkeypatch.setattr(mono, "stamp_graph", lambda *args: tmp_path / "stamped.yaml") + monkeypatch.setattr( + rollout, "rollout", lambda **kwargs: observed.update(kwargs) or {"ok": True} + ) + mono.run(tmp_path, tmp_path / "m.py", [0], 1, run_id="engine-test", sim_engine="nexus") + assert observed["sim_engine"] == "nexus" + + +def test_skill_registration_forwards_the_engine_to_the_eval_rollout(monkeypatch): + """CAP-6, ADR-67: a skill's evalcard records a measured pass rate, so the + eval must run on the engine the operator selected.""" + from aisle.harness import skill as skill_module + + observed = {} + + def fake_rollout(**kwargs): + observed.update(kwargs) + return {"ok": True, "pass1": 1.0} + + fake_skill = type( + "S", + (), + { + "eval_cfg": { + "tier": "T0", + "episodes": 1, + "seeds": "0", + "embodiment": "franka", + "suite": "s", + }, + "path": Path("."), + "manifest": {"id": "x"}, + }, + )() + monkeypatch.setattr(skill_module, "_eval_graph_path", lambda skill, root: Path("g.yaml")) + rate = skill_module.run_skill_eval(fake_skill, Path("."), fake_rollout, "rid", "nexus") + assert rate == 1.0 + assert observed["sim_engine"] == "nexus" + + +def test_fault_calibration_rung_passes_the_engine_to_the_rollout_argv(tmp_path, monkeypatch): + """FLT-9, ADR-67: a calibration ladder measures severities against a + clean baseline, so every rung must run the campaign's engine rather than + the rollout default.""" + from aisle.harness import fault_calibration as fc + + calls = [] + + class Completed: + returncode = 0 + stdout = '{"ok": true}' + stderr = "" + + def fake_run(cmd, **kwargs): + calls.append(cmd) + return Completed() + + monkeypatch.setattr(fc.subprocess, "run", fake_run) + monkeypatch.setattr(fc, "materialize", lambda *args, **kwargs: {"receipt": "r"}) + outcome = fc.run_rung( + root=tmp_path, + worktree=tmp_path / "wt", + clean_commit="abc", + instance={"opaque_id": "i"}, + severity_index=0, + clean_hash="h", + graph_rel="graphs/expert_t1.yaml", + seeds="0..1", + tier="T1", + embodiment="franka", + perception="L1", + run_id="rid", + sim_engine="nexus", + ) + assert outcome["result"]["ok"] is True + rollout_argv = next(cmd for cmd in calls if "rollout" in cmd) + assert rollout_argv[rollout_argv.index("--sim-engine") + 1] == "nexus" + # no engine selected leaves the graph's own declaration in charge + calls.clear() + fc.run_rung( + root=tmp_path, + worktree=tmp_path / "wt2", + clean_commit="abc", + instance={"opaque_id": "i"}, + severity_index=0, + clean_hash="h", + graph_rel="graphs/expert_t1.yaml", + seeds="0..1", + tier="T1", + embodiment="franka", + perception="L1", + run_id="rid2", + ) + assert "--sim-engine" not in next(cmd for cmd in calls if "rollout" in cmd) + + +@pytest.mark.parametrize( + "declared,requested,installed,expected", + [ + ("nexus", "genesis", True, "graph declares engine nexus"), + (None, "rapier", False, "'rapier' is not installed"), + ], +) +def test_fleet_refuses_before_launch( + tmp_path, monkeypatch, capsys, declared, requested, installed, expected +): + """ADR-67, CON-8: fleet runs the same engine gate as rollout. A bridge's + own AISLE_SIM_ENGINE beats the child env dora is given, so a conflicting + flag would run one engine and report another; a missing engine would die + inside the bridge. Both refuse before anything launches.""" + import json + import subprocess + import sys + + from aisle.harness import cli + + env = {} if declared is None else {"AISLE_SIM_ENGINE": declared} + graph = _graph_with_bridge_env(tmp_path, env) + monkeypatch.setattr("aisle.sim.engine_available", lambda engine: installed) + monkeypatch.setattr(subprocess, "Popen", lambda *a, **k: pytest.fail("fleet launched")) + monkeypatch.setattr( + sys, + "argv", + ["harness", "fleet", "--graph", str(graph), "--agents", "2", "--episodes", "1"] + + ["--seeds", "0", "--root", str(REPO_ROOT), "--out", str(tmp_path / "out")] + + ["--sim-engine", requested], + ) + assert cli.main() == 1 + report = json.loads(capsys.readouterr().out) + assert report["ok"] is False and report["refused"]["gate"] == "sim_engine" + assert expected in report["refused"]["detail"] + assert not (tmp_path / "out").exists() + + +def test_fleet_report_records_the_engine_it_ran(tmp_path): + """CON-5, ADR-67: `harness fleet` sets the engine in the child env after + the scrub, so its report is where the run says which physics it measured + — otherwise a fleet run carries no engine evidence at all.""" + from aisle.harness.fleet import run_fleet + + graph = REPO_ROOT / "graphs" / "expert_t0.yaml" + out_dir = tmp_path / "fleet" + + def launch(graph_path): + for agent in range(2): + (out_dir / f"results_a{agent}.jsonl").write_text( + '{"episode": 0, "seed": 0, "status": "success"}\n' + ) + return lambda: 0 + + report = run_fleet( + graph, + 2, + 1, + [0], + out_dir, + 5.0, + launch, + root=REPO_ROOT, + sim_engine="nexus", + sim_backend="webgpu", + sim_device="webgpu", + ) + assert report["sim_engine"] == "nexus" + assert report["sim_backend"] == "webgpu" + assert report["sim_device"] == "webgpu" + + +# -- 6. the whole run: what is injected, and what is attested ------------- + + +def _stub_run(tmp_path, monkeypatch, engine="genesis"): + """A fake root whose dora launch is replaced by a results writer; returns + (root, graph, spawned-env dict).""" + import json + + from test_idea_gate import _fake_root + + from aisle.harness import rollout as rollout_module + + root = _fake_root(tmp_path) + spawned: dict = {} + + def fake_spawn(exec_graph, run_dir, env, relaunch=0): + spawned.update(env) + results = Path(env["AISLE_RESULTS"]) + seeds = [int(s) for s in env["AISLE_SEEDS"].split(",")] + with open(results, "w") as handle: + for index, seed in enumerate(seeds): + handle.write( + json.dumps( + {"episode": index, "seed": seed, "status": "success", "failure": None} + ) + + "\n" + ) + + class FakeProc: + """Exited: the runner then stops waiting as soon as the seeded + rows are on disk, which keeps this a unit test.""" + + pid = 0 + + def poll(self): + return 0 + + return FakeProc() + + monkeypatch.setattr(rollout_module, "_spawn_dora", fake_spawn) + monkeypatch.setattr(rollout_module, "_terminate", lambda proc: None) + monkeypatch.setattr( + rollout_module, + "resolve_sim_identity", + lambda extra, requested="genesis": { + "ok": True, + "sim_extra": extra, + "sim_engine": engine, + "sim_backend": "webgpu" if engine == "nexus" else "cpu", + "sim_device": "webgpu" if engine == "nexus" else "cpu", + }, + ) + return root, root / "graphs" / "expert_t0.yaml", spawned + + +def test_an_ambient_engine_never_reaches_the_launched_dataflow(tmp_path, monkeypatch): + """CON-5, ADR-67: the leak this scrub closes. With AISLE_SIM_ENGINE=nexus + exported in the shell, the run must still launch — and attest — the + engine the gate resolved.""" + import json + + monkeypatch.setenv("AISLE_SIM_ENGINE", "nexus") + monkeypatch.setenv("AISLE_SIM_BACKEND", "webgpu") + root, graph, spawned = _stub_run(tmp_path, monkeypatch) + from aisle.harness import rollout as rollout_module + + report = rollout_module.rollout( + root=root, + graph=graph, + tier="T0", + episodes=1, + seeds=[0], + reset_mode="teleport", + verifier="oracle", + run_id="ambient-engine", + branch="b", + no_idea_gate=True, + env_baseline="local", + ) + assert report["ok"] is True, report + assert spawned["AISLE_SIM_ENGINE"] == "genesis" + assert spawned["AISLE_SIM_BACKEND"] == "cpu" + manifest = json.loads((root / "runs" / "ambient-engine" / "manifest.json").read_text()) + assert manifest["sim_engine"] == "genesis" + + +def test_the_run_attests_the_resolved_engine_and_its_build_grace(tmp_path, monkeypatch): + """HAR-1, HAR-4, ADR-67: the manifest records the engine the gate + resolved (a lost key must fail, never attest genesis), and the launch + clamps against that engine's build grace.""" + import json + + from aisle.harness import rollout as rollout_module + from aisle.harness.rollout import NEXUS_BUILD_BUDGET_S + + root, graph, _ = _stub_run(tmp_path, monkeypatch, engine="nexus") + report = rollout_module.rollout( + root=root, + graph=graph, + tier="T0", + episodes=1, + seeds=[0], + reset_mode="teleport", + verifier="oracle", + run_id="nexus-grace", + branch="b", + no_idea_gate=True, + env_baseline="local", + ) + assert report["ok"] is True + manifest = json.loads((root / "runs" / "nexus-grace" / "manifest.json").read_text()) + assert manifest["sim_engine"] == "nexus" + assert manifest["sim_backend"] == "webgpu" and manifest["sim_device"] == "webgpu" + assert manifest["build_grace_s"] == NEXUS_BUILD_BUDGET_S + + +def test_a_gate_without_an_engine_fails_instead_of_attesting_genesis(tmp_path, monkeypatch): + """CON-5, HAR-4: `gates.get("sim_engine", "genesis")` turned a lost key + into a genesis attestation on a run that may have been anything. The key + is now required, so the failure is loud.""" + from aisle.harness import rollout as rollout_module + + root, graph, _ = _stub_run(tmp_path, monkeypatch, engine="nexus") + real_run_gates = rollout_module.run_gates + + def gates_without_engine(*args, **kwargs): + result = real_run_gates(*args, **kwargs) + return {k: v for k, v in result.items() if k != "sim_engine"} + + monkeypatch.setattr(rollout_module, "run_gates", gates_without_engine) + with pytest.raises(KeyError, match="sim_engine"): + rollout_module.rollout( + root=root, + graph=graph, + tier="T0", + episodes=1, + seeds=[0], + reset_mode="teleport", + verifier="oracle", + run_id="no-engine", + branch="b", + no_idea_gate=True, + env_baseline="local", + ) + + +def test_a_rollout_refuses_a_graph_declaring_another_engine(tmp_path, monkeypatch): + """ADR-67, HAR-2: the refusal reaches the CON-8 report as its own gate, + before any budget reservation or launch.""" + from aisle.harness import rollout as rollout_module + + root, graph, spawned = _stub_run(tmp_path, monkeypatch) + doc = yaml.safe_load(graph.read_text()) + bridge = next(node for node in doc["nodes"] if node["id"] == "dora-genesis") + bridge["env"] = {**(bridge.get("env") or {}), "AISLE_SIM_ENGINE": "nexus"} + graph.write_text(yaml.safe_dump(doc, sort_keys=False)) + report = rollout_module.rollout( + root=root, + graph=graph, + tier="T0", + episodes=1, + seeds=[0], + reset_mode="teleport", + verifier="oracle", + run_id="conflict", + branch="b", + no_idea_gate=True, + env_baseline="local", + sim_engine="genesis", + ) + assert report["ok"] is False + assert report["refused"]["gate"] == "sim_engine" + assert spawned == {} # refused before launch + + +# -- 7. validation knows the variable exists ------------------------------ + + +def test_validator_refuses_an_unknown_engine_on_the_bridge(): + """VAL-8's companion for ADR-67: the engine rides the bridge's env like + the rung, so a typo must be a validation error rather than a launch-time + surprise hours later.""" + from aisle.harness.validate import graph_sim_engine_errors, validate_nodes + + nodes = [{"id": "dora-genesis", "env": {"AISLE_SIM_ENGINE": "bullet"}}] + errors = graph_sim_engine_errors(nodes, ["dora-genesis"]) + assert [e["code"] for e in errors] == ["SIM_ENGINE_UNKNOWN"] + assert "bullet" in errors[0]["detail"] and "nexus" in errors[0]["hint"] + reported, _ = validate_nodes(nodes, BRIDGE_MANIFESTS, set(), "franka", allow_unproven=True) + assert "SIM_ENGINE_UNKNOWN" in [e["code"] for e in reported] + + +def test_validator_accepts_known_engines_and_an_absent_declaration(): + """ADR-67: every pre-ADR-67 graph declares nothing and MUST keep + validating unchanged; a known engine is not an error either.""" + from aisle.harness.validate import graph_sim_engine_errors + + assert graph_sim_engine_errors([{"id": "dora-genesis"}], ["dora-genesis"]) == [] + for engine in ("nexus", " Genesis "): + nodes = [{"id": "dora-genesis", "env": {"AISLE_SIM_ENGINE": engine}}] + assert graph_sim_engine_errors(nodes, ["dora-genesis"]) == [] diff --git a/tests/unit/test_idea_gate.py b/tests/unit/test_idea_gate.py index bc67e5b6..20e0427a 100644 --- a/tests/unit/test_idea_gate.py +++ b/tests/unit/test_idea_gate.py @@ -17,6 +17,14 @@ REPO_ROOT = Path(__file__).resolve().parents[2] +@pytest.fixture(autouse=True) +def engines_installed(monkeypatch): + """CI's unit tier syncs without the sim extra, so no engine is importable + and the gate would refuse on availability (ADR-67); these tests are about + the other gates. Engine refusal has its own tests.""" + monkeypatch.setattr("aisle.sim.engine_available", lambda engine: True) + + def test_rollout_cli_exposes_the_attested_simulation_extra(monkeypatch): """HAR-1, CON-5: the public rollout path exposes the exact dependency selection; CUDA cannot be activated by an ambient hardware probe.""" @@ -141,9 +149,10 @@ def test_gate_attests_the_selected_sim_extra_and_backend(tmp_path, monkeypatch): monkeypatch.setattr( rollout_module, "resolve_sim_identity", - lambda extra: { + lambda extra, engine="genesis": { "ok": True, "sim_extra": extra, + "sim_engine": engine, "sim_backend": "cuda", "sim_device": "NVIDIA Test GPU", }, @@ -177,7 +186,7 @@ def test_gate_fails_closed_when_requested_backend_is_unavailable(tmp_path, monke monkeypatch.setattr( rollout_module, "resolve_sim_identity", - lambda extra: { + lambda extra, engine="genesis": { "ok": False, "gate": "sim_backend", "detail": "CUDA device unavailable", diff --git a/tests/unit/test_nexus_grasp_replay.py b/tests/unit/test_nexus_grasp_replay.py new file mode 100644 index 00000000..7fa06c3d --- /dev/null +++ b/tests/unit/test_nexus_grasp_replay.py @@ -0,0 +1,34 @@ +"""The grasp replay probe keeps the CLI contract on a run with nothing to +replay (CON-8): JSON on stdout and a nonzero exit, not a traceback.""" + +from __future__ import annotations + +import json +import sys + +import pyarrow as pa +import pytest +from cli_helpers import REPO_ROOT + +sys.path.insert(0, str(REPO_ROOT / "tools")) + +import nexus_grasp_replay # noqa: E402 + +pytestmark = pytest.mark.unit + + +def _empty_trace(path): + schema = pa.schema([("sim_time_ns", pa.int64()), ("data", pa.list_(pa.float32()))]) + with pa.OSFile(str(path), "wb") as sink, pa.ipc.new_stream(sink, schema): + pass + + +def test_an_empty_trace_is_a_json_refusal(tmp_path, capsys): + """CON-8: an empty command trace refuses with JSON before any engine loads.""" + traces = tmp_path / "traces" + traces.mkdir() + for name in ("budget-guard__joint_cmd_safe", "budget-guard__gripper_cmd_safe"): + _empty_trace(traces / f"{name}.arrow") + assert nexus_grasp_replay.main(["--run", str(tmp_path)]) == 1 + report = json.loads(capsys.readouterr().out) + assert report["ok"] is False and "no joint or gripper commands" in report["error"] diff --git a/tests/unit/test_nexus_runtime.py b/tests/unit/test_nexus_runtime.py new file mode 100644 index 00000000..13d90bd6 --- /dev/null +++ b/tests/unit/test_nexus_runtime.py @@ -0,0 +1,85 @@ +"""Unit tests for tools/nexus_runtime.py's build-provenance reader (ADR-67). + +The Nexus wheel is built from sibling checkouts and installed outside the uv +lock, so `uv.lock` says nothing about it and the receipt is gitignored: the +only way a run stays traceable to the engine sources that produced it is for +the rollout manifest to record the receipt. These tests pin the reader the +manifest calls, and its CON-8 CLI contract. +""" + +import json +import sys +from pathlib import Path + +import pytest +from cli_helpers import REPO_ROOT, run_tool + +pytestmark = pytest.mark.unit + +RECEIPT_NAME = ".nexus-runtime-receipt.json" + + +def runtime_module(): + sys.path.insert(0, str(REPO_ROOT / "tools")) + import nexus_runtime + + return nexus_runtime + + +def write_receipt(root: Path, payload: dict) -> Path: + path = root / RECEIPT_NAME + path.write_text(json.dumps(payload)) + return path + + +def test_read_receipt_returns_the_engine_build_provenance(tmp_path): + """CON-5, ADR-67: a run manifest can carry the wheel, feature, platform + and the nexus/rapier/kiss3d commits behind the physics it measured.""" + payload = { + "schema_version": 1, + "wheel": "dimforge_nexus3d-0.1.0-cp39-abi3-macosx_11_0_arm64.whl", + "feature": "metal", + "version": "0.1.0", + "platform": "darwin-arm64", + "sources": {"nexus": {"commit": "a" * 40, "branch": "aisle-backend", "dirty": True}}, + } + path = write_receipt(tmp_path, payload) + + result = runtime_module().read_receipt(tmp_path) + + assert result == { + "installed": True, + "path": str(path), + "receipt": payload, + "problem": None, + } + + +def test_read_receipt_reports_absence_and_corruption_as_facts(tmp_path): + """CON-8, ADR-67: an engine that was never built, or a receipt that no + longer parses, is a reported problem the manifest can record — never an + exception escaping into the rollout.""" + module = runtime_module() + + missing = module.read_receipt(tmp_path) + assert missing["installed"] is False and missing["receipt"] is None + assert "no nexus build receipt" in missing["problem"] + + (tmp_path / RECEIPT_NAME).write_text("{not json") + broken = module.read_receipt(tmp_path) + assert broken["installed"] is False and broken["receipt"] is None + assert "unreadable" in broken["problem"] + + (tmp_path / RECEIPT_NAME).write_text('["a list"]') + wrong_shape = module.read_receipt(tmp_path) + assert wrong_shape["installed"] is False and "malformed" in wrong_shape["problem"] + + +def test_receipt_cli_emits_json_and_exits_on_the_verdict(): + """CON-8: JSON on stdout, logs on stderr, exit 0 iff a usable receipt + exists — the same contract every AISLE CLI holds to.""" + proc = run_tool("nexus_runtime.py", "receipt") + report = json.loads(proc.stdout) + assert report["ok"] is report["installed"] + assert proc.returncode == (0 if report["installed"] else 1) + assert report["path"].endswith(RECEIPT_NAME) diff --git a/tests/unit/test_rollout_engine_attestation.py b/tests/unit/test_rollout_engine_attestation.py new file mode 100644 index 00000000..f6784b68 --- /dev/null +++ b/tests/unit/test_rollout_engine_attestation.py @@ -0,0 +1,99 @@ +"""The run records which engine build produced it (ADR-67, CON-5). + +`env_hash` is engine neutral by construction: the frozen set does not contain +`src/aisle/sim`, so two runs on different engines, or on the same engine with +different solver settings, hash identically. The gate therefore asks the +trusted checker for the engine digest as well and carries it into the manifest. +""" + +from __future__ import annotations + +import json +import subprocess +from pathlib import Path + +import pytest + +from aisle.harness import rollout as rollout_module + +pytestmark = pytest.mark.unit + +ENGINE_FACTS = { + "engine": "nexus", + "sim_engine_hash": "b" * 64, + "n_files": 3, + "build": {"version": "0.1.0", "sources": {"nexus": {"commit": "c" * 40}}}, +} + + +@pytest.fixture +def captured_hash_cmd(monkeypatch): + """Answer the trusted env_hash checker with a canned report, keeping the + argv it was called with. Every other subprocess call is refused so the + test cannot silently exercise something else. The engine wheels are + outside the lock, so the unit tier has none installed (ADR-67).""" + monkeypatch.setattr("aisle.sim.engine_available", lambda engine: True) + seen: dict = {} + real_run = subprocess.run + + def fake_run(cmd, *args, **kwargs): + if isinstance(cmd, list) and cmd and str(cmd[-1]).endswith("env_hash.py"): + raise AssertionError(f"unexpected env_hash invocation shape: {cmd}") + if isinstance(cmd, list) and any(str(part).endswith("env_hash.py") for part in cmd): + seen["cmd"] = [str(part) for part in cmd] + return subprocess.CompletedProcess( + cmd, + 0, + stdout=json.dumps({"ok": True, "env_hash": "a" * 64, "sim": ENGINE_FACTS}), + stderr="", + ) + return real_run(cmd, *args, **kwargs) + + monkeypatch.setattr(rollout_module.subprocess, "run", fake_run) + return seen + + +def _gates(root: Path, engine: str) -> dict: + return rollout_module.run_gates( + root=root, + graph=root / "graphs" / "expert_t0.yaml", + branch="feat/test", + no_idea_gate=True, + env_baseline="local", + sim_engine=engine, + ) + + +def test_gate_asks_the_checker_for_the_engine_digest(captured_hash_cmd): + """ADR-67: the engine name reaches the trusted checker, so the digest it + reports is the one for the engine this run will actually launch. Without + it the checker would report the default engine's digest for a Nexus run, + which is the recorded-vs-actual divergence the digest exists to close.""" + root = Path(__file__).resolve().parents[2] + gates = _gates(root, "nexus") + assert gates["ok"] is True, gates + cmd = captured_hash_cmd["cmd"] + assert "--sim-engine" in cmd + assert cmd[cmd.index("--sim-engine") + 1] == "nexus" + + +def test_gate_carries_the_engine_build_into_the_manifest_facts(captured_hash_cmd): + """CON-5/ADR-67: the digest and the out-of-lock engine's build receipt are + facts of the run, recorded verbatim. The receipt is the only trace of which + engine sources produced a result: the wheel is not in the lock, so nothing + else in the manifest can name them.""" + root = Path(__file__).resolve().parents[2] + gates = _gates(root, "nexus") + assert gates["sim_engine_build"] == ENGINE_FACTS + assert gates["sim_engine_build"]["sim_engine_hash"] != gates["env_hash"] + + +def test_gate_refuses_an_installed_out_of_lock_engine_without_a_receipt(monkeypatch): + """CON-5/ADR-67: importability does not identify an out-of-lock wheel. + The trusted checker must stop the run when its only source receipt is + absent, before validation or launch can turn it into evidence.""" + monkeypatch.setattr("aisle.sim.engine_available", lambda engine: True) + root = Path(__file__).resolve().parents[2] + refused = _gates(root, "nexus") + assert refused["ok"] is False and refused["gate"] == "env_hash" + assert "build receipt" in refused["detail"] diff --git a/tests/unit/test_rollout_metrics.py b/tests/unit/test_rollout_metrics.py index 2dca49a2..14b4f9d6 100644 --- a/tests/unit/test_rollout_metrics.py +++ b/tests/unit/test_rollout_metrics.py @@ -133,6 +133,9 @@ def fake_popen(cmd, cwd=None, env=None, **kwargs): "ok": True, "env_hash": "x", "sim_extra": "sim", + # ADR-67: the gate always resolves an engine; the runner now + # requires the key rather than defaulting to genesis + "sim_engine": "genesis", "sim_backend": "metal", "sim_device": "mps", }, @@ -334,6 +337,9 @@ def fake_popen(cmd, cwd=None, env=None, **kwargs): "ok": True, "env_hash": "x", "sim_extra": "sim", + # ADR-67: the gate always resolves an engine; the runner now + # requires the key rather than defaulting to genesis + "sim_engine": "genesis", "sim_backend": "metal", "sim_device": "mps", }, @@ -491,6 +497,9 @@ def fake_popen(cmd, cwd=None, env=None, **kwargs): "ok": True, "env_hash": "x", "sim_extra": "sim", + # ADR-67: the gate always resolves an engine; the runner now + # requires the key rather than defaulting to genesis + "sim_engine": "genesis", "sim_backend": "metal", "sim_device": "mps", }, @@ -810,7 +819,11 @@ def test_rollout_refuses_behavioral_before_reserving_any_episodes(tmp_path, monk would still pass.""" from aisle.harness import rollout as ro - monkeypatch.setattr(ro, "run_gates", lambda *a, **k: {"ok": True, "sim_backend": "genesis"}) + monkeypatch.setattr( + ro, + "run_gates", + lambda *a, **k: {"ok": True, "sim_engine": "genesis", "sim_backend": "genesis"}, + ) def run(mode): return ro.rollout( @@ -864,3 +877,137 @@ def test_authored_work_binding_is_refused(tmp_path): instrumented_graph( graph, REPO_ROOT, tmp_path, graph_snapshot=yaml.safe_dump(doc).encode(), work_launch=0 ) + + +# -- HAR-1 operator progress ---------------------------------------------------- + + +def test_progress_report_distinguishes_build_from_running_episode(): + """HAR-1 (operator feedback), CON-8 (stdout stays JSON): the periodic + stderr line names the phase — scene build before the first trace byte, + the running episode against its wall budget after — plus the last + verdict and the run deadline, so a long build reads differently from a + wedged episode.""" + from aisle.harness.rollout import progress_report + + building = progress_report( + now=100.0, + started=10.0, + deadline=1000.0, + lines=0, + episodes=2, + seeds=[0, 1], + last_line_t=10.0, + episode_grace_s=660.0, + traces_size=0, + last_record=None, + ) + assert building.startswith("[rollout +1m30s]") + assert "episode 1/2 (seed 0): building the scene, no traces yet (1m30s)" in building + assert building.endswith("run deadline in 15m00s") + + running = progress_report( + now=200.0, + started=10.0, + deadline=1000.0, + lines=1, + episodes=2, + seeds=[0, 1], + last_line_t=150.0, + episode_grace_s=240.0, + traces_size=3_100_000, + last_record={"episode": 0, "seed": 0, "status": "success", "failure": None, "t_end": 20.83}, + relaunches=1, + ) + assert "episode 2/2 (seed 1) running 50s of 4m00s wall budget, traces 3.1 MB" in running + assert "last: episode 0 seed 0 success at t=20.8s" in running + assert "relaunches 1" in running + + failed = progress_report( + now=200.0, + started=10.0, + deadline=1000.0, + lines=2, + episodes=2, + seeds=[0, 1], + last_line_t=150.0, + episode_grace_s=240.0, + traces_size=10, + last_record={ + "episode": 1, + "seed": 1, + "status": "fail", + "failure": "collision", + "t_end": 0.3, + }, + ) + assert "all 2 episodes recorded, waiting for the client to exit" in failed + assert "fail (collision) at t=0.3s" in failed + + +def test_progress_interval_env_override(): + """HAR-1: AISLE_ROLLOUT_PROGRESS_S sets the cadence, 0 disables, and a + malformed value is refused rather than silently muting the report.""" + from aisle.harness.rollout import PROGRESS_INTERVAL_S, progress_interval_s + + assert progress_interval_s({}) == PROGRESS_INTERVAL_S + assert progress_interval_s({"AISLE_ROLLOUT_PROGRESS_S": "5"}) == 5.0 + assert progress_interval_s({"AISLE_ROLLOUT_PROGRESS_S": "0"}) == 0.0 + with pytest.raises(ValueError, match="AISLE_ROLLOUT_PROGRESS_S"): + progress_interval_s({"AISLE_ROLLOUT_PROGRESS_S": "soon"}) + + +def test_timing_summary_weights_windows_and_reports_rtf(): + """ADR-67: the manifest's `sim_timing` is the step- and frame-weighted + aggregate of the bridge's sidecar rows, with the real-time factor as sim + seconds over the wall seconds of the bridge's ticks. A tick already + contains the renders it triggered, so they must not be added again (PR + #594 review); the progress phrase omits what the engine did not report.""" + from aisle.harness.rollout import summarize_timing, timing_phrase + + rows = [ + { + "engine": "nexus", + "steps": 100, + "step_ms_mean": 6.0, + "step_ms_max": 9.0, + "frames": 15, + "tick_ms_mean": 6.3, + "render_ms_mean": 2.0, + "gpu_ms_mean": 5.0, + "sim_window_s": 1.0, + }, + { + "engine": "nexus", + "steps": 100, + "step_ms_mean": 8.0, + "step_ms_max": 12.0, + "frames": 15, + "tick_ms_mean": 8.6, + "render_ms_mean": 4.0, + "gpu_ms_mean": None, + "sim_window_s": 1.0, + }, + ] + summary = summarize_timing(rows) + assert summary["steps"] == 200 and summary["frames"] == 30 + assert summary["step_ms_mean"] == pytest.approx(7.0) + assert summary["step_ms_max"] == 12.0 + assert summary["render_ms_mean"] == pytest.approx(3.0) + assert summary["tick_ms_mean"] == pytest.approx(7.45) + assert summary["gpu_ms_mean"] == pytest.approx(5.0) # only the window that reported it + # 2 sim seconds over 1.49 s of ticks, whose renders are already inside + assert summary["rtf"] == pytest.approx(2.0 / 1.49, rel=1e-6) + # an older row with no tick: its renders are outside the step, so they count + legacy = [{k: v for k, v in rows[0].items() if k != "tick_ms_mean"}] + assert summarize_timing(legacy)["rtf"] == pytest.approx(1.0 / (0.6 + 0.03), rel=1e-6) + assert ( + timing_phrase(summary) + == "physics 7.0 ms/step, tick 7.5 ms, gpu 5.0 ms, render 3.0 ms/frame, rtf 1.34x" + ) + genesis = summarize_timing( + [{**rows[1], "engine": "genesis", "frames": 0, "render_ms_mean": None}] + ) + assert genesis["gpu_ms_mean"] is None and genesis["render_ms_mean"] is None + assert timing_phrase(genesis) == "physics 8.0 ms/step, tick 8.6 ms, rtf 1.16x" + assert summarize_timing([]) == {} and timing_phrase({}) == "" diff --git a/tests/unit/test_rollout_typed_shutdown.py b/tests/unit/test_rollout_typed_shutdown.py index df930503..2e5a028c 100644 --- a/tests/unit/test_rollout_typed_shutdown.py +++ b/tests/unit/test_rollout_typed_shutdown.py @@ -53,6 +53,7 @@ def spawn(graph, run_dir, environment, **kwargs): "ok": True, "env_hash": "x", "sim_extra": "sim", + "sim_engine": "genesis", "sim_backend": "metal", "sim_device": "mps", }, diff --git a/tests/unit/test_run_attestation.py b/tests/unit/test_run_attestation.py index b3da3613..673a326f 100644 --- a/tests/unit/test_run_attestation.py +++ b/tests/unit/test_run_attestation.py @@ -110,9 +110,10 @@ def mutate_after_gate(*args, **kwargs): monkeypatch.setattr( rollout_module, "resolve_sim_identity", - lambda extra: { + lambda extra, engine="genesis": { "ok": True, "sim_extra": extra, + "sim_engine": engine, "sim_backend": "cuda", "sim_device": "NVIDIA Test GPU", }, @@ -191,6 +192,8 @@ def test_settle_records_actual_episode_count(tmp_path, monkeypatch): from aisle.harness import rollout as rollout_module + # the unit tier has no sim extra, so no engine is importable (ADR-67) + monkeypatch.setattr("aisle.sim.engine_available", lambda engine: True) root = _fake_root(tmp_path) # the trusted gate demands dist attestation evidence (ADR-24 D2/D3); # extend the fixture's stub checker to supply it diff --git a/tests/unit/test_sim_engine.py b/tests/unit/test_sim_engine.py new file mode 100644 index 00000000..226f01ae --- /dev/null +++ b/tests/unit/test_sim_engine.py @@ -0,0 +1,220 @@ +"""Engine selection for the bridge and the harness (ADR-67, ADR-68). + +Unit-marked: nothing here imports a simulator. The Nexus and rapier backend +modules are imported to prove they stay sim-free at import time (CON-12), +like `aisle.scenes.pharmacy`. +""" + +import sys + +import numpy as np +import pytest + +from aisle.sim import ( + DEFAULT_ENGINE, + ENGINE_BACKENDS, + ENGINE_MODULES, + ENGINES, + normalize_engine, + select_engine, + select_nexus_backend, + select_rapier_backend, + select_sim_backend, + validate_backend, +) + +pytestmark = pytest.mark.unit + + +def test_default_engine_is_genesis(): + """ADR-67: the frozen Genesis path stays the default; the engine is an + explicit, graph-declared choice.""" + assert DEFAULT_ENGINE == "genesis" + assert select_engine({}) == "genesis" + assert select_engine({"AISLE_SIM_ENGINE": "nexus"}) == "nexus" + assert select_engine({"AISLE_SIM_ENGINE": " Nexus "}) == "nexus" + assert select_engine({"AISLE_SIM_ENGINE": "rapier"}) == "rapier" + + +def test_unknown_engine_is_refused_not_defaulted(): + """ADR-67 (same rule as TC-9's rung): a typo must not silently attest + another engine.""" + with pytest.raises(ValueError, match="unknown simulation engine"): + normalize_engine("bullet") + with pytest.raises(ValueError, match="unknown simulation engine"): + select_engine({"AISLE_SIM_ENGINE": ""}) + + +@pytest.mark.parametrize( + ("sim_extra", "platform_name", "cuda_available", "expected"), + [ + ("sim", "Darwin", False, "metal"), + ("sim", "Darwin", True, "metal"), + ("sim", "Linux", False, "webgpu"), + ("sim", "Linux", True, "webgpu"), + ("cuda", "Linux", True, "cuda"), + ], +) +def test_select_nexus_backend(sim_extra, platform_name, cuda_available, expected): + """ADR-67: the portable extra never changes physics because a GPU is + visible; CUDA is the explicit Linux opt-in.""" + assert select_nexus_backend(sim_extra, platform_name, cuda_available) == expected + + +@pytest.mark.parametrize( + ("sim_extra", "platform_name", "cuda_available", "message"), + [ + ("cuda", "Darwin", True, "only on Linux"), + ("cuda", "Linux", False, "requires an available CUDA device"), + ("gpu", "Linux", True, "unknown simulation extra"), + ], +) +def test_select_nexus_backend_fails_closed(sim_extra, platform_name, cuda_available, message): + with pytest.raises(ValueError, match=message): + select_nexus_backend(sim_extra, platform_name, cuda_available) + + +def test_select_rapier_backend_is_cpu_only(): + """ADR-68: rapier steps on the CPU on every platform, so the portable + extra resolves to `cpu` and the CUDA extra selects nothing rather than + silently handing back a backend rapier cannot run.""" + assert select_rapier_backend("sim", "Darwin") == "cpu" + assert select_rapier_backend("sim", "Linux", cuda_available=True) == "cpu" + assert ENGINE_BACKENDS["rapier"] == ("cpu",) + with pytest.raises(ValueError, match="CPU only"): + select_rapier_backend("cuda", "Linux", cuda_available=True) + with pytest.raises(ValueError, match="unknown simulation extra"): + select_rapier_backend("gpu", "Linux") + + +def test_rapier_engine_declares_its_renderer(): + """ADR-68: the rapier engine renders through the Nexus viewer, so both + wheels are part of the engine. A missing nexus3d must be refused at + selection time, not at the first render.""" + assert ENGINE_MODULES["rapier"] == ("rapier3d", "nexus3d") + + +def test_select_sim_backend_dispatches_per_engine(): + """ADR-67, ADR-68: the Genesis table is the frozen module's own, + unchanged, and each engine resolves through its own selector.""" + assert select_sim_backend("genesis", "sim", "Darwin") == "metal" + assert select_sim_backend("genesis", "sim", "Linux") == "cpu" + assert select_sim_backend("nexus", "sim", "Linux") == "webgpu" + assert select_sim_backend("rapier", "sim", "Darwin") == "cpu" + + +def test_validate_backend_per_engine(): + """BRG-6: AISLE_SIM_BACKEND must name a backend the engine can run.""" + for engine in ENGINES: + for backend in ENGINE_BACKENDS[engine]: + assert validate_backend(engine, backend) == backend + assert validate_backend("nexus", None) is None + with pytest.raises(ValueError, match="unknown simulation backend"): + validate_backend("genesis", "webgpu") + + +def test_backend_modules_stay_sim_free(): + """CON-12: importing the selection package and either engine backend must + import none of genesis, nexus3d or rapier3d.""" + import importlib + + importlib.import_module("aisle.sim") + importlib.import_module("aisle.sim.nexus_backend") + importlib.import_module("aisle.sim.rapier_backend") + assert "genesis" not in sys.modules + assert "nexus3d" not in sys.modules + assert "rapier3d" not in sys.modules + + +def test_nexus_physics_constants_are_declared(): + """SCN-2 spirit: the Nexus realization constants live in a toml, outside + the frozen scene directory (CON-7).""" + from aisle.sim.nexus_backend import load_nexus_physics + + physics = load_nexus_physics() + assert physics["robot"]["default_kp"] == 100.0 # Genesis's URDF default + assert physics["robot"]["default_kv"] == 10.0 + assert len(physics["ground"]["size"]) == 3 + # the pinch-grasp solver settings (ADR-67): friction solved with the + # normals, Genesis's max() friction rule + assert physics["sim"]["friction_in_bias_pass"] is True + assert physics["sim"]["friction_combine_rule"] == "max" + assert physics["sim"]["internal_pgs_iterations"] >= 1 + assert isinstance(physics["sim"]["implicit_coriolis"], bool) + # render settings the Nexus viewer applies to its sensor cameras + assert physics["camera"]["msaa_samples"] in (1, 4) + assert 0.0 <= physics["camera"]["shadow_softness"] <= 1.0 + assert len(physics["ground"]["color"]) == 4 + assert physics["camera"]["shadow_resolution"] >= 1024 + assert 1 <= physics["camera"]["shadow_atlas_layers"] <= 16 + + +def test_rapier_physics_constants_are_declared(): + """SCN-2 spirit (ADR-68): the rapier realization constants live in their + own toml beside the Nexus one, outside the frozen scene directory + (CON-7), and the blocks that describe the shared renderer are copies of + it rather than a second opinion.""" + from aisle.sim.nexus_backend import load_nexus_physics + from aisle.sim.rapier_backend import load_rapier_physics + + physics = load_rapier_physics() + sim = physics["sim"] + # single-threaded stepping is what buys the reproducible step (CON-5) + assert sim["num_threads"] == 1 + assert sim["num_solver_iterations"] >= 1 + assert sim["num_internal_pgs_iterations"] >= 1 + assert sim["friction_combine_rule"] == "max" # Genesis's rule, as on Nexus + assert sim["contact_natural_frequency"] > 0.0 + assert sim["allowed_linear_error"] > 0.0 + # rapier's damped least squares needs its own step, but the frozen + # tolerances in physics.toml stay the acceptance gate (SCN-3) + assert 0.0 < physics["ik"]["damping"] <= 1.0 + assert physics["ik"]["epsilon_linear"] < 0.015 + assert physics["robot"]["default_kp"] == 100.0 # Genesis's URDF default + assert physics["robot"]["default_kv"] == 10.0 + # the Nexus-only scheduling knobs have no rapier counterpart + assert not {"friction_in_bias_pass", "implicit_coriolis", "substep_refresh"} & set(sim) + nexus = load_nexus_physics() + for section in ("camera", "light", "ground"): + assert physics[section] == nexus[section], section + + +def test_checkerboard_texture_alternates(): + """The floor texture is a deterministic checkerboard of the two ground + colors, `squares` cells a side at `px_per_square` pixels each.""" + from aisle.sim.nexus_backend import checkerboard_texture + + image = checkerboard_texture(4, [0.0, 0.0, 0.0], [1.0, 1.0, 1.0, 1.0], px_per_square=2) + assert image.shape == (8, 8, 3) and image.dtype == np.uint8 + assert image[0, 0].tolist() == [0, 0, 0] and image[0, 2].tolist() == [255, 255, 255] + assert image[2, 0].tolist() == [255, 255, 255] and image[2, 2].tolist() == [0, 0, 0] + assert image[1, 1].tolist() == [0, 0, 0] # 2x2 pixel cells + + +def test_uv_cube_asset_parses(): + """T2 labels on Nexus reuse the committed UV cube: 12 faces, one vertex + per (position, uv) pair, uvs inside the unit square.""" + from aisle.sim.nexus_backend import load_uv_cube + + positions, faces, uvs = load_uv_cube() + assert len(faces) == 12 + assert len(positions) == len(uvs) == 24 + assert all(0.0 <= u <= 1.0 and 0.0 <= v <= 1.0 for u, v in uvs) + assert all(abs(abs(c) - 0.5) < 1e-9 for p in positions for c in p) + + +def test_lookat_transform_matches_genesis_convention(): + """VER-8 / SCN-5: the Nexus camera pose is Genesis's look-at transform + (OpenGL frame, up = +z): -Z looks at the target, +Y is up.""" + import numpy as np + + from aisle.sim.nexus_backend import pos_lookat_up_to_transform + + transform = pos_lookat_up_to_transform((0.55, 0.0, 1.2), (0.55, 0.0, 0.2)) + forward = -transform[:3, 2] + assert np.allclose(forward, [0.0, 0.0, -1.0]) + assert np.allclose(transform[:3, 3], [0.55, 0.0, 1.2]) + assert np.allclose(transform[:3, :3] @ transform[:3, :3].T, np.eye(3), atol=1e-9) + # a tilted look-at keeps world +z in the camera's up half-plane + tilted = pos_lookat_up_to_transform((1.0, 0.0, 1.0), (0.0, 0.0, 0.0)) + assert tilted[2, 1] > 0.0 diff --git a/tests/unit/test_swap.py b/tests/unit/test_swap.py index 0df6af38..60409686 100644 --- a/tests/unit/test_swap.py +++ b/tests/unit/test_swap.py @@ -476,6 +476,27 @@ def test_bridge_is_a_trust_anchor(): assert isinstance(result, str) and "trust anchor" in result +def test_the_engine_realization_is_a_trust_anchor(tmp_path): + """HAR-10, CON-7, ADR-67: the engine realization (`src/aisle/sim`) is + the bridge's other live half — it decides WHICH physics steps the scene + and, through nexus_physics.toml, the solver settings it steps with. It + is outside the env_hash fence by decision, so a live swap would be the + one way to change the physics of a running attested dataflow.""" + graphs = tmp_path / "graphs" + graphs.mkdir() + graph = graphs / "toy.yaml" + graph.write_text( + yaml.safe_dump( + {"nodes": [{"id": "sim-engine", "path": "../src/aisle/sim/nexus_backend.py"}]}, + sort_keys=False, + ) + ) + result = swapped_graph_doc( + graph, "sim-engine", {"id": "sim-engine", "path": "replacement.py"}, tmp_path + ) + assert isinstance(result, str) and "trust anchor" in result + + def test_swap_refuses_a_rung_change_even_on_an_unanchored_bridge(tmp_path): """Issue #127 defense in depth: anchoring dora-genesis closes today's path, but the rung rule binds ANY sim_bridge provider — a future diff --git a/tests/unit/test_typed_rollout_gates.py b/tests/unit/test_typed_rollout_gates.py index 31ac9cdc..c8ccdde5 100644 --- a/tests/unit/test_typed_rollout_gates.py +++ b/tests/unit/test_typed_rollout_gates.py @@ -17,7 +17,13 @@ def _gate_inputs(tmp_path, monkeypatch, *, env_ok=True, episodes_left=10, attest stage, record = _stage(tmp_path) graph = Path(record["snapshot_record"]["snapshot_root"]) / "graphs/expert_t1.yaml" monkeypatch.setattr( - rollout, "resolve_sim_identity", lambda _: {"ok": True, "sim_backend": "genesis"} + rollout, + "resolve_sim_identity", + lambda extra, engine="genesis": { + "ok": True, + "sim_engine": engine, + "sim_backend": "genesis", + }, ) monkeypatch.setattr(rollout, "resolve_trusted_baseline", lambda *a: ("1" * 40, None)) monkeypatch.setattr( diff --git a/tests/unit/test_typed_rollout_lifecycle.py b/tests/unit/test_typed_rollout_lifecycle.py index 335fe70f..aa1a6da5 100644 --- a/tests/unit/test_typed_rollout_lifecycle.py +++ b/tests/unit/test_typed_rollout_lifecycle.py @@ -49,9 +49,10 @@ def validation(stage, record, **kwargs): monkeypatch.setattr( rollout, "resolve_sim_identity", - lambda extra: { + lambda extra, engine="genesis": { "ok": True, "sim_extra": extra, + "sim_engine": engine, "sim_backend": "genesis", "sim_device": "fixture", }, diff --git a/tools/docs_inventory.py b/tools/docs_inventory.py index 67b674cb..60d73210 100644 --- a/tools/docs_inventory.py +++ b/tools/docs_inventory.py @@ -110,6 +110,12 @@ def _graph_inventory(root: Path, tracked: set[str] | None) -> list[dict]: "perception": ( env.get("AISLE_PERCEPTION", "L0 (default)") if env is not None else "—" ), + # ADR-67: the engine is declared on the bridge like the rung, + # and a run on another engine is another environment; without + # this column every graph reads as Genesis. + "engine": ( + env.get("AISLE_SIM_ENGINE", "genesis (default)") if env is not None else "—" + ), "nodes": node_ids, } ) @@ -283,7 +289,7 @@ def _test_inventory(root: Path, tracked: set[str] | None) -> tuple[list[str], li "skills": "registered agent-authored skills", "skills_pending_review": "recovered campaign skills, not registered", "specs": "numbered specs with MUST ids (000 = constitution)", - "src": "the aisle package: scenes, bridge, verifier, reset, harness, nodes", + "src": "the aisle package: scenes, bridge, sim engines, verifier, reset, harness, nodes", "templates": "scaffolding for new artifacts", "tests": "unit / sim / graph suites", "tools": "CI, env_hash, trace_check, campaign runners", @@ -370,8 +376,8 @@ def render_inventory(root: Path) -> tuple[str, dict[str, int]]: "", "## Graphs", "", - "| Graph | Scene/scenario | Embodiment | Perception | Nodes |", - "|---|---|---|---|---:|", + "| Graph | Scene/scenario | Embodiment | Perception | Engine | Nodes |", + "|---|---|---|---|---|---:|", ] for row in graphs: relative = _relative(row["path"], root) @@ -383,6 +389,7 @@ def render_inventory(root: Path) -> tuple[str, dict[str, int]]: _cell(f"{row['scene']} / {row['scenario']}"), _cell(row["embodiment"]), _cell(row["perception"]), + _cell(row["engine"]), str(len(row["nodes"])), ] ) diff --git a/tools/engine_sources.py b/tools/engine_sources.py new file mode 100644 index 00000000..5e0ec92c --- /dev/null +++ b/tools/engine_sources.py @@ -0,0 +1,114 @@ +#!/usr/bin/env python3 +"""Materialize the pinned engine sources from GitHub (ADR-67, ADR-68). + +`engine-runtime.json` pins the two repositories AISLE builds wheels from, +nexus and rapier, the same way `dora-runtime.json` pins the Dora CLI. Each is +fetched by exact commit. + +kiss3d is deliberately absent, and so are the rapier crates the engine links +against: nexus's own Cargo manifest patches kiss3d from git by rev and takes +the published rapier release from crates.io, so cargo resolves and caches +both. Pinning them here as well would be two sources of +truth for one dependency. + +The engine installers use this when no local checkout is passed, so a machine +with no sibling working copies can still build the wheels, and the commit a +wheel came from is the pinned one rather than whatever happened to be checked +out. CON-8: JSON on stdout, logs on stderr, exit 0 iff ok. +""" + +from __future__ import annotations + +import argparse +import json +import subprocess +import sys +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +PINS = ROOT / "engine-runtime.json" +DEFAULT_SOURCES = ROOT / ".engine-sources" +NAMES = ("nexus", "rapier") + + +def load_pins(path: Path | None = None) -> dict: + """The pinned sources, refusing a file that does not name both.""" + pins = json.loads((path or PINS).read_text()) + missing = [n for n in NAMES if n not in pins.get("sources", {})] + if missing: + raise ValueError(f"engine pins miss {missing}; both of {NAMES} are built here") + for name, source in pins["sources"].items(): + for field in ("repository", "branch", "commit"): + if not source.get(field): + raise ValueError(f"engine pin {name!r} has no {field}") + if len(source["commit"]) != 40: + raise ValueError(f"engine pin {name!r} must name a full commit, not a prefix") + return pins + + +def _run(command: list[str], cwd: Path) -> str: + proc = subprocess.run(command, cwd=cwd, capture_output=True, text=True) + if proc.returncode != 0: + raise RuntimeError(f"{' '.join(command)} failed:\n{proc.stderr.strip()}") + return proc.stdout.strip() + + +def fetch(name: str, source: dict, into: Path) -> dict: + """One pinned checkout at `into/`, fetched by exact commit. + + Reuses an existing checkout when it already sits on the pinned commit, so + a rebuild does not re-download or invalidate the cargo target directory.""" + repo = into / name + if (repo / ".git").is_dir(): + # an interrupted fetch leaves an initialized repo with no HEAD, so ask + # for the revision rather than assuming one exists + head = subprocess.run( + ["git", "rev-parse", "--verify", "-q", "HEAD"], + cwd=repo, + capture_output=True, + text=True, + ).stdout.strip() + if head == source["commit"]: + return {"name": name, "path": str(repo), "commit": head, "fetched": False} + else: + repo.mkdir(parents=True, exist_ok=True) + _run(["git", "init", "-q"], repo) + _run(["git", "remote", "add", "origin", source["repository"]], repo) + _run(["git", "fetch", "--depth", "1", "origin", source["commit"]], repo) + _run(["git", "checkout", "-q", "--detach", "FETCH_HEAD"], repo) + head = _run(["git", "rev-parse", "HEAD"], repo) + if head != source["commit"]: + raise ValueError(f"{name}: fetched {head} but the pin names {source['commit']}") + return {"name": name, "path": str(repo), "commit": head, "fetched": True} + + +def materialize(into: Path | None = None, pins_path: Path | None = None) -> dict: + """The pinned checkouts. Returns {name: {path, commit, fetched}}.""" + into = Path(into) if into else DEFAULT_SOURCES + pins = load_pins(pins_path) + into.mkdir(parents=True, exist_ok=True) + results = [fetch(name, pins["sources"][name], into) for name in NAMES] + return { + "ok": True, + "sources_dir": str(into), + "sources": {r["name"]: r for r in results}, + } + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--into", type=Path, default=DEFAULT_SOURCES) + parser.add_argument("--pins", type=Path, default=PINS) + args = parser.parse_args(argv) + try: + report = materialize(args.into, args.pins) + except (RuntimeError, ValueError, OSError, json.JSONDecodeError) as exc: + print(json.dumps({"ok": False, "error": str(exc)})) + print(str(exc), file=sys.stderr) + return 1 + print(json.dumps(report)) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/env_hash.py b/tools/env_hash.py index e358f2bf..b912e513 100755 --- a/tools/env_hash.py +++ b/tools/env_hash.py @@ -15,10 +15,18 @@ its blob at — regenerating the local json or editing the checker after changing frozen code no longer blesses the change. JSON on stdout, logs on stderr, exit 0 iff ok. + +--sim-engine adds ADR-67's engine attestation to any mode: the +`sim` block (`sim_engine_hash` over src/aisle/sim/** plus the engine build +receipt) and that engine's own distribution in the attested set. The digest +is recorded beside the engine-neutral frozen-set hash; an out-of-lock engine +without a usable clean-source receipt is refused because the receipt is its +only source provenance. """ import argparse import hashlib +import importlib import json import re import subprocess @@ -150,6 +158,21 @@ def _baseline_hash(root: Path, ref: str) -> tuple[str | None, str | None]: # the attested set beyond registry pip: sources — the sim core whose code # decides physics and gate verdicts (ADR-24 resolved question 1) ATTESTED_SIM_CORE = ("genesis-world", "torch", "dora-rs", "pyarrow") +# ADR-67: the distribution that actually steps the scene, per engine. The +# core above is engine neutral, so a Nexus run used to attest genesis-world +# and say nothing about the wheel whose solver produced the trajectories; +# attested_set() adds the running engine's dist instead. +ENGINE_DISTS = { + "genesis": ("genesis-world",), + "nexus": ("dimforge-nexus3d",), + # rapier steps the scene and the Nexus viewer renders it (ADR-68) + "rapier": ("rapier3d", "dimforge-nexus3d"), +} +#: The engine realization package (ADR-67, ADR-68): the engine selector, the +#: Nexus and rapier backends, and their solver settings. Deliberately outside +#: FROZEN_DIRS (widening the fence moves every Genesis env_hash and breaks +#: the measured record's continuity), so it carries its own digest instead. +SIM_REALIZATION_DIR = "src/aisle/sim" LOCK_FILE = "uv.lock" @@ -234,8 +257,120 @@ def classify_direct_url(direct_url: dict) -> str | None: return None -def attested_set(root: Path) -> list[str]: - return sorted(set(registry_pip_dists(root)) | set(map(canonical_name, ATTESTED_SIM_CORE))) +def attested_set(root: Path, engine: str | None = None) -> list[str]: + """The dists whose provenance is checked. `engine` (ADR-67) adds the + running engine's own distribution; None keeps the pre-ADR-67 set.""" + names = set(registry_pip_dists(root)) | set(map(canonical_name, ATTESTED_SIM_CORE)) + names |= {canonical_name(d) for d in ENGINE_DISTS.get(engine or "", ())} + return sorted(names) + + +def sim_realization_files(root: Path) -> list[Path]: + """Files under SIM_REALIZATION_DIR, sorted, __pycache__ excluded.""" + base = root / SIM_REALIZATION_DIR + if not base.is_dir(): + return [] + return sorted( + (p for p in base.rglob("*") if p.is_file() and "__pycache__" not in p.parts), + key=lambda p: p.relative_to(root).as_posix(), + ) + + +def sim_engine_hash(root: Path, engine: str, build: dict | None = None) -> dict: + """ADR-67 engine attestation: the identity env_hash cannot carry. + + One sha256 over the engine name, the realization package + (`src/aisle/sim/**`, which holds nexus_physics.toml's substeps, PGS + iterations, contact frequencies and friction rule) and the engine build + provenance. Two Nexus runs with different solver settings, or with the + same settings from different engine sources, differ here while their + env_hash is byte-identical; the manifest's `sim_engine` string alone + could not tell them apart. + + `build` is the out-of-lock engine's build receipt, i.e. + `nexus_runtime.read_receipt(root)["receipt"]`, or None for an engine + that comes from the lock. Returns the facts for the run manifest: + {"engine", "sim_engine_hash", "n_files", "build"}; refusal policy is + the caller's (CON-8).""" + files = sim_realization_files(root) + digest = hashlib.sha256() + digest.update(engine.encode()) + digest.update(b"\0") + for path in files: + digest.update(path.relative_to(root).as_posix().encode()) + digest.update(b"\0") + # same framing as compute_env_hash: fixed-length per-file digests + digest.update(hashlib.sha256(path.read_bytes()).digest()) + digest.update(json.dumps(build, sort_keys=True).encode()) + return { + "engine": engine, + "sim_engine_hash": digest.hexdigest(), + "n_files": len(files), + "build": build, + } + + +def _engine_build(root: Path, engine: str) -> dict | None: + """The engine's build receipt, read through the runtime tool's own reader + so the receipt format is never copied here (ADR-33's rule). Engines + that come from the lock have no receipt. The rapier engine renders + through the Nexus viewer, so both receipts describe its build.""" + module = {"nexus": "nexus_runtime", "rapier": "rapier_runtime"}.get(engine) + if module is None: + return None + sys.path.insert(0, str(Path(__file__).resolve().parent)) + try: + reader = importlib.import_module(module).read_receipt + except (ImportError, AttributeError): + return None + build = reader(root).get("receipt") + if engine == "rapier" and build is not None: + from nexus_runtime import read_receipt + + build = {**build, "renderer": read_receipt(root).get("receipt")} + return build + + +def _receipt_problem(receipt: object, *, label: str, source: str) -> str | None: + """Why one out-of-lock wheel receipt cannot identify its source bytes.""" + if not isinstance(receipt, dict): + return f"{label} build receipt is missing or malformed" + if receipt.get("schema_version") != 1: + return f"{label} build receipt has an unsupported schema" + for field in ("wheel", "version", "platform"): + if not isinstance(receipt.get(field), str) or not receipt[field].strip(): + return f"{label} build receipt has no {field}" + sources = receipt.get("sources") + if not isinstance(sources, dict) or source not in sources: + return f"{label} build receipt does not identify its {source} source" + for name, identified in sources.items(): + if not isinstance(identified, dict): + return f"{label} build receipt has a malformed {name} source" + commit = identified.get("commit") + if not isinstance(commit, str) or re.fullmatch(r"[0-9a-f]{40}", commit) is None: + return f"{label} build receipt has no full commit for {name}" + if identified.get("dirty") is not False: + return f"{label} build receipt identifies a dirty {name} source" + return None + + +def engine_build_problem(engine: str, build: object) -> str | None: + """Fail-closed provenance verdict for an engine's out-of-lock wheels. + + Genesis comes from the uv lock and has no separate receipt. Nexus needs + one clean-source receipt; rapier needs both its solver receipt and the + nested Nexus renderer receipt (ADR-67/ADR-68, CON-5). + """ + if engine == "genesis": + return None + if engine == "nexus": + return _receipt_problem(build, label="Nexus", source="nexus") + if engine == "rapier": + problem = _receipt_problem(build, label="rapier", source="rapier") + if problem is not None: + return problem + return _receipt_problem(build.get("renderer"), label="Nexus renderer", source="nexus") + return f"unknown simulation engine {engine!r}" def environment_inventory() -> dict: @@ -263,7 +398,11 @@ def environment_inventory() -> dict: def dist_attestation( - root: Path, baseline_ref: str | None, extras: list[str], groups: list[str] | None = None + root: Path, + baseline_ref: str | None, + extras: list[str], + groups: list[str] | None = None, + engine: str | None = None, ) -> dict: """The ADR-24 D2 gate facts: fingerprint + problems (empty = attested) + the full environment inventory the post-session audit verifies @@ -296,7 +435,7 @@ def dist_attestation( "environment out of sync with the locked selection " f"(uv sync --locked --check: {(sync.stderr or sync.stdout).strip()[-200:]})" ) - for dist_name in attested_set(root): + for dist_name in attested_set(root, engine): try: dist = importlib.metadata.distribution(dist_name) except importlib.metadata.PackageNotFoundError: @@ -411,6 +550,12 @@ def main() -> int: help="declared uv extras for the attestation selection (repeatable); " "with --check, enables the ADR-24 dist attestation", ) + parser.add_argument( + "--sim-engine", + default=None, + help="ADR-67 engine (genesis|nexus|rapier): report the engine-specific " + "sim_engine_hash and attest that engine's own distribution", + ) args = parser.parse_args() if args.verify_records: @@ -436,6 +581,15 @@ def main() -> int: env_hash, n_files = compute_env_hash(args.root) report: dict = {"ok": True, "env_hash": env_hash, "n_files": n_files} + if args.sim_engine: + # ADR-67: the digest is recorded beside the engine-neutral env_hash. + # An out-of-lock engine must also carry the clean build receipt that + # is its only source provenance. + build = _engine_build(args.root, args.sim_engine) + report["sim"] = sim_engine_hash(args.root, args.sim_engine, build) + build_problem = engine_build_problem(args.sim_engine, build) + if build_problem is not None: + report.update(ok=False, error=build_problem, engine_build_problem=build_problem) if args.write: hash_path = args.root / HASH_FILE @@ -467,7 +621,9 @@ def main() -> int: # ADR-24: attestation FACTS ride the same trusted-blob check; # refusal policy is the gate's (HAR-2) groups = [g for g in (args.groups or []) if g] if args.groups is not None else None - report["dist"] = dist_attestation(args.root, args.baseline, args.extras, groups) + report["dist"] = dist_attestation( + args.root, args.baseline, args.extras, groups, args.sim_engine + ) print(json.dumps(report)) if not report["ok"]: diff --git a/tools/nexus_grasp_replay.py b/tools/nexus_grasp_replay.py new file mode 100644 index 00000000..ebf2a81f --- /dev/null +++ b/tools/nexus_grasp_replay.py @@ -0,0 +1,134 @@ +#!/usr/bin/env python3 +"""Replay a recorded run's joint and gripper commands into a fresh Nexus scene. + +The recorded `budget-guard__joint_cmd_safe` / `gripper_cmd_safe` traces of a +rollout are fed tick by tick to the Nexus pharmacy scene built for the same +seed, so the physics can be re-run offline under different solver settings +(`--substeps`, `--contact-frequency`, `--pgs`) without dora. Reports how far +the target box rose with the hand: the pinch-grasp fidelity check behind the +`[sim]` defaults of src/aisle/sim/nexus_physics.toml (ADR-67). CON-8: JSON on +stdout, logs on stderr, exit 0 iff the replay ran (not iff the grasp held). +""" + +from __future__ import annotations + +import argparse +import json +import sys +from pathlib import Path + +import numpy as np + +ROOT = Path(__file__).resolve().parents[1] + + +def _load(path: Path) -> dict: + import pyarrow as pa + + with pa.memory_map(str(path)) as src: + try: + table = pa.ipc.open_stream(src).read_all() + except pa.ArrowInvalid: + table = pa.ipc.open_file(src).read_all() + return table.to_pydict() + + +def replay( + run_dir: Path, + seed: int, + embodiment: str, + target: str, + steps: int, + substeps: int | None, + contact_frequency: float | None, + pgs: int | None, +) -> dict: + traces = run_dir / "traces" + joints = _load(traces / "budget-guard__joint_cmd_safe.arrow") + grips = _load(traces / "budget-guard__gripper_cmd_safe.arrow") + jt = np.asarray(joints["sim_time_ns"], dtype=np.int64) + gt = np.asarray(grips["sim_time_ns"], dtype=np.int64) + if jt.size == 0 or gt.size == 0: + raise ValueError(f"{traces} holds no joint or gripper commands to replay") + + import aisle.sim.nexus_backend as nb + from aisle.scenes.pharmacy import load_physics, to_numpy + + profile = load_physics()["embodiment"][embodiment] + open_m, close_m = float(profile["gripper_open_m"]), float(profile["gripper_close_m"]) + + base = nb.load_nexus_physics() + cfg = json.loads(json.dumps(base)) + if substeps is not None: + cfg["sim"]["substeps"] = substeps + if contact_frequency is not None: + cfg["sim"]["contact_natural_frequency"] = contact_frequency + cfg["sim"]["static_contact_natural_frequency"] = 2.0 * contact_frequency + if pgs is not None: + cfg["sim"]["internal_pgs_iterations"] = pgs + nb.load_nexus_physics = lambda: cfg + + handle = nb.build_scene(seed=seed, embodiment=embodiment, n_envs=1, headless=True) + robot, box = handle.robot, handle.boxes[target] + start = to_numpy(box.get_pos()).reshape(-1).copy() + samples = [] + for step in range(steps): + t_ns = step * 10_000_000 + if t_ns >= jt[0]: + cmd = np.asarray(joints["data"][int(np.searchsorted(jt, t_ns, side="right")) - 1]) + cmd = cmd.astype(np.float32).copy() + if t_ns >= gt[0]: + grip = float( + np.asarray(grips["data"][int(np.searchsorted(gt, t_ns, side="right")) - 1])[0] + ) + cmd[len(cmd) - int(profile["gripper_dofs"]) :] = open_m + ( + close_m - open_m + ) * np.clip(grip, 0.0, 1.0) + robot.control_dofs_position(cmd) + handle.scene.step() + if step % 50 == 49: + pos = to_numpy(box.get_pos()).reshape(-1) + samples.append({"t_s": (step + 1) / 100, "box_rise_m": float(pos[2] - start[2])}) + rise = max(s["box_rise_m"] for s in samples) if samples else 0.0 + return { + "run": str(run_dir), + "target": target, + "solver": handle.scene.state.rbd_solver_params(), + "samples": samples, + "max_box_rise_m": rise, + "final_box_rise_m": samples[-1]["box_rise_m"] if samples else 0.0, + } + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--run", type=Path, required=True, help="runs/ directory") + parser.add_argument("--seed", type=int, default=0) + parser.add_argument("--embodiment", default="franka") + parser.add_argument("--target", default="amoxicillin") + parser.add_argument("--steps", type=int, default=800, help="10 ms ticks to replay") + parser.add_argument("--substeps", type=int, default=None) + parser.add_argument("--contact-frequency", type=float, default=None) + parser.add_argument("--pgs", type=int, default=None) + args = parser.parse_args(argv) + try: + report = replay( + args.run.resolve(), + args.seed, + args.embodiment, + args.target, + args.steps, + args.substeps, + args.contact_frequency, + args.pgs, + ) + except (FileNotFoundError, KeyError, ImportError, ValueError) as exc: + print(json.dumps({"ok": False, "error": str(exc)})) + print(str(exc), file=sys.stderr) + return 1 + print(json.dumps({"ok": True, **report})) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/nexus_runtime.py b/tools/nexus_runtime.py new file mode 100644 index 00000000..c49568c1 --- /dev/null +++ b/tools/nexus_runtime.py @@ -0,0 +1,237 @@ +#!/usr/bin/env python3 +"""Build and install AISLE's Nexus Python module from sibling checkouts (ADR-67). + +Nexus is an optional engine outside the uv lock: this builds the `nexus3d` +wheel with maturin from a nexus checkout (whose Cargo manifest resolves the +rapier and kiss3d crates it links), installs it into the project environment +with `uv pip`, and writes a receipt naming both source commits so a run's +`bridge_info` can be traced to the exact engine sources. `read_receipt()` +is the public reader for that provenance: the run manifest and +`env_hash.sim_engine_hash` record it, since the receipt is gitignored and +local to the machine that built the wheel. CON-8: JSON on stdout, logs on +stderr, exit 0 iff ok. +""" + +from __future__ import annotations + +import argparse +import json +import platform +import subprocess +import sys +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +RECEIPT = ROOT / ".nexus-runtime-receipt.json" +WHEEL_GLOB = "dimforge_nexus3d-*.whl" + + +def receipt_path(root: Path | None = None) -> Path: + """Where the build receipt lives for a checkout (default: this one).""" + return (ROOT if root is None else Path(root)) / RECEIPT.name + + +def read_receipt(root: Path | None = None) -> dict: + """The ADR-67 engine build provenance, for a run manifest to carry. + + The receipt is gitignored and local to the machine that built the + wheel, so recording it at rollout time is the only trace of which + engine sources produced a run. Never raises: an absent or malformed + receipt is a reported fact, not an exception. + + Returns {"installed": bool, "path": str, "receipt": dict | None, + "problem": str | None}; `receipt` holds the wheel name, version, + cargo feature, platform and the nexus/rapier/kiss3d commits. + """ + path = receipt_path(root) + result: dict = {"installed": False, "path": str(path), "receipt": None, "problem": None} + if not path.is_file(): + result["problem"] = ( + f"no nexus build receipt at {path}; run `tools/nexus_runtime.py install`" + ) + return result + try: + receipt = json.loads(path.read_text()) + except (OSError, json.JSONDecodeError) as exc: + result["problem"] = f"unreadable nexus build receipt at {path}: {exc}" + return result + if not isinstance(receipt, dict): + result["problem"] = f"malformed nexus build receipt at {path}: expected a JSON object" + return result + return {"installed": True, "path": str(path), "receipt": receipt, "problem": None} + + +def resolve_sources( + nexus: Path | None, rapier: Path | None, kiss3d: Path | None +) -> tuple[Path, Path | None, Path | None]: + """Local checkouts when given, otherwise the pinned GitHub source. + + Passing any path keeps the whole set local, so a developer working across + sibling checkouts never gets a surprise mix of local and pinned sources + (ADR-67). With none given, only nexus is fetched: its manifest takes rapier + from crates.io and patches kiss3d from git by rev, and cargo resolves both.""" + if nexus is not None or rapier is not None or kiss3d is not None: + return (nexus or ROOT.parent / "nexus", rapier, kiss3d) + # pinned build: cargo fetches rapier from crates.io and kiss3d from git + # by rev, so no local copy of either is needed + sys.path.insert(0, str(Path(__file__).resolve().parent)) + from engine_sources import materialize + + report = materialize() + paths = report["sources"] + print(f"using the pinned engine sources in {report['sources_dir']}", file=sys.stderr) + return (Path(paths["nexus"]["path"]), None, None) + + +def _run(command: list[str], cwd: Path) -> str: + proc = subprocess.run(command, cwd=cwd, capture_output=True, text=True) + if proc.returncode != 0: + raise RuntimeError(f"{' '.join(command)} failed:\n{proc.stderr.strip()}") + return proc.stdout + + +def _commit(repo: Path) -> dict: + head = _run(["git", "rev-parse", "HEAD"], repo).strip() + dirty = bool(_run(["git", "status", "--porcelain"], repo).strip()) + branch = _run(["git", "rev-parse", "--abbrev-ref", "HEAD"], repo).strip() + return {"path": str(repo), "commit": head, "branch": branch, "dirty": dirty} + + +def gpu_feature(system: str) -> str: + """The nexus3d cargo feature for this host's GPU API.""" + return "metal" if system == "Darwin" else "webgpu" + + +def install( + nexus: Path, rapier: Path | None, kiss3d: Path | None, python: Path, feature: str +) -> dict: + nexus = nexus.resolve() + if not (nexus / "crates" / "nexus_python3d" / "Cargo.toml").is_file(): + raise FileNotFoundError(f"{nexus} is not a nexus checkout") + sources = {"nexus": _commit(nexus)} + if rapier is not None: + rapier = rapier.resolve() + if not (rapier / "crates" / "rapier3d-urdf").is_dir(): + raise FileNotFoundError(f"{rapier} is not a rapier checkout") + sources["rapier"] = _commit(rapier) + if kiss3d is not None: + kiss3d = kiss3d.resolve() + if not (kiss3d / "src" / "window").is_dir(): + raise FileNotFoundError(f"{kiss3d} is not a kiss3d checkout") + sources["kiss3d"] = _commit(kiss3d) + _run( + [ + "maturin", + "build", + "--release", + "-m", + "crates/nexus_python3d/Cargo.toml", + "--features", + feature, + "-i", + str(python), + ], + nexus, + ) + wheels = sorted((nexus / "target" / "wheels").glob(WHEEL_GLOB), key=lambda p: p.stat().st_mtime) + if not wheels: + raise FileNotFoundError("maturin produced no wheel") + wheel = wheels[-1] + _run( + [ + "uv", + "pip", + "install", + "--python", + str(python), + "--reinstall-package", + "dimforge-nexus3d", + str(wheel), + ], + ROOT, + ) + version = _run( + [str(python), "-c", "import importlib.metadata as m; print(m.version('dimforge-nexus3d'))"], + ROOT, + ) + receipt = { + "schema_version": 1, + "wheel": wheel.name, + "feature": feature, + "version": version.strip(), + "platform": f"{platform.system().lower()}-{platform.machine()}", + "sources": sources, + } + RECEIPT.write_text(json.dumps(receipt, indent=2) + "\n") + return receipt + + +def verify(python: Path) -> dict: + read = read_receipt() + if not read["installed"]: + raise FileNotFoundError(read["problem"]) + receipt = read["receipt"] + version = _run( + [str(python), "-c", "import importlib.metadata as m; print(m.version('dimforge-nexus3d'))"], + ROOT, + ).strip() + if version != receipt["version"]: + raise ValueError( + f"installed nexus3d {version} differs from the receipt's {receipt['version']}" + ) + for name, source in receipt["sources"].items(): + repo = Path(source["path"]) + if repo.is_dir() and _commit(repo)["commit"] != source["commit"]: + raise ValueError( + f"{name} checkout moved off the receipt's commit {source['commit'][:12]}" + ) + return receipt + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + sub = parser.add_subparsers(dest="command", required=True) + inst = sub.add_parser("install", help="build the wheel and install it into the project venv") + inst.add_argument( + "--nexus", + type=Path, + default=None, + help="nexus checkout to build (default: the commit engine-runtime.json pins, " + "fetched from GitHub)", + ) + inst.add_argument( + "--rapier", type=Path, default=None, help="rapier checkout the nexus manifest patches to" + ) + inst.add_argument( + "--kiss3d", type=Path, default=None, help="kiss3d checkout the nexus manifest patches to" + ) + inst.add_argument("--python", type=Path, default=Path(sys.executable)) + inst.add_argument( + "--feature", default=gpu_feature(platform.system()), help="nexus3d GPU feature" + ) + ver = sub.add_parser("verify", help="check the installed wheel against the receipt") + ver.add_argument("--python", type=Path, default=Path(sys.executable)) + sub.add_parser("receipt", help="print the build provenance a run manifest records") + args = parser.parse_args(argv) + if args.command == "receipt": + read = read_receipt() + print(json.dumps({"ok": read["installed"], **read})) + if read["problem"]: + print(read["problem"], file=sys.stderr) + return 0 if read["installed"] else 1 + try: + if args.command == "install": + nexus, rapier, kiss3d = resolve_sources(args.nexus, args.rapier, args.kiss3d) + receipt = install(nexus, rapier, kiss3d, args.python, args.feature) + else: + receipt = verify(args.python) + except (RuntimeError, FileNotFoundError, ValueError) as exc: + print(json.dumps({"ok": False, "error": str(exc)})) + print(str(exc), file=sys.stderr) + return 1 + print(json.dumps({"ok": True, **receipt})) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/rapier_runtime.py b/tools/rapier_runtime.py new file mode 100644 index 00000000..141336b1 --- /dev/null +++ b/tools/rapier_runtime.py @@ -0,0 +1,199 @@ +#!/usr/bin/env python3 +"""Build and install AISLE's rapier Python module from a sibling checkout (ADR-68). + +The rapier engine is optional and outside the uv lock, like Nexus: this builds +the `rapier3d` wheel with maturin from a rapier checkout, installs it into the +project environment with `uv pip`, and writes a receipt naming the source +commit so a run's engine build is traceable. It renders through the Nexus +viewer, so `tools/nexus_runtime.py install` has to have run too; `verify` +says so rather than letting the first render fail. `read_receipt()` is the +public reader the run manifest and `env_hash.sim_engine_hash` use. CON-8: +JSON on stdout, logs on stderr, exit 0 iff ok. +""" + +from __future__ import annotations + +import argparse +import json +import os +import platform +import subprocess +import sys +from pathlib import Path + +from nexus_runtime import ROOT, _commit, _run + +RECEIPT = ROOT / ".rapier-runtime-receipt.json" +WHEEL_GLOB = "rapier3d-*.whl" +MANIFEST = "bindings/python/rapier-py-3d/Cargo.toml" + + +def receipt_path(root: Path | None = None) -> Path: + """Where the build receipt lives for a checkout (default: this one).""" + return (ROOT if root is None else Path(root)) / RECEIPT.name + + +def read_receipt(root: Path | None = None) -> dict: + """The ADR-68 engine build provenance, for a run manifest to carry. + + Same contract as `nexus_runtime.read_receipt`: never raises, and an + absent or malformed receipt is a reported fact. Returns {"installed", + "path", "receipt", "problem"}. + """ + path = receipt_path(root) + result: dict = {"installed": False, "path": str(path), "receipt": None, "problem": None} + if not path.is_file(): + result["problem"] = ( + f"no rapier build receipt at {path}; run `tools/rapier_runtime.py install`" + ) + return result + try: + receipt = json.loads(path.read_text()) + except (OSError, json.JSONDecodeError) as exc: + result["problem"] = f"unreadable rapier build receipt at {path}: {exc}" + return result + if not isinstance(receipt, dict): + result["problem"] = f"malformed rapier build receipt at {path}: expected a JSON object" + return result + return {"installed": True, "path": str(path), "receipt": receipt, "problem": None} + + +def resolve_rapier(rapier: Path | None) -> Path: + """The local checkout when given, otherwise the pinned GitHub source + (ADR-68). The bindings live in the same repository the Nexus build + patches, so both installers resolve to the same pinned commit.""" + if rapier is not None: + return rapier + sys.path.insert(0, str(Path(__file__).resolve().parent)) + from engine_sources import materialize + + report = materialize() + print(f"using the pinned engine sources in {report['sources_dir']}", file=sys.stderr) + return Path(report["sources"]["rapier"]["path"]) + + +def _maturin_env() -> dict: + """maturin refuses to run with both VIRTUAL_ENV and CONDA_PREFIX set + (rapier's own bindings/python/dev.sh unsets them for the same reason).""" + env = dict(os.environ) + env.pop("CONDA_PREFIX", None) + return env + + +def install(rapier: Path, python: Path, determinism: bool) -> dict: + rapier = rapier.resolve() + if not (rapier / MANIFEST).is_file(): + raise FileNotFoundError(f"{rapier} is not a rapier checkout with Python bindings") + sources = {"rapier": _commit(rapier)} + command = ["maturin", "build", "--release", "-m", MANIFEST, "-i", str(python)] + if determinism: + # libm transcendentals: cross-platform bit reproducibility (CON-5) + command += ["-F", "determinism"] + proc = subprocess.run(command, cwd=rapier, capture_output=True, text=True, env=_maturin_env()) + if proc.returncode != 0: + raise RuntimeError(f"{' '.join(command)} failed:\n{proc.stderr.strip()}") + wheels = sorted( + (rapier / "target" / "wheels").glob(WHEEL_GLOB), key=lambda p: p.stat().st_mtime + ) + if not wheels: + raise FileNotFoundError("maturin produced no wheel") + wheel = wheels[-1] + _run( + [ + "uv", + "pip", + "install", + "--python", + str(python), + "--reinstall-package", + "rapier3d", + str(wheel), + ], + ROOT, + ) + version = _run( + [str(python), "-c", "import importlib.metadata as m; print(m.version('rapier3d'))"], ROOT + ) + receipt = { + "schema_version": 1, + "wheel": wheel.name, + "determinism": determinism, + "version": version.strip(), + "platform": f"{platform.system().lower()}-{platform.machine()}", + "sources": sources, + } + RECEIPT.write_text(json.dumps(receipt, indent=2) + "\n") + return receipt + + +def verify(python: Path) -> dict: + read = read_receipt() + if not read["installed"]: + raise FileNotFoundError(read["problem"]) + receipt = read["receipt"] + version = _run( + [str(python), "-c", "import importlib.metadata as m; print(m.version('rapier3d'))"], ROOT + ).strip() + if version != receipt["version"]: + raise ValueError( + f"installed rapier3d {version} differs from the receipt's {receipt['version']}" + ) + for name, source in receipt["sources"].items(): + repo = Path(source["path"]) + if repo.is_dir() and _commit(repo)["commit"] != source["commit"]: + raise ValueError( + f"{name} checkout moved off the receipt's commit {source['commit'][:12]}" + ) + import nexus_runtime + + renderer = nexus_runtime.read_receipt() + if not renderer["installed"]: + raise FileNotFoundError( + "the rapier engine renders through the Nexus viewer (ADR-68) and " + f"nexus3d is not installed: {renderer['problem']}" + ) + return {**receipt, "renderer": renderer["receipt"]} + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + sub = parser.add_subparsers(dest="command", required=True) + inst = sub.add_parser("install", help="build the wheel and install it into the project venv") + inst.add_argument( + "--rapier", + type=Path, + default=None, + help="rapier checkout to build (default: the commit engine-runtime.json pins, " + "fetched from GitHub)", + ) + inst.add_argument("--python", type=Path, default=Path(sys.executable)) + inst.add_argument( + "--determinism", + action="store_true", + help="build with rapier's enhanced-determinism feature (libm transcendentals)", + ) + ver = sub.add_parser("verify", help="check the installed wheel against the receipt") + ver.add_argument("--python", type=Path, default=Path(sys.executable)) + sub.add_parser("receipt", help="print the build provenance a run manifest records") + args = parser.parse_args(argv) + if args.command == "receipt": + read = read_receipt() + print(json.dumps({"ok": read["installed"], **read})) + if read["problem"]: + print(read["problem"], file=sys.stderr) + return 0 if read["installed"] else 1 + try: + if args.command == "install": + receipt = install(resolve_rapier(args.rapier), args.python, args.determinism) + else: + receipt = verify(args.python) + except (RuntimeError, FileNotFoundError, ValueError) as exc: + print(json.dumps({"ok": False, "error": str(exc)})) + print(str(exc), file=sys.stderr) + return 1 + print(json.dumps({"ok": True, **receipt})) + return 0 + + +if __name__ == "__main__": + sys.exit(main())