feat(inspect-ai): add VM-mode Capsem SandboxEnvironment integration

[tholop]

Summary

Adds the Inspect AI sandbox provider (@sandboxenv(name="capsem") in integrations/inspect-ai, distribution inspect-capsem-sandbox, module inspect_capsem) on top of the 0.7 Python SDK in VM mode, superseding #291 (tracking issue #310, handoff issue #342). Each Inspect sample runs in a fresh ephemeral Capsem micro-VM with bounded file transfers and label-scoped cleanup; OCI container execution (#339) and opt-in host Dockerfile builds (#340) stack on top. Stacked on #337.

Changes

• Package & VM lifecycle (integrations/inspect-ai/inspect_capsem/):
  • Splits the provider into focused modules under 300 lines at 100 columns (__init__.py, config.py, sandbox.py, _lifecycle.py, _cleanup.py, _controller.py, _exec.py, _files.py, _transfer.py, _tools.py, _registry.py) plus the committed integrations/inspect-ai/uv.lock.
  • Creates one fresh ephemeral VM per sample (persistent=False, labeled managed-by=inspect-capsem plus optional inspect-capsem-prefix / inspect-capsem-task and a per-create inspect-capsem-nonce), and scopes task_cleanup and cli_cleanup (inspect sandbox cleanup capsem) strictly to matching persistent=False labeled VMs.
  • Uses the 0.7 SDK surface from feat(service,sdk): attach labels to VMs at create and expose them in list #336 and feat(sdk,service): add typed ErrorCode enum, timeout/error helpers, and discovery #337 (Hypervisor.connect, discover_gateway, VmLifecycleState, and typed VmNotFoundError / CreateTimeoutError / ExecTimeoutError, cleaning up half-created VMs on HTTP 504 CreateTimeoutError via CreateTimeoutError.vm_id or the per-create nonce label).
  • Exposes connection() as SandboxConnection(type="capsem", command="capsem shell <vm_id>").
• Bounded memory & file transfers (_files.py, _transfer.py, _exec.py):
  • Enforces SandboxEnvironmentLimits.MAX_EXEC_OUTPUT_SIZE (10 MiB per stream) and SandboxEnvironmentLimits.MAX_READ_FILE_SIZE.
  • Requires regular files ([ -f ]) on reads, derives staged transfer chunk sizes from MAX_REQUEST_BODY_BYTES, and caps guest reads at limit + 1 bytes while streaming (head -c + per-part accounting in _transfer._staged_download).
  • Routes paths that capsem-service stores verbatim (_DIRECT_REL_RE) through the direct Files API and falls back to staged exec (shlex.quote) for paths containing spaces or special characters.
• Gate & installed-package qualification (config/gate.toml, build_system/, tests/ironbank/test_sdk_live.py):
  • Replaces 0.7's interim inspect-extraction ownership with standalone [integrations_inspect_ai] package ownership and [[surface]] id = "integrations" under lint, typecheck, 300-line limit, and coverage enforcement.
  • Wires fast.integrations.inspect-ai.{lint,types,tests,build} and integrations.inspect-ai.prewarm into capsem-gate (ONLINE_FAST), CI scope routing, offline wheel + sdist entry-point verification (integrations/inspect-ai/tests/image_package_acceptance.py), a portable macOS timeout test shim (integrations/inspect-ai/tests/conftest.py), and the live VM gate lane (tests/ironbank/test_sdk_live.py running integrations/inspect-ai/tests/live_acceptance.py).
  • Namespaces pytest --basetemp per CAPSEM_TEST_RUN_ID in integrations/inspect-ai/tests/conftest.py and sdk/python/tests/conftest.py so parallel gate pytest steps cannot collide with fast.citadel's basetemp directory.

Mapping to Elie's #291 Review

Every item from the #291 review, and where it is resolved in the stack:

#	#291 item	Where	Notes
B1	300-line limit (integrations/ not in [boundary.scripts].roots; 6 files over)	PR 2b (#338) + PR 3 (#339)	integrations/inspect-ai added to roots; VM-mode (_cleanup.py, _lifecycle.py, etc.) and container modules split by responsibility and stay under 300 lines.
S1	Host memory bound on downloads (_staged_download no byte cap, /dev/zero, _save_built_image_to_cache)	PR 2b (#338) + PR 3 (#339)	_files.read_guest_file requires a regular file ([ ! -f ] -> NOT_REGULAR), _staged_download runs [ -f ] && head -c <limit+1> | split and stops at limit + 1 while joining parts. The in-guest image cache that owned _save_built_image_to_cache is deleted with the 0.6 dockerd backend in PR 3.
S2	Ephemeral by default (named sample VMs were persistent; "add an SDK/service way to label or find ephemeral VMs. Happy to do that side.")	PR 1 (#336) + PR 2b (#338)	We built the SDK/service side ourselves in PR 1 rather than taking Elie's offer; PR 2b creates unnamed persistent=False VMs with managed-by=inspect-capsem.
S3	Cleanup scope ("require the inspect-capsem- prefix")	PR 2b (#338)	Deliberate adaptation: sample VMs are persistent=False (vm-N display names, UUID IDs), so scoping is by persistent=False + label (managed-by=inspect-capsem and matching inspect-capsem-prefix when CAPSEM_VM_PREFIX is set); inspect sandbox cleanup capsem <id> on any persistent or non-matching VM logs a warning and leaves it alone.
S4	Host environment and paths (Compose interpolation, bare environment: [KEY], bind volumes)	PR 3 (#339)	Compose parsing is reconciled onto 0.7's carry in PR 3: default-deny host environment interpolation (SAMPLE_METADATA_* allowlist with .env spoofing rejection) and os.path.realpath project-root containment of bind sources.
S5	SDK floor (capsem>=0.6.3 while using newer APIs)	PR 2b (#338)	integrations/inspect-ai/pyproject.toml declares capsem>=0.7.0, matching sdk/python/pyproject.toml.
S6	Move shared logic into the SDK (gateway discovery, copied sanitize_file_path, error-text parsing)	PR 2a (#337) + PR 2b (#338)	PR 2a adds discover_gateway / Hypervisor.connect, structured ErrorResponse.code + typed exceptions; PR 2b consumes them. #291's character-stripping _sanitize_file_path helper is replaced by a direct-Files-API eligibility check (_DIRECT_REL_RE in _transfer.py) that routes only paths capsem-service stores verbatim through the direct Files API and falls back to staged exec (shlex.quote) for paths containing spaces or special characters (because crates/capsem-service/src/fs_utils.rs sanitize_file_path currently strips characters outside [A-Za-z0-9._\-/] with HTTP 200 rather than returning HTTP 400). No re.search over error text in inspect_capsem/.
S7	A VM-backed test in the gate	PR 2b (#338) + PR 3 (#339)	tests/ironbank/test_sdk_live.py installs built wheel + sdist offline into clean prefixes outside the repo and runs integrations/inspect-ai/tests/live_acceptance.py in the VM lane without opt-in flags.
N1	Nit: drop X as X re-exports in buildschema.py	Superseded upstream	The gate refactor now lives on 0.7 (kept under Pierre's authorship); test_qualification_schema_reexports_keep_one_model_identity on 0.7 asserts buildschema.X is qualifyschema.X, i.e. the re-exports are now pinned by upstream's own test.
N2	Nit: gate refactor commit also adds SourcePackageConfig, only the next commit uses it	Superseded upstream	0.7 already carries SourcePackageConfig and tests it (_source_package_type). PR 2b is the first consumer ([integrations_inspect_ai] in config/gate.toml).
N3	Nit: pass the settings explicitly in sdkchecks.py instead of isinstance / fallback branches	PR 2b (#338)	sdkchecks.py takes settings: SourcePackageConfig explicitly; no isinstance fallback remains.
N4	Nit: avoid assert in production code (sandbox.py)	PR 2b (#338)	No assert statements in inspect_capsem/.
SC	Scope suggestion: 3 PRs (gate refactor / VM-mode sandbox / container mode)	Whole stack	Gate refactor -> landed on 0.7; VM-mode sandbox -> PR 2b (#338); OCI container mode -> PR 3 (#339); SDK/service prerequisites -> PR #341 / PR 1 (#336) / PR 2a (#337); opt-in host Dockerfile / Compose build: (HostBuildGrant) -> PR 4 (#340), stacked on #339 for #342 step 4.
—	ModulesConfig.transition: TransitionSettings conflict in qualifyschema.py	n/a on 0.7	Present at qualifyschema.py:70 on 0.7@upstream, so there is no conflict to resolve.

Qualification (#342)

• Static & unit/package gate (integrations/inspect-ai/tests, linux/x86_64, Python 3.12.14): ruff check, ruff format --check, ty check --error-on-warning --python-platform all, and pytest integrations/inspect-ai/tests -q (68 passed on #338 VM-only), plus built capsem-0.7.0 and inspect_capsem_sandbox-0.1.0 wheel and sdist archives.
• Hermetic installed-package live VM acceptance (tests/ironbank/test_sdk_live.py::test_inspect_ai_live_vm_sandbox_acceptance): passed across both offline-installed wheel and sdist consumers (python -I), emitting SDK_IMAGE_PACKAGE_ACCEPTANCE_OK and INSPECT_CAPSEM_VM_ACCEPTANCE_OK, with 43/43 Inspect self_check pass (test_read_and_write_large_file_binary skipped), live eval_async VM task (accuracy=1.000, 4s), verified history(layer=EXEC) + session.db exec_events (target="vm"), foreign persistent=True VM survival, orphan ephemeral sweep, and 0 leaked managed VMs.

Stack Overview (Supersedes #291, Rebased on 0.7)

1. PR refactor(api): share VM name validation across service and gateway #341 (feat/shared-vm-name-rule) — shared VM-name validation rule across capsem-api and capsem-service
2. PR 1 / feat(service,sdk): attach labels to VMs at create and expose them in list #336 (feat/sdk-vm-labels) — VM labels on create, fork, and list
3. PR 2a / feat(sdk,service): add typed ErrorCode enum, timeout/error helpers, and discovery #337 (feat/sdk-typed-helpers) — structured ErrorCode enum, timeout/lookup exceptions, and gateway discovery
4. PR 2b / feat(inspect-ai): add VM-mode Capsem SandboxEnvironment integration #338 (feat/inspect-capsem) (this PR) — VM-mode inspect-capsem SandboxEnvironment integration
5. PR 3 / feat(inspect-ai): add OCI container execution mode and Compose parser #339 (feat/inspect-capsem-containers) — OCI container execution mode and reconciled 0.7 Compose parser
6. PR 4 / feat(inspect-ai): add host-side Dockerfile and Compose build image support #340 (feat/inspect-capsem-host-build) — default-off, operator-granted host docker build (HostBuildGrant) and hermetic live gate acceptance (Integrate and qualify Inspect AI on current 0.7 (Pierre handoff) #342 step 4)

[codecov-commenter]

❌ 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
6052	1	6051	0

View the top 3 failed test(s) by shortest run time

capsem-assets::oci::worker::tests::inventory_worker_publishes_only_quiet_bounded_observations_and_tracks_nested_changes

Stack Traces | 2.49s run time

thread 'oci::worker::tests::inventory_worker_publishes_only_quiet_bounded_observations_and_tracks_nested_changes' (139004) panicked at .../oci/worker/tests.rs:53:6:
called `Result::unwrap()` on an `Err` value: TimedOut { label: "changed-inventory", attempts: 8, timeout: 2s }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

capsem-assets::oci::worker::tests::incompatible_receipts_are_observed_once_until_their_metadata_changes

Stack Traces | 2.67s run time

thread 'oci::worker::tests::incompatible_receipts_are_observed_once_until_their_metadata_changes' (138933) panicked at .../oci/worker/tests.rs:221:10:
called `Result::unwrap()` on an `Err` value: TimedOut { label: "foreign-cache-observed", attempts: 8, timeout: 2s }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

capsem-core::fs_monitor::tests::a_workspace_swapped_for_a_host_link_is_never_walked_or_read

Stack Traces | 360s run time

No failure message available

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}