feat(inspect-ai): add inspect-capsem SandboxEnvironment integration

[tholop]

This PR creates a Python package that registers Capsem as a sandbox provider for Inspect AI, similar to the third-party sandbox plugins at https://inspect.aisi.org.uk/extensions-sandboxes.html. I've been testing the plugin locally on various Inspect tasks. A local package works fine but we can also register an official extension in the future with a small PR to Inspect-AI.

Also, a big part of the PR is about handling Docker compose setups required by many Inspect AI tasks. As far as I understand the SDK doesn't provide this functionality, but it sounds useful so I kept the code self-contained in case we want to migrate it later. I didn't implement multi-machine setups yet since I believe there are further networking changes required, but happy to open a follow up PR once this is clear.

AI summary next:

---

Inspect AI sandbox provider registered as capsem through the inspect_ai entry point. Once it's installed, inspect eval <task> --sandbox capsem runs a task's tool calls in a Capsem VM, with no changes to the task. It only uses the Python gateway SDK.

Prerequisites

• fix(sdk): stream large binary request bodies #286 (stream large binary request bodies): file reads and writes, including the multi-MiB inspect-sandbox-tools bundle, go through the Files API in one request.
• fix(sdk): give create the service's container-ready deadline #287 (give create the service's container-ready deadline): cold container pulls no longer hit the client's default timeout.
• feat(sdk): expose Hypervisor.vm for existing VMs by name or id #290 (Hypervisor.vm): handles for existing VMs by id, on the hypervisor's transport.
• SDK pin: pyproject.toml declares capsem>=0.6.3, but the plugin also needs SDK APIs that are on main and not yet released (OCI create fields, EXEC_TIMEOUT_CEILING_SECS, GATEWAY_REQUEST_BUDGET_SECS, decode_exec_output), plus the three PRs above. The plugin isn't meant to be pip-installable yet; in the repo it resolves the SDK from sdk/python. Could the SDK fixes ship in the next pip release? I'll then bump the floor in a one-line follow-up.

What it does

• vm mode (default): exec and file I/O run directly in a VM created from a profile template (default code). The gateway is found like the CLI does: CAPSEM_GATEWAY_URL/CAPSEM_GATEWAY_TOKEN, else CAPSEM_RUN_DIR, CAPSEM_HOME/run or ~/.capsem/run.
• container mode: a task that ships an image, a Dockerfile or a single-service compose.yaml gets that container inside the VM.
  • A plain prebuilt image boots directly through Hypervisor.create(image=...).
  • Anything else (a Dockerfile, volumes, ports, user, healthcheck, and so on) runs under the guest's Docker. Built images are cached on the host by content.
  • Compose and .env parsing follow Docker Compose 2.40. Files without services:, multi-service files, depends_on and env_file raise ValueError; other unknown service keys log a warning.
  • The container code lives in inspect_capsem/containers/ and doesn't import inspect_ai (a test enforces this), so it could later move into the SDK or be replaced by native service support.
• Lifecycle:
  • The provider names the VMs it creates inspect-capsem-* and deletes them in sample_cleanup, task_cleanup, cli_cleanup (inspect sandbox cleanup capsem, which only touches inspect-capsem-* VMs, or the id you pass) and at exit. This includes VMs whose create hit the service's readiness deadline (HTTP 504). If a stop fails, the VM stays tracked and later cleanup retries it.
  • --no-sandbox-cleanup prints the ids of surviving VMs.
• Guest setup: extracts Inspect's inspect-sandbox-tools in the guest and installs the Capsem MITM CA so that TLS clients and Chromium (NSS) trust it.
• Docs: a short package README, plus a user page at web/docs/src/content/docs/usage/inspect-ai.md (modes, config, env vars, how the image cache and CA trust work). The README links the in-repo file because the docs site is still a holding build. That link adds one record to Citadel's web_benchmark_boundary_debt.toml (327 -> 328): its docs/ pattern also matches the new web/docs/ root.

Commits

1. refactor(gate): moves the qualification and audit schema models from buildschema.py to qualifyschema.py, keeping re-exports, so the module stays under the 300-line ceiling. It also adds SourcePackageConfig for hand-written packages that have no specification or codegen check.
2. feat(inspect-ai): the package, its tests, README, docs page and CHANGELOG entry, plus its gate lane (Ruff, ty, pytest with a 90% branch-coverage floor, wheel/sdist build) in config/gate.toml, the fast-gate and CI workflows, codecov.yml, and Citadel ownership.

Validation

• integrations/inspect-ai: pytest -W error -> 94 passed, 2 skipped (the skipped tests are the live checks), 95.3% branch coverage; ruff check, ruff format --check and ty check are clean; uv lock --check passes.

• Live, with CAPSEM_LIVE_SELF_CHECK=1 against a running service, in vm and container modes: the full suite (96 tests, including Inspect's self_check) passes under -W error, and no VMs are left behind.

• Repository Citadel and gate suite at each commit: 5,694 passed / 46 skipped (commit 1), 5,701 passed / 46 skipped (commit 2).

• Unmodified inspect_evals / inspect_harbor tasks with --sandbox capsem and google/gemini-3.5-flash, run one at a time on one host. Every sample completed with 0 sample errors and no VMs were left behind; scores reflect the model, not the sandbox:

  Task	Samples	Score	Wall time
  vimgolf_single_turn --limit 5 (Dockerfile)	5/5	1/5	3m34s
  gdm_intercode_ctf --limit 5 (Compose + Dockerfile, uncached builds)	5/5	5/5	24m32s
  swe_bench_verified_mini, 3 samples, --message-limit 100	3/3	1/3	6m10s
  terminal_bench_2 (harbor), 3 samples	3/3	0/3	11m32s

  A two-service Compose task fails at sandbox init with the multi-service ValueError and creates no VMs.

[ebursztein]

Thanks a lot for this, Pierre. It's a really useful integration, and the core design is sound: everything runs inside the Capsem VM via exec or the guest's own Docker, so we found no sandbox escape. The workflow changes are clean too (no new triggers, pinned actions, only the existing CODECOV_TOKEN). Citadel, the package tests (94 passed, 95% branch coverage), ruff, ty and the OSV scan of uv.lock are all green.

We've merged your three SDK PRs (#286, #287, #290) into our integration batch with your authorship. For this one, here's what we'd like changed before it lands.

Blocking

• 300-line ceiling. [boundary.scripts].roots (config/gate.toml) lists sdk/python but not integrations/, so the package isn't covered. Six source files are over the limit: sandbox.py (869), dockerfile.py (711), compose.py (592), runtime.py (525), image_cache.py (487) and _controller.py (380). Please add integrations/inspect-ai to the roots and split those files by responsibility.

Should fix

• Host memory bound on downloads. read_file checks the size with stat in the guest, then _staged_download joins every part with no byte cap. A guest can grow the file after the check, or point at /dev/zero (size 0, not a directory). Please count bytes while streaming, stop at limit + 1, and require [ -f ]. _save_built_image_to_cache reads the whole archive before checking its size, which has the same issue.
• Ephemeral by default. Naming each sample VM makes it persistent (_controller.py), which breaks Capsem's ephemeral-by-default rule and leaves workspaces in ~/.capsem/run/persistent after a crash. We'd rather add an SDK/service way to label or find ephemeral VMs. Happy to do that side.
• Cleanup scope. inspect sandbox cleanup capsem <id> deletes any matching VM, including a user's own named VMs. Please require the inspect-capsem- prefix.
• Host environment and paths. Compose interpolation, bare environment: [KEY] entries and bind volumes read host env vars and host paths into the guest. That matches Docker Compose, but it lets a task file carry host credentials past Capsem's credential broker. Please make this opt-in (an allowlist), or at minimum document it prominently.
• SDK floor. pyproject.toml declares capsem>=0.6.3, but the code uses APIs that are newer than 0.6.5. Please raise it to the first release that has them.
• Move shared logic into the SDK. Gateway URL/token discovery, the copied sanitize_file_path regex, and parsing error text for VM ids and timeouts should be typed SDK APIs. We can add them.
• A VM-backed test in the gate. test_live.py only runs with CAPSEM_LIVE_SELF_CHECK. One acceptance test in the VM lane would make this really safe to depend on.

Nits

• Drop the X as X re-exports in buildschema.py and import from qualifyschema directly.
• 8a146e35 also adds SourcePackageConfig, which only the next commit uses.
• Pass the settings explicitly in sdkchecks.py instead of the isinstance and fallback branches.
• Avoid assert in production code (sandbox.py).

Scope suggestion. It would review and land faster as three PRs:

1. The gate refactor.
2. The VM-mode sandbox (controller, sandbox, tools).
3. Container mode (compose, dockerfile, image cache, CA trust, runtime; about 2.4k lines).

There's one conflict with current work: ModulesConfig gained a transition: TransitionSettings field, so it needs adding where you moved the class (qualifyschema.py).

Thanks again. This is great to have, and we're glad to help with the SDK pieces.

[tholop]

Thanks for the feedback, I split this PR into 5 PRs: #336 to #340. On the way I found an unrelated fix #341.

More details about the split in #338.

[tholop]

Closing in favor of the rebased 0.7 PR stack tracked in #342: prerequisites #341 (feat/shared-vm-name-rule), #336 (feat/sdk-vm-labels), and #337 (feat/sdk-typed-helpers), followed by Draft PRs #338 (feat/inspect-capsem), #339 (feat/inspect-capsem-containers), and #340 (feat/inspect-capsem-host-build).