Skip to content

Dev experience: one-command VM/e2e test runner + setup doctor; nextest on macOS #161

Description

@cspinetta

Context

The VM-test honesty redesign (PR #157, ADR-0011) made the VM/e2e suites auto-provision their images and skip only on genuine incapability. But running them locally is still per-suite and platform-specific: a developer hand-runs cargo test --test <suite> -- --ignored --test-threads=1 and has to know which suites can run on their platform, remember VOID_BOX_REQUIRE_VM=1, and have the right toolchain staged. This tracks improving the local dev loop on macOS (VZ) and Linux (KVM).

Deliverable 1 — one-command VM/e2e runner (first priority, small)

A single entry point — cargo xtask vm-test is preferred over a Makefile (in-repo, cross-platform, cargo-native, no make dependency) — that:

  • Detects the platform: macOS → VZ, Linux → KVM.
  • Runs exactly the suites that can run there. The cfg(target_os = "linux") suites (e2e_telemetry, e2e_skill_pipeline, oci_integration, and the KVM-only kvm_integration / snapshot_integration / persistent_channel) are skipped on macOS automatically; the VZ suites (conformance, e2e_mount, snapshot_vz_integration, e2e_credential_proxy, …) run there.
  • Sets VOID_BOX_REQUIRE_VM=1 and picks the right runner: per-suite cargo test … --ignored --test-threads=1 on macOS; the nextest vm-group lane on Linux (see .config/nextest.toml, .github/workflows/e2e.yml).
  • Prints a clean per-suite pass / fail / skip summary.
  • Supports --list to show which suites run on the current platform without running them, and --suite <name> to run one.

Deliverable 2 — setup doctor / pre-flight (small, fold into #1 as a pre-check)

cargo xtask vm-doctor that verifies the real requirements and prints what is missing plus the fix:

  • musl-linux target + a cross-linker (the macOS guest-image build cross-compiles guest-agent; today a missing toolchain fails mid-build cryptically).
  • /dev/kvm and /dev/vhost-vsock access (Linux).
  • The VZ entitlement / codesign path (macOS).
  • A static Linux busybox (needed by e2e_mount's shell commands).
  • cargo-nextest installed (Linux lane).

Deliverable 3 — make nextest work on macOS (follow-up investigation, medium/uncertain)

The blocker: .cargo/config.toml routes every test binary through the codesigning target-runner scripts/run_voidbox_macos.sh, and under nextest a non-VM binary (daemon_unix_perms) was SIGKILL'd during nextest's --list phase. Likely direction: codesign the test binaries once as a build step so the runner becomes a passthrough (or bypass the runner for nextest), letting the vm group's bounded parallelism work on macOS too. The per-suite cargo test path already works, so this is a nice-to-have, not a blocker.

Notes

  • Keep cargo test and the existing CI lanes working unchanged; the xtask is additive.
  • The platform suite matrix the runner encodes should stay the single source of truth — consider having --list print it so AGENTS.md can point at the command instead of duplicating the table.

Related: PR #157 (honesty redesign), ADR-0011, follow-ups #158 / #159.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:testsFix lands in test code, not shipped behaviorenhancementNew feature or requestmacosmacOS/VZ-specific

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions