Skip to content

calque real's docker-mode/Dockerfile-build path assumes docker is preinstalled on the AMI — fails on a non-GPU auto-selected AMI #195

Description

@scttfrdmn

Context

Found via a real calque real run against AI-Almanac's blending_app.py (--script .../blending_app.py --function inspect_netcdf_bundle --instance m6i.large --item-file ... --pip xarray --pip netCDF4). The picked unit's resolved image (blending_imagedebian_slim() + pip_install("uv") + .run_commands(git clone ...) + .pip_install("google-cloud-storage")) has real steps beyond a bare pullable ref, so internal/image.NeedsBuild correctly routed this to the on-instance Dockerfile-build path (calque#177).

Real bootstrap log tail:

+ sudo docker build -t calque-local:55979e8c0b7e67a8 -f /tmp/calque/Dockerfile /tmp/calque
sudo: docker: command not found

AMI used: ami-07a5b367e8dc8bd92 (spawn's auto-selected AMI for m6i.large — a non-GPU instance type). Instance was cleanly terminated by calque's own deferred cleanup; no leaked resources, no wasted spend beyond ~1 minute of m6i.large time.

Root cause

internal/exec/bootstrap.go's docker-mode branch (Command(), both the plain-pull and BuildDockerfile sub-paths) has NO docker-install step anywhere — confirmed by grep, zero command -v docker/apt-get install docker.io/dnf install docker lines exist in this file. It has always implicitly assumed docker is already present, which is true for the AWS Deep Learning AMIs spawn auto-selects for GPU instance types (g6/g6e/g7/g7e, calque#75) — but NOT true for the plain AL2023 AMI spawn auto-selects for a non-GPU type like m6i.large.

Every prior real-hardware verification in this codebase's history (calque#176/#177/#178's closing comments) used a GPU instance (g7e.2xlarge etc.) for exactly this reason — docker happened to already be there. This is the first real run to hit a non-GPU instance type with a resolved image needing docker, and it surfaced a gap that's been latent all along.

Ask

bootstrap.go's docker-mode Command() should install docker if it's absent, mirroring the existing idempotent command -v X >/dev/null || install X pattern already used throughout this file for aws/uv/git (see HostMode's own command -v uv >/dev/null || curl ... and --pip's git-URL handling for the exact style). Something like:

command -v docker >/dev/null || (sudo dnf install -y docker || sudo apt-get update && sudo apt-get install -y docker.io)
sudo systemctl enable --now docker 2>/dev/null || true

placed before the first docker invocation in both the plain-pull and BuildDockerfile sub-paths. Note spawn's OWN pkg/userdata/container.go (GenerateContainerUserData, spore-host#353) already does exactly this pattern (command -v docker >/dev/null 2>&1 || ... (dnf install -y docker || apt-get install -y docker.io) ... systemctl enable --now docker) — worth copying that exact idiom rather than inventing a new one, even though calque can't call that function directly today (calque#2's own tracked seam gap: Acquirer/Snipe can't reach launcher.Options.ContainerScript).

Also worth a docs note (docs/guide/cli-reference.md's --instance flag doc, or a new troubleshooting.md entry) that a --script run whose resolved image needs docker build/pull, combined with a non-GPU --instance, previously silently assumed a docker-preinstalled AMI — once fixed, this becomes a non-issue, but documenting the historical assumption helps anyone debugging an older calque version.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions