Skip to content

HostMode real runs (real/fleetrun/spawn-run) never install modal itself — only dry-run's uvPythonArgv does #200

Description

@scttfrdmn

Context

Found via a real calque spawn-run run against AI-Almanac's forecasts_app.py (after calque#198's sibling-function fix landed and correctly shipped run_season_forecast_bundle's real extras). The run failed with:

LEAK[integration_edge] runner warm-up failed (restart 1): @enter failed: No module named 'modal'
...
warmd error: warm-up failed after 6 restarts: @enter failed: No module named 'modal'

Confirmed via the real manifest: extra_imports correctly includes {"name": "modal", "source": "import modal"} (calque#146's free-reference resolution working exactly as designed — one of the shipped extras, e.g. _season_bundle_impl, bare-references modal — likely via a .Volume/.Secret call inside a sibling helper). The import statement gets shipped and exec'd, but the modal PACKAGE was never installed into the instance's Python environment at all.

Root cause

cmd/calque/run.go's uvPythonArgv (dry-run's mechanism) has an explicit, deliberate design decision:

"modal" is ALWAYS included: a real script's body routinely references modal.Secret/modal.Volume/etc. directly even though Modal's own SDK is never itself a pip_install(...) entry in the script's OWN .image chain (confirmed live — this is exactly the first failure a real AI-Almanac script's dry-run hit).

internal/exec/bootstrap.go's HostMode branch (used by calque real/fleetrun/spawn-run for a picked unit with no resolvable docker image) has NO equivalent guarantee. Its uv pip install step is entirely gated behind if len(b.PipPackages) > 0 — with --pip unset (the common case, and specifically spawn-run's case today since it has no --pip flag at all), zero packages are installed, not even modal itself. Confirmed by reading the code directly: the uv venv/uv python install lines always run, but the uv pip install line is entirely skipped when PipPackages is empty — there's no separate, unconditional modal install the way dry-run's uvPythonArgv has.

Why this matters

This is NOT a narrow case — per uvPythonArgv's own comment, referencing modal.Secret/modal.Volume/etc. from a real script's body is common enough that dry-run was specifically hardened against it. Every HostMode real run (real/fleetrun/spawn-run) with a body that bare-references modal anywhere (directly, or transitively through a shipped sibling function/extra) will hit this identical NameError-equivalent failure. spawn-run is doubly exposed since it has no --pip flag at all (a separate gap) to work around this manually.

Ask

internal/exec/bootstrap.go's HostMode branch should unconditionally ensure modal is installed into the uv venv, mirroring uvPythonArgv's own explicit design decision — e.g. always include modal in the packages passed to uv pip install, the same way uvPythonArgv always adds it to its --with list, rather than gating entirely on len(b.PipPackages) > 0. Concretely: change the guard so the uv pip install line ALWAYS runs with at least modal in the package list, appending any caller-supplied --pip packages on top (deduped), instead of skipping the whole block when PipPackages is empty.

Real repro: any HostMode script whose picked unit (or a shipped sibling/extra) bare-references modal.Anything with no --pip modal passed explicitly. Worth a new fixture exercising this specifically — testdata/scripts/free_refs_*.py may already be close but should be checked for whether any existing HostMode integration test actually verifies modal importability, or only tests dry-run's uvPythonArgv path.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions