Skip to content

Regression: ARG declared between two stages (not before the very first FROM) breaks FROM resolution with "no FROM statement found" #7016

Description

@flypirat

Issue Description

An ARG instruction placed after an earlier stage's instructions but before a later stage's FROM — a pattern Docker/BuildKit documents and supports — silently stops being resolved. The later FROM ${ARG} then evaluates to an empty image reference, and the build fails with:

Error: determining starting point for build: no FROM statement found

This is a regression, not a pre-existing limitation: it reproduces starting at buildah v1.37.0 and is absent in v1.36.0 and earlier. Bisecting the vendored github.com/openshift/imagebuilder bump in that release points to a specific behavior change (root-caused below and cross-filed at openshift/imagebuilder).

Steps to reproduce the issue

Containerfile:

FROM scratch AS config
COPY somefile /build/

ARG UPSTREAM_IMAGE
FROM ${UPSTREAM_IMAGE} AS base

RUN echo "base stage ran"
$ buildah bud --build-arg UPSTREAM_IMAGE=busybox -f Containerfile -t repro .
Error: determining starting point for build: no FROM statement found

Moving the exact same ARG UPSTREAM_IMAGE line above the first FROM (true global/header scope) makes it build successfully, the value and default-handling are otherwise identical, only the line's position moved:

ARG UPSTREAM_IMAGE

FROM scratch AS config
COPY somefile /build/

FROM ${UPSTREAM_IMAGE} AS base

RUN echo "base stage ran"

The failure is independent of:

  • whether the ARG has a default value (ARG UPSTREAM_IMAGE=busybox fails identically)
  • whether --build-arg is passed at all (fails even parsing-only, before any value would matter)
  • whether the earlier stage is FROM scratch or a real image
  • --skip-unused-stages=false vs the default true
  • whether the earlier stage's output is actually consumed downstream (e.g. via RUN --mount=type=bind,from=config,...) or not
  • buildah bud vs podman build (podman vendors buildah and reproduces identically)
  • rootless vs rootful, storage driver (overlay locally, vfs in nested test containers), OCI runtime (checked because my primary repro environment is rootless buildah under WSL2). None of it matters, since the same failure reproduces on stock quay.io/buildah/stable container images with none of that host-specific context

Describe the results you received

Error: determining starting point for build: no FROM statement found

Describe the results you expected

base stage ran

buildah version output

Version:         1.39.3
Go Version:      go1.24.4
Image Spec:      1.1.0
Runtime Spec:    1.2.0
CNI Spec:        1.0.0
libcni Version:  
image Version:   5.34.2
Git Commit:      
Built:           Thu Jan  1 01:00:00 1970
OS/Arch:         linux/amd64
BuildPlatform:   linux/amd64

buildah info output

{
    "host": {
        "CgroupVersion": "v2",
        "Distribution": {
            "distribution": "debian",
            "version": "13"
        },
        "MemFree": 5581897728,
        "MemTotal": 8157081600,
        "OCIRuntime": "crun",
        "SwapFree": 2147483648,
        "SwapTotal": 2147483648,
        "arch": "amd64",
        "cpus": 12,
        "hostname": "hostname",
        "kernel": "6.6.87.2-microsoft-standard-WSL2",
        "os": "linux",
        "rootless": true,
        "uptime": "2h 27m 38.49s (Approximately 0.08 days)",
        "variant": ""
    },
    "store": {
        "ContainerStore": {
            "number": 4
        },
        "GraphDriverName": "overlay",
        "GraphOptions": null,
        "GraphRoot": "/home/hostname/.local/share/containers/storage",
        "GraphStatus": {
            "Backing Filesystem": "extfs",
            "Native Overlay Diff": "true",
            "Supports d_type": "true",
            "Supports shifting": "false",
            "Supports volatile": "true",
            "Using metacopy": "false"
        },
        "ImageStore": {
            "number": 80
        },
        "RunRoot": "/run/user/1000/containers"
    }
}

Provide your storage.conf

[storage]

# Default storage driver, must be set for proper operation.
driver = "overlay"
# Temporary storage location
runroot = "/run/containers/storage"
# Primary Read/Write location of container storage
graphroot = "/var/lib/containers/storage"

[storage.options]
# Storage options to be passed to underlying storage drivers

# AdditionalImageStores is used to pass paths to additional Read/Only image stores
# Must be comma separated list.
additionalimagestores = [
]

[storage.options.pull_options]

[storage.options.overlay]

# mountopt specifies comma separated list of extra mount options
mountopt = "nodev"

Upstream Latest Release

Yes

Additional environment details

  • WSL version: 2.6.3.0
  • Reproduced on buildah 1.39.3 (Debian 13 "trixie" package, 1.39.3+ds1-1+b7, rootless, kernel 6.6.87.2-microsoft-standard-WSL2, crun/overlay) and confirmed version-independent via quay.io/buildah/stable images for v1.37.0 through v1.45.0.
  • podman 5.4.2 (podman build) reproduces identically, as expected since it vendors buildah.
  • Note for anyone else debugging this: neither buildah version nor buildah info exposes the vendored openshift/imagebuilder version, so there's no quick way to check from the CLI whether a given buildah build is affected short of checking its release version against the table above (or grepping the binary/build info for module versions).

Additional information

Bisection

Tested via quay.io/buildah/stable:<tag>, --storage-driver=vfs, same Containerfile/build-arg each time:

Version Result
v1.33 works (this is what our GitHub Actions' ubuntu-latest runner ships)
v1.36.0 works
v1.37.0 fails
v1.39.3 fails
v1.45.0 (via the rhcontainerbot/podman-next Fedora COPR build 1.45.0-1.20260731201829300100.main.6.g4206064af, self-reporting 1.46.0-dev) still fails

Possible cause

go.mod at v1.36.0 pins github.com/openshift/imagebuilder v1.2.9; at v1.37.0 it's bumped to v1.2.14 (commit 7f1fe46, "bump github.com/openshift/imagebuilder to v1.2.14"). That range includes openshift/imagebuilder@dc7bc4f ("NewStages(): only allow header args", first released in imagebuilder v1.2.12):

 func NewStages(node *parser.Node, b *Builder) (Stages, error) {
 	var stages Stages
-	var allDeclaredArgs []string
-	for _, root := range SplitBy(node, command.Arg) {
-		argNode := root.Children[0]
-		if argNode.Value == command.Arg {
-			// extract declared variable
-			s := strings.SplitN(argNode.Original, " ", 2)
-			if len(s) == 2 && (strings.ToLower(s[0]) == command.Arg) {
-				allDeclaredArgs = append(allDeclaredArgs, s[1])
-			}
-		}
-	}
+	var headingArgs []string
 	if err := b.extractHeadingArgsFromNode(node); err != nil {
 		return stages, err
 	}
+	for k := range b.HeadingArgs {
+		headingArgs = append(headingArgs, k)
+	}
 	for i, root := range SplitBy(node, command.From) {
 		...
 		stages = append(stages, Stage{
 			...
-			Builder:  b.builderForStage(allDeclaredArgs),
+			Builder:  b.builderForStage(headingArgs),
 			...

extractHeadingArgsFromNode collects ARG nodes only until it hits the first FROM, then stops for good:

for _, child := range node.Children {
    if extract && child.Value == command.Arg {
        args = append(args, child)
    } else {
        if child.Value == command.From { extract = false }
        children = append(children, child)
    }
}

So an ARG declared between stage N and stage N+1's FROM is no longer part of the set of args considered "allowed" when resolving stage N+1 — previously every ARG anywhere in the file was collected (allDeclaredArgs, via SplitBy(node, command.Arg)) and offered to every stage, which is presumably what the commit intended to narrow (to stop genuinely stage-local ARGs from leaking into unrelated stages). The side effect is that it also excludes the narrower, still-valid pattern of "declare an ARG once, use it in the very next FROM," which Docker's own docs describe as supported:

To use the default value of an ARG declared before the first FROM, use an ARG instruction without a value inside a build stage. ... [an ARG] declared before a FROM ... can be used in any FROM instruction in the build.

(from Docker's "Understand how ARG and FROM interact" documentation)

Impact

This silently breaks any multi-stage Containerfile that parameterizes a later stage's base image via an ARG declared just above it (a very common idiom for keeping the ARG declaration next to the FROM it controls, rather than hoisting every such ARG to the top of the file). There's no warning or deprecation notice, the build just fails with a message ("no FROM statement found") that doesn't point at ARG placement at all, making it hard to diagnose without bisecting versions (see above).

Suggested fix / discussion

Either:

  1. In NewStages(), continue considering an ARG declared anywhere before the FROM that references it (not just before the very first FROM) as "allowed" for that FROM's resolution (restoring compatibility with the documented Docker/BuildKit behavior) while still not leaking ARGs declared after a FROM (inside a stage body) into later, unrelated stages.
  2. If the narrower "header-only" scope is intentional going forward, document it as a breaking change in the changelog/release notes for v1.37.0, and ideally have the parser emit a clear diagnostic (e.g. "ARG declared inside a stage body is not visible to later FROM statements; move it above the first FROM") instead of the generic "no FROM statement found."

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions