ADD from a URL serves a stale cached layer when the remote content changes

[mzihlmann]

Problem

ADD <url> keys its layer off the instruction text alone, so a change to the remote content does not move the cache key and kaniko serves the previously cached layer. The build silently produces an image whose file contents do not match what the URL currently returns.

addCmdFilesUsedFromContext (pkg/commands/add.go) drops remote sources before the file list is built:

for _, src := range srcs {
    if util.IsSrcRemoteFileURL(src) {
        continue
    }

so nothing content-derived reaches compositeKey.AddPath. The skip happens regardless of --checksum. With a checksum the key is correct by side effect, because the digest is part of the instruction text that String() returns, so editing the Dockerfile busts the key.

docker does not behave this way. It re-fetches and folds the fetched content into the key with no --checksum involved.

Reproducer

A local http server serving payload.txt, and:

FROM scratch
ADD http://web:8080/payload.txt /dst/

kaniko, --cache --cache-copy-layers:

remote content	cache key	result
AAAA, cold	238cbc1b...	no cached layer
AAAA, unchanged	238cbc1b...	found cached layer
CCCC, changed	238cbc1b...	found cached layer

The third build serves the layer holding AAAA. Adding remote URL is logged only on the cold run, so on a hit the URL is not even fetched.

docker 29.7.2 with buildx 0.36.1, pruned builder cache, same Dockerfile and server:

remote content	build step
AAAA, cold	DONE, ran
AAAA, unchanged	CACHED
CCCC, changed	DONE, ran again

Scope

The server was busybox httpd over plain http, which sends Last-Modified and no ETag, so this shows that docker invalidates but not which header or digest it keys on. Untested whether docker revalidates when the mtime moves and the bytes do not.

Related: mz883 covers the same FilesUsedFromContext path being walked twice per build.