Conversation
Merging release v2.5.0
Outlines four phases of work with actionable checklists: - Phase 0: groundwork - route logging to stderr and extract the shared tag-resolution rule so CopyExistingImageTag and TagAndPushNewImages stop duplicating it - Phase 1: fix CheckManifestHead treating every registry error as "image does not exist", which silently forces a full rebuild and push - Phase 2: machine-readable output via --output-format=json and $GITHUB_OUTPUT so pipelines can consume the hash, cache-hit and tags - Phase 3: respect .dockerignore in both the hash and the build context tar, which are currently honoured in neither - Phase 4: BuildKit and multi-platform builds via docker buildx, replacing the deprecated classic daemon builder Also records the deferred backlog, notably --watch-base-image.
Describes how the four phases should actually be landed: nine stacked feature branches across two releases, rather than one large change set. Key sequencing decisions recorded: - The two Phase 0 refactors go first because they touch nearly every file in utils/, so landing them alone keeps every later diff clean - Rungs 2 and 3 are the only pair that can run in parallel - v2.6.0 ships the output contract before the cache reset, so pipelines can adopt cache-hit and primary-tag while their caches are still warm - .dockerignore splits across two rungs: the capability lands opt-in and non-breaking, then a three-line rung flips the default and resets the cache, so the breaking change is reviewable on its own - buildx credential handling is split from the build path because writing a config.json with a password to disk deserves its own review - The platform list and the buildx build path must land together, or an intermediate state publishes multi-arch images under a hash that ignores the platform list Also records rebase-not-merge discipline for the stack.
Add LogInfo/LogWarn/LogError in cli/utils/log.go and mechanically replace every fmt.Printf/fmt.Print/fmt.Println in cli/utils/*.go with them. All human-readable output now goes to stderr instead of stdout, which rung 4 will rely on to keep stdout clean for JSON. Message wording is unchanged; LogWarn/LogError prepend the WARN:/ERROR: prefixes the messages already carried, so rendered output is byte-identical to before.
…found
CheckManifestHead used to map every registry error to a bare false, so
an expired token, a rate limit, or a 5xx were indistinguishable from a
genuine "tag not found" - the tool would silently rebuild and push,
and if the cause was auth, the push then failed too after the build
was already paid for.
CheckManifestHead now returns (bool, error) and classifies the error
with errors.Is against regclient/types/errs: ErrNotFound is the normal
build-it path (false, nil); ErrHTTPUnauthorized and ErrHTTPRateLimit
get a specific, loud ERROR: pointing at credentials or rate limits
respectively; anything else falls back to the raw error. The old
strings.Contains("failed to request manifest head") heuristic is gone,
replaced by these typed checks.
By default the new error is only logged as a WARN and the build
proceeds exactly as before - the default path is behaviourally
unchanged. Passing the new --strict-registry flag makes it fatal
instead, aborting the build so a flaky or misconfigured registry check
can't silently mask itself as a cache miss. BuildLog now records
headCheckError and headCheckSkipped so tests can assert on which
branch was taken.
Adds cli/utils/check_manifest_head_test.go, which drives the
classification against an httptest.Server returning 404/401/429/500 -
no real registry or credentials required.
… output Add an exported BuildResult type (BuildLog.Result()) and two emitters: WriteBuildOutput, which prints an indented JSON BuildResult to stdout (or --output-file) when --output-format=json, and WriteGitHubOutput, which appends hash/cache-hit/image/version/primary-tag/tags/platforms to $GITHUB_OUTPUT whenever it is set, regardless of --output-format. On a build failure, cmd/build.go still emits the partial JSON result so a caller can see whatever was computed - eg. the hash - before the error. Implements Phase 2 of PLAN.md (Rung 4 of the stacked branch ladder, feature/build-result on feature/manifest-head-errors). Deferred: the e2e JSON-parseability assertion (needs a real registry) and the kerren/setup-dockem follow-up issue (external repo) - both left unchecked in PLAN.md with a note.
Add opt-in .dockerignore support so that ignored files (node_modules, dist, .git, coverage output, etc.) stop feeding the image hash and stop being streamed to the daemon. A single pattern list - the .dockerignore parsed by ReadDockerignore plus any --exclude patterns - is resolved once in BuildDockerImage and threaded to both the hash (HashDirectory / HashWatchDirectories) and the build-context tar (TarBuildContext), so the files that decide whether a build is skipped are exactly the files that get built. HashDirectory is an exclusion-aware replacement for dirhash.HashDir: with an empty pattern list it is byte-identical to the old hash, so non-adopters see no cache impact. It always retains the Dockerfile and .dockerignore regardless of the patterns (mirroring Docker, and so that editing .dockerignore invalidates the cache), and it skips broken symlinks instead of aborting the whole hash. The tar re-includes the Dockerfile it builds from (the temporary Dockerfile.* copy in the out-of-context case) and .dockerignore via trailing negations, so a Dockerfile* rule cannot drop them and break the build. Flags added: --respect-dockerignore / --no-respect-dockerignore (default off on this release; the default flips to true in v3), --ignore-file, and --exclude.
Flip --respect-dockerignore to default true (--no-respect-dockerignore
remains as the opt-out), and seed the image hash with a new hashVersion
constant ("dockem-hash-v2") so future changes to hash composition can
reset the cache deliberately instead of by accident. Adds the
testing/e2e/dockerignore-test-image/ fixture, which writes a random file
into a directory excluded by .dockerignore on every run to prove the hash
stays stable and keeps hitting the registry-side copy path. Documents the
hashVersion bump rule in CLAUDE.md and adds a README "Cache identity"
section.
BREAKING CHANGE: every hash tag published by dockem before this change is
now unreachable, because both the new default (build directories are now
hashed with .dockerignore patterns applied) and the new dockem-hash-v2
prefix change what the image hash is computed from. The first build of
each image after upgrading will be a full build-and-push even if nothing
about the image itself has changed; after that first build, caching
behaves as before.
Add --cache-from/--cache-to (repeatable) to BuildDockerImageParams and cli/cmd/build.go, forwarded verbatim as one --cache-from/--cache-to flag per value by a new pure assembleBuildxArgs helper extracted from BuildImageBuildx, enabling type=gha and other BuildKit cache backends on the buildx path. They are deliberately excluded from overallHash - a new guardrail test fails the build if build_docker_image.go's hash accumulation ever comes to reference them - since they only affect build speed, never the produced image. BuildDockerImage now warns via LogWarn when either flag is supplied but the resolved builder is not buildx, since they have no effect on the classic path. Also appends two e2e tests (single-platform buildx hash-miss/hash-hit, and --builder=docker still exercising the classic path), adds docker/setup-buildx-action to the CI workflow, documents the new flags and a GitHub Actions cache example in the README, ticks the two buildx roadmap items, extends CLAUDE.md's build-paths section, and updates PLAN.md's Phase 4.7/4.8 and cross-cutting checklists.
…the build path Rung 7 of the buildx ladder (PLAN.md Phase 4.2, 4.5). Adds builder selection and the subprocess-credential helper, but wires up no buildx build path yet: --builder resolves, logs its decision, records it, and falls through to the existing classic daemon builder unchanged. Rung 8 wires the real buildx build. - Add Platform []string and Builder string to BuildDockerImageParams, and the --platform (repeatable and comma-splittable) and --builder (auto|buildx| docker, default auto, validated via AssertOneOf) flags to cmd/build.go. - Add DetectBuildx() probing `docker buildx version`, and a pure ResolveBuilder that picks the backend. Multi-platform with buildx unavailable, or under --builder=docker, is a hard error rather than a silent single-arch fallback. - Record builder and platforms on BuildLog; surface Platforms in BuildResult. - Add TempDockerConfig writing a 0600 config.json in a 0700 temp dir for the future buildx subprocess, returning an always-safe cleanup; no-op with an empty dir when credentials are absent so an existing `docker login` still works. The password is only ever written to the 0600 file, never logged and never reaches BuildLog or the JSON BuildResult. - Pure unit tests for the resolution truth table and TempDockerConfig (no registry, no credentials, no docker needed for the error path). - Document --platform and --builder in the README; tick PLAN.md 4.2/4.5.
…ildx Wire up the real buildx build path (Phase 4.3, 4.4, 4.6). On a hash miss, when the resolved builder is buildx, dockem now shells out to a single `docker buildx build --file <dockerfile> [--platform <list>] --tag <hash> --tag <each target> --push [--progress plain] <context>`, which builds every requested platform and pushes every tag in one invocation. The classic daemon path (--builder=docker, or auto without buildx) is unchanged. - Fold the sorted, comma-joined --platform list into overallHash, immediately after the Dockerfile hash, so a change to the target architectures is a cache miss. Nothing is appended when --platform is unset, so single-platform users keep the pre-buildx hash exactly (proven by a literal-baseline unit test). - Add BuildImageBuildx (cli/utils/build_image_buildx.go). It resolves the full tag list up front via ResolveTargetTags, so TagAndPushImage / TagAndPushNewImages are not called on this path; it emits the same per-branch log lines and populates buildLog.outputTags exactly as the classic push path does, streams the subprocess stdout+stderr to os.Stderr (keeping stdout clean for --output-format=json), and passes the real Dockerfile path as --file so the out-of-context temp-file workaround in TarBuildContext is not needed. - Credentials: explicit --docker-username/--docker-password go through TempDockerConfig with DOCKER_CONFIG set on the subprocess env only, removed via defer even on failure; with no credentials the environment is inherited so an existing `docker login` works. The password never reaches the command line, logs, BuildLog or the JSON result. - Verify (by reading regclient v0.6.0 image.go) that ImageCopy copies a multi-platform image index, not a single manifest: imageCopyOpt recurses over every child manifest of a manifest.Indexer and ManifestPuts the index last. Add TestMultiPlatformBuildCopiesImageIndex asserting the copied tag resolves for both platforms (compiles here; needs a real registry to run). - Docs: CLAUDE.md describes the two build paths and their selection; README gains a multi-arch example and drops the stale "not wired up yet" note. BREAKING CHANGE: docker buildx is now required for multi-platform builds. A build that names more than one --platform errors when buildx is unavailable rather than silently producing a single-arch image. --builder=docker preserves the classic single-platform daemon path unchanged.
Dockerfiles that need a credential at build time (a private npm token, a PyPI password, an SSH key) use BuildKit secret mounts, and dockem had no way to supply one. Add --secret, a repeatable string array forwarded to `docker buildx build --secret` verbatim, one flag per value. The shape mirrors the --cache-from/--cache-to passthrough: dockem never interprets the value, so id=, env=, src= and any future buildx syntax pass straight through. In particular the value is NOT comma-split the way --platform is, since "id=npmrc,src=./.npmrc" is a single secret. It differs from the cache flags in two deliberate ways: Buildx-only by hard error, not by warning. BuildImage builds through the Engine API's types.ImageBuildOptions, which has no secrets field at all, so the classic builder would hand the Dockerfile an empty /run/secrets/<id> and publish an image built without the secret - which dockem would then copy forward under that hash tag on every later run. Ignoring a cache flag costs speed; ignoring a secret poisons the cache. ResolveBuilder therefore gained a trailing secretsRequested bool and errors under both --builder=docker and the --builder=auto fallback, alongside the existing multi-platform rule. Excluded from the image hash. A build secret is a credential the build uses, not an input that defines what it produces, and a CI token that rotates every run would otherwise change the hash every run and rebuild every image every time. hashVersion is unchanged, so no published tag is invalidated. Symbols touched: BuildDockerImageParams.Secret; the --secret flag in cli/cmd/build.go; the passthrough loop in assembleBuildxArgs, positioned after the --cache-to values and before the first --tag; ResolveBuilder's new parameter and its two error branches; BuildDockerImage's call site. Note that build_docker_image.go's `overallHash := hashVersion` seed moved below the ResolveBuilder call. That is inert - nothing reads overallHash before that point - but necessary: the seed is the start marker of the source region the hash-exclusion guardrail tests scan, and the ResolveBuilder call legitimately mentions params.Secret. Tests: three assembleBuildxArgs tests covering verbatim passthrough, its position, repeatability across both secret forms, and omission when unset; TestSecretsExcludedFromImageHash and TestSecretsDoNotAlterAssembledHashFormula as the hash guardrails; TestResolveBuilderSecretsRequestedTruthTable and TestResolveBuilderSecretsWithoutBuildxErrors for the hard error; TestBuildDockerImageSecretsWithClassicBuilderErrorBeforeAnyRegistryWork to pin the call site by asserting no hash was ever computed; and the e2e TestBuildxSecretReachesTheBuild with a new testing/e2e/secret-test-image fixture whose Dockerfile fails unless the secret arrives. The e2e test supplies its own visible non-secret value via t.Setenv and the env= form, so no CI credential is involved and testing.yaml needs no change. Docs: README gains the --secret help line, a "Build Secrets" section and a "Cache identity" note; CLAUDE.md's hash-inputs paragraph, both build path bullets, the selection rule and the subprocess-credentials section; PLAN.md gains rung 10 and Phase 4.9, and the --build-arg backlog entry now warns against citing this exclusion as precedent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vtb2HuWWRHicdEv6YF4BV4
Merge pull request #19 from kerren/feature/claude
Merge pull request #20 from kerren/feature/log-helpers
The stderr logging refactor left the blank-line separator in the build command as a raw fmt.Print, which wrote to stdout while every other human-readable message went to stderr via the log helpers. Use LogInfo so the separator lands on the same stream as the error that follows, and drop the now-unused fmt import.
Merge pull request #21 from kerren/feature/resolve-target-tags
…found Merge pull request #22 from kerren/feature/manifest-head-errors
Develop carried rewritten (but patch-equivalent) copies of this branch's logging, tag-resolution and manifest-head commits, plus one new fix: fix(logging), which routed the pre-panic separator in cmd/build.go through LogInfo and dropped the now-unused fmt import. That collided with the build-result work, which had wrapped the same separator in an `outputFormat != "json"` guard so that a JSON run kept stdout free of anything but the result. LogInfo writes to stderr, so the guard's only justification is gone: keep the build-result structure and emit the separator unconditionally through LogInfo, which leaves stdout pure JSON on the failure path as well. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CShtKNBF7tSFSiXBpCPXiB
… output Merge pull request #23 from kerren/feature/build-result
Merge pull request #24 from kerren/feature/dockerignore-hashing
BREAKING CHANGE Merge pull request #25 from kerren/feature/dockerignore-default
# Conflicts: # cli/cmd/build.go
…the build path Merge pull request #26 from kerren/feature/buildx-detect
…ildx Merge pull request #27 from kerren/feature/buildx-build
Merge pull request #28 from kerren/feature/buildx-cache
Resolved conflicts in six util files where develop changed function signatures and bodies that this branch had documented: - build_docker_image.go: kept develop's hashVersion seed, builder resolution and named-result timing; re-headed BuildDockerImage's doc block with the summary line from this branch. - build_image.go / tar_build_context.go: took develop's excludePatterns parameter and updated the doc comments to describe it. - check_manifest_head.go: took develop's (bool, error) signature and its doc comment, which supersedes this branch's now-inaccurate description of a bool-only return. - hash_watch_directories.go: took develop's excludePatterns parameter and merged both doc comments. - hash_watch_files.go: kept this branch's early return, took develop's LogInfo logging, and dropped the now-unused fmt import. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01736aPGnaHbEfDTJTzyCqNE
Merge pull request #18 from kerren/feature/util-function-documentation
Switch the `task release` and `task release-major` targets from `entro-version` to `brrelease` (https://github.com/kerren/brrelease). Flag mapping: - `--main-branch-name=main` -> `--merge-into-branch=main` - `--commit-and-tag-version-flag="--release-as=major"` -> `--release-as=major` - `entro-version` pushed the branches and tag by default (opt out via `--no-push`); `brrelease` does not, so `--auto-push` is passed to keep the existing behaviour. `brrelease` is not published on npm (its npm entry was unpublished), so it cannot be invoked through `npx` the way `entro-version` was. The tasks now call it off the `PATH` and gate on a precondition that points at the Homebrew tap when it is missing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E8fh3EVau9ZMS24We7ZgDY
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a comprehensive feature branch implementing Phases 0-4 of the development plan (PLAN.md), introducing BuildKit/multi-platform builds, .dockerignore support, machine-readable output, and improved error handling.
Summary
This PR adds four major capabilities to dockem:
docker buildxwith--platformsupport.dockerignorerespect – exclude files from both the image hash and build context tar, fixing spurious cache misses$GITHUB_OUTPUTfor pipeline integrationKey Changes
Core Build Logic
build_docker_image.go: Refactored to support both classic and BuildKit paths; addedhashVersionconstant to manage cache identity across releases; integrated.dockerignorepattern matching into hash computation; added duration tracking forBuildResultbuild_image_buildx.go(new): Implements BuildKit build path withdocker buildx build, handling multi-platform builds, cache flags, and secrets in a single invocationresolve_builder.go(new): Decides betweenbuildxanddockerbased on--builderpreference, platform requirements, and buildx availabilitydetect_buildx.go(new): Probes fordocker buildxplugin availability.dockerignoreSupporthash_directory.go(new): Exclusion-aware directory hashing that drops files matching.dockerignorepatterns; byte-identical todirhash.HashDirwhen no patterns are providedread_dockerignore.go(new): Parses.dockerignorefiles using moby's pattern matcher; feeds both hash and tar operationstar_build_context.go: Updated to respect.dockerignorepatterns when building the context tarMachine-Readable Output
build_result.go(new): Stable, exported struct containing hash, cache-hit status, resolved tags, platforms, and durationwrite_build_output.go(new): EmitsBuildResultas indented JSON to stdout or a specified filewrite_github_output.go(new): Appends build result to$GITHUB_OUTPUTfor GitHub Actions integrationresolve_target_tags.go(new): Centralizes tag resolution logic withTagReasonenum so both copy and push paths emit identical log messagesRegistry Error Handling
check_manifest_head.go: Changed signature fromboolto(bool, error); classifies errors witherrors.Isagainsterrs.ErrNotFound,ErrHTTPUnauthorized,ErrHTTPRateLimit; added--strict-registryflag to fail fast on auth/rate-limit errors instead of falling through to a buildLogging & Utilities
log.go(new): Centralized logging functions (LogInfo,LogWarn,LogError) writing to stderrassert_one_of.go(new): Flag validator for enum-like values (e.g.,--output-format)temp_docker_config.go(new): Creates temporary docker config.json for buildx authentication (required because buildx doesn't use the daemon's credentials)CLI Updates
build.go: Added flags:--builder,--cache-from,--cache-to,--platform,--secret,--respect-dockerignore,--output-format,--strict-registry; integrated new output and builder resolution logicbuild_docker_image_params.go: Extended struct with new fields for BuildKit, caching, secrets, and output optionsTesting
Comprehensive test coverage added:
build_docker_image_cache_hash_test.go, `build_docker_image_https://claude.ai/code/session_01E8fh3EVau9ZMS24We7ZgDY