Skip to content

fix(docker): rebuild the pipeline image when its build inputs change #103

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
flamboh merged 1 commit into from
Aug 24, 2026
Merged

fix(docker): rebuild the pipeline image when its build inputs change #103

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/user/setup-pipeline.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Container options come before the pipeline command:
./scripts/netflow-db-docker.sh --capture-root /absolute/path/to/captures pipeline ...
```

The wrapper builds the image when it is missing; pass `--build` to rebuild it after a source update. Each `--capture-root` mounts read-only at the same absolute path inside the container, so `root_path` in `datasets.json` needs no change. Output stays under the repository's `data/` directory, owned by you.
The wrapper builds the image when it is missing or when its build inputs change, so pulling an update rebuilds automatically. Pass `--build` to force a rebuild. Each `--capture-root` mounts read-only at the same absolute path inside the container, so `root_path` in `datasets.json` needs no change. Output stays under the repository's `data/` directory, owned by you.

On macOS, bind mounts over large capture trees are slower than native filesystem access.

Expand Down
36 changes: 35 additions & 1 deletion scripts/netflow-db-docker.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,43 @@ if [[ "$submodule_mode" != "160000" || "$submodule_type" != "commit" || "$submod
exit 1
fi

if ((force_build)) || ! docker image inspect "$IMAGE" >/dev/null 2>&1; then
# Fingerprint every input that changes the image, so pulling a fix rebuilds
# instead of silently reusing a stale image. Hashing uses git, which this
# wrapper already requires, rather than a coreutils-only checksum tool.
fingerprint="$(
cd "$ROOT_DIR"
{
printf '%s\n' "$nfdump_commit"
git hash-object \
Dockerfile \
.dockerignore \
rust-toolchain.toml \
Cargo.toml \
Cargo.lock \
tools/netflow-db/Cargo.toml \
vendor/scripts/compile-nfdump.sh
find tools/netflow-db/src -type f | LC_ALL=C sort | xargs git hash-object
} | git hash-object --stdin
)"

image_fingerprint="$(docker image inspect "$IMAGE" \
--format '{{ index .Config.Labels "atlantis.build-fingerprint" }}' 2>/dev/null || true)"

if ((force_build)); then
build_reason="requested"
elif [[ -z "$image_fingerprint" ]]; then
build_reason="missing"
elif [[ "$image_fingerprint" != "$fingerprint" ]]; then
build_reason="out of date"
else
build_reason=""
fi

if [[ -n "$build_reason" ]]; then
echo "building $IMAGE ($build_reason)" >&2
docker build \
--build-arg "NFDUMP_COMMIT=$nfdump_commit" \
--label "atlantis.build-fingerprint=$fingerprint" \
--tag "$IMAGE" \
"$ROOT_DIR"
fi
Expand Down
Loading