Skip to content

Replace single-stage Dockerfile with multi-stage build to remove devDependencies from production image #1823

Description

@arnoldcastro5000

Single-stage Dockerfile includes devDependencies in the production image

Context

This was identified while investigating periodic memory pressure on the microservices node hosting treetracker-web-map-client. As part of understanding the pod's memory footprint, the Dockerfile and dependency tree were reviewed. The findings below are related to but independent of the resource limits issue tracked separately.


Background: how Docker images affect container memory

A Docker image is made up of layers. Every file in every layer is available to the container at runtime via the union filesystem. When Node.js starts, its module resolver walks node_modules to build its module registry. The larger node_modules is, the more filesystem entries the resolver scans at startup, and the more memory is consumed just initializing the runtime before the application serves its first request.

Beyond startup cost, large node_modules directories also mean:

  • Larger image pull times on pod reschedule (relevant when the node evicts and reschedules a pod under pressure)
  • More filesystem pages mapped into the container's address space, contributing to RSS
  • A broader attack surface — build tooling in production containers has known security implications

Current state

The current Dockerfile is single-stage:

FROM node:16-alpine
WORKDIR /app
ENV PATH /app/node_modules/.bin:$PATH
COPY package.json ./
COPY package-lock.json ./
RUN npm ci --silent
COPY . ./
RUN npm run build
CMD ["npm", "run", "start"]

npm ci --silent without --omit=dev installs all dependencies — both production and development. This means the production image includes packages that are only needed to build or test the application, not to run it.

Examples of devDependencies currently shipped to production:

Package Purpose Approximate size
cypress End-to-end testing framework ~200MB (includes browser binaries)
webpack Module bundler ~30MB
webpack-dev-server Local development server ~10MB
jest Unit testing framework ~20MB
@cypress/react Cypress React component testing ~5MB
@svgr/webpack SVG-to-React-component transform ~5MB
url-loader Webpack URL loader ~2MB

None of these serve requests in production. They are build and test tools that have no runtime role.


Why this matters for memory pressure

When next start launches, Node.js initializes its module registry by scanning node_modules. Even packages that are never explicitly require()d contribute to this scan. A node_modules directory inflated by devDependencies means:

  1. Higher baseline RSS at startup — more filesystem pages are mapped into the container's address space during module resolution, before the application handles a single request
  2. Slower cold starts — on pod reschedule (which happens during eviction events), a heavier node_modules means longer time before the pod is Ready, extending the window during which requests fail
  3. Wasted memory that cannot be reclaimed — unlike V8 heap memory which can be GC'd, filesystem-mapped memory pages for unused modules sit in RSS for the lifetime of the process

On a node already running at ~1008Mi with no memory ceiling set, every megabyte of avoidable baseline consumption narrows the headroom before memory pressure triggers eviction.


Why a simple --omit=dev flag is not sufficient

The intuitive fix — adding --omit=dev to the existing npm ci command — would break the build. The sequence in the current Dockerfile is:

RUN npm ci --silent          # installs deps
RUN npm run build            # next build runs here — needs build tooling
CMD ["npm", "run", "start"] # production server

next build calls webpack internally and the webpack() function in next.config.js references @svgr/webpack and url-loader as loaders. If devDependencies are excluded before the build step, next build fails with module-not-found errors.

The solution is a multi-stage Docker build — separating the build environment from the production runtime environment.


Proposed fix: multi-stage Dockerfile

# ── Stage 1: Builder ─────────────────────────────────────────────
FROM node:16-alpine AS builder
WORKDIR /app

# Copy manifests first for layer cache efficiency
# npm ci only re-runs when package*.json changes, not on every source change
COPY package.json package-lock.json ./

# Install ALL deps — build tooling is needed here
RUN npm ci --silent

# Copy source and build
COPY . ./
RUN npm run build


# ── Stage 2: Runner ──────────────────────────────────────────────
FROM node:16-alpine AS runner
WORKDIR /app

COPY package.json package-lock.json ./

# Install production deps only — no Cypress, webpack, jest
RUN npm ci --silent --omit=dev

# Copy only what next start needs at runtime
COPY --from=builder /app/.next ./.next
COPY --from=builder /app/public ./public
COPY --from=builder /app/next.config.js ./next.config.js

CMD ["npm", "run", "start"]

How this works:

The builder stage installs everything and runs next build, producing the compiled output in /app/.next. This stage is then discarded entirely — it never appears in the final image.

The runner stage starts clean from node:16-alpine, installs only production dependencies, and copies just the artefacts that next start needs at runtime: the .next build output, the public folder for static assets, and next.config.js which is read by next start on initialization.

Safety validation:

Before proposing this change, the following files were reviewed to confirm that next start does not require any devDependency at runtime:

  • next.config.js — only top-level require() is next-images, a production dependency. The webpack() function referencing @svgr/webpack and url-loader is only called during next build, not next start
  • src/pages/_app.js — all top-level imports resolve to production dependencies or local files. The only devDependency reference (../mocks via msw) is behind a process.env.NEXT_PUBLIC_API_MOCKING === 'enabled' guard, which is a build-time constant and never true in production ✓
  • src/pages/_document.js — imports only next/document, tss-react/nextJs, and a local file ✓
  • src/models/oidcConfig.js — imports only loglevel, a production dependency ✓
  • src/mapContext.js — imports only from react
  • src/context/configContext.js — imports only from react

No server-side file that executes during next start requires a devDependency.


Additional benefit: faster CI builds

The multi-stage Dockerfile places COPY package*.json and RUN npm ci before COPY . ./. Docker caches layers based on their inputs — so npm ci only re-runs when package.json or package-lock.json actually changes. Source code changes no longer invalidate the dependency install layer, saving several minutes per CI build.


Expected outcome

  • Production image no longer contains Cypress, webpack, jest, or other build tooling
  • Container baseline RSS at startup is reduced — less node_modules for Node's module resolver to scan
  • Pod cold start time on reschedule is faster — smaller image to pull, less initialization overhead
  • Reduced attack surface — build tooling and test fixtures are not present in the production runtime
  • Combined with the resource limits and NODE_OPTIONS changes tracked in the companion issue, this contributes to a lower and more stable memory baseline on the microservices node

Notes

  • The proposed runner stage COPY list may need to be extended if next.config.js references additional files at runtime (e.g. .env.production, custom server files). This should be verified during the PR.
  • Node.js 16 is end-of-life. The multi-stage build is a good opportunity to upgrade the base image to node:20-alpine, though that should be treated as a separate concern and tested independently.
  • A PR will follow this issue.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions