Ensuring Docker Builds work on Linux

[DragonBishop]

Docker Desktop for Mac resolves host.docker.internal automatically and its bind-mount layer doesn't enforce real uid/gid ownership. Plain Docker Engine on Linux does neither: bind mounts are real directories with real permission bits, and nothing defines host.docker.internal unless you tell it to.

On a fresh clone, I found the stack breaks in four separate places. They all seem to be the product of designing this build on/for Mac. I've tried to make sure these changes will help NewsBlur run on Linux without breaking it on other operating systems.

While this pull request was prepared with the assistance of artificial work, this request and the changes associated with it have all been reviewed by me as well.

1. haproxy won't start at all

Every backend in haproxy.docker-compose.cfg has a fallback line so that if it can't resolve a hostname, it just disables that one backend instead of crashing. One backend, camera_monitor, is missing that line. Here's what happens with and without it:

# as it is now:
[ALERT] (8) : 'server camera_monitor/camera_monitor' : could not resolve address 'host.docker.internal'.
[ALERT] (8) : Failed to initialize server(s) addr.
[WARNING] (1) : Failed to load worker (8) exited with code 1 (Exit)

# with the same fallback line the other backends already have:
[NOTICE] (8) : 'server camera_monitor/camera_monitor' : could not resolve address 'host.docker.internal', disabling server.
# haproxy keeps starting normally

Fix: add the same resolvers docker init-addr last,libc,none fallback the other backends already use.

2. host.docker.internal doesn't resolve on Linux

Confirmed directly:

$ docker run --rm alpine getent hosts host.docker.internal
(nothing, it doesn't resolve)

$ docker run --rm --add-host host.docker.internal:host-gateway alpine getent hosts host.docker.internal
172.17.0.1        host.docker.internal  host.docker.internal

Fix: add this to the haproxy service in docker-compose.yml:

extra_hosts:
  - "host.docker.internal:host-gateway"

host-gateway resolves to the host's IP from inside the container, on every platform, so it's a no-op on Mac (Desktop already defines the name) and fills the gap on Linux. Per Docker's docs on --add-host. Not verified on an actual Mac.

3. Elasticsearch crashes on boot: a JVM flag that only exists on ARM

ES_JAVA_OPTS had -XX:UseSVE=0 hardcoded in:

$ docker run -e "ES_JAVA_OPTS=-Xms384m -Xmx384m -XX:UseSVE=0" ... elasticsearch:8.17.0
Unrecognized VM option 'UseSVE=0'
Error: Could not create the Java Virtual Machine.

This is a hardware issue: -XX:UseSVE=0 is ARM-only, so this would crash the same way on any x86_64 host, Intel Mac included.

Fix: check the container's actual architecture at boot and only add the flag when it's really arm64, in docker/elasticsearch/entrypoint.sh, following the same pattern docker/postgres/entrypoint.sh already used previously in this project.

docker/elasticsearch/entrypoint.sh:

#!/bin/bash
arch="$(uname -m)"
if [[ "$arch" == "aarch64" || "$arch" == "arm64" ]]; then
  export ES_JAVA_OPTS="$ES_JAVA_OPTS -XX:UseSVE=0"
  export CLI_JAVA_OPTS="$CLI_JAVA_OPTS -XX:UseSVE=0"
fi

exec /bin/tini -- /usr/local/bin/docker-entrypoint.sh eswrapper

docker-compose.yml:

entrypoint: ["/bin/bash", "/entrypoint.sh"]
volumes:
  - ./docker/elasticsearch/entrypoint.sh:/entrypoint.sh:ro

Considered, but rejected: -XX:+IgnoreUnrecognizedVMOptions: it would silently swallow any bad flag going forward, not just this one.

CLI_JAVA_OPTS is load-bearing: bin/elasticsearch sources elasticsearch-cli, which reads CLI_JAVA_OPTS for its own small bootstrap JVM before launching the real server, so both env vars need the flag.

The script checks two spellings, aarch64 and arm64, because the same physical chip reports differently depending on the OS: Linux's uname -m says aarch64, macOS's says arm64.

4. Elasticsearch crashes on boot: the data folder ends up owned by root

On a truly fresh clone, the docker/volumes/elasticsearch folder doesn't exist yet. Docker creates it for you when the container starts, but it creates it as root:

$ ls -lan /tmp/es-repro-fresh
drwxr-xr-x 2 0 0 40 ... .

$ docker run -v /tmp/es-repro-fresh:/usr/share/elasticsearch/data ... elasticsearch:8.17.0
"error.message":"failed to obtain node locks, tried [/usr/share/elasticsearch/data]; ..."
"...Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/node.lock..."

Elasticsearch's container runs as a non-root user (uid 1000) and never chowns its own data folder. Docker Desktop's bind-mount layer doesn't enforce real uid/gid, so this never surfaces there. On native Linux the bind mount is a directory with real permission bits, so a root-owned folder genuinely can't be written by uid 1000.

Fix: a tiny one-off container that runs before Elasticsearch and fixes the folder's ownership.

newsblur_db_elasticsearch_permissions:
  container_name: newsblur_db_elasticsearch_permissions
  image: alpine:latest
  command: chown -R 1000:0 /usr/share/elasticsearch/data
  volumes:
    - ./docker/volumes/elasticsearch:/usr/share/elasticsearch/data
  restart: "no"

Then make Elasticsearch wait for it to finish:

depends_on:
  newsblur_db_elasticsearch_permissions:
    condition: service_completed_successfully

This runs every time you start the stack, but it's cheap and a no-op if the ownership is already correct, so it doesn't hurt anything on Mac either. Result:

$ curl localhost:9200/_cluster/health
{"cluster_name":"newsblur-local","status":"green", ...}

5. imageproxy crashes: wrong image for the CPU

imageproxy was pinned to yusukeito/imageproxy:v0.11.2, which is an ARM-only build:

$ docker run yusukeito/imageproxy:v0.11.2 ...
WARNING: The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v4)
exec /app/imageproxy: exec format error

Fix: use ghcr.io/willnorris/imageproxy:latest instead, a proper multi-arch image:

$ docker run --entrypoint /app/imageproxy ghcr.io/willnorris/imageproxy:latest -addr 0.0.0.0:8088 ...
# starts fine, no crash

This was already fixed once, then broken again by an unrelated commit. On 2022-05-25 (656479a876), the default was ghcr.io/willnorris/imageproxy:latest, the multi-arch image, with the arm64-only build left commented out as an opt-in. On 2022-12-26, commit 1065c964fd ("Fixing elasticsearch to allow consul to assume its OK") flipped it back to the arm64-only image as the default.

Two things I couldn't verify myself:

• The ARM branch of the fix in Keyboard Shortcuts #3 looks right on paper, but I don't have an arm64 machine to actually run it on.
• The claim in Feed takes over my newsblur page #2 that host-gateway is harmless on Mac comes from Docker's own docs, not from testing on an actual Mac.