Optimal Docling

A fully offline, GPU-accelerated Docling Docker image with every model baked in: high-accuracy document conversion (PDF, DOCX, PPTX, HTML, EPUB, and images to Markdown, JSON, or HTML) that runs air-gapped, with no network access at runtime.

Build it once with internet access, then run Docling 2.107.0 on NVIDIA, AMD, or Intel GPUs (or plain CPU). The default image targets the latest stable NVIDIA CUDA line; AMD ROCm, Intel XPU, and CPU variants build from the same Dockerfile. Installed with uv and tuned for maximum accuracy, it suits self-hosted, data-sensitive document processing and RAG pipelines.

Component	Version	Why
Base image	nvidia/cuda:13.3.0-cudnn-runtime-ubuntu24.04	Latest stable CUDA line; apt-get upgrade is run at build start
Package manager	uv 0.11.20 (pinned)	Fast resolution, BuildKit-cached, bytecode precompiled (UV_COMPILE_BYTECODE=1)
PyTorch	2.12.0+cu130 (default PyPI Linux wheel)	Bundles the full CUDA 13 runtime via nvidia-* wheels, matching the base image
Docling	2.107.0 + [vlm,easyocr,rapidocr]	Latest release; rapidocr extra already includes CPU onnxruntime (see below)
Python	3.12 (Ubuntu 24.04 system interpreter in a uv venv)	Full wheel coverage for all extras
PID 1	tini	Clean signal handling / zombie reaping

Baked-in models (/opt/docling-models, DOCLING_ARTIFACTS_PATH): layout: Heron, heron-101 (most accurate, arXiv:2509.11720; max default), Egret-XLarge · TableFormer v1 + v2 · code/formula model · picture classifier · RapidOCR PP-OCRv4 (torch + onnx backends × en + zh + latin) · EasyOCR (craft + english_g2 + latin_g2 → en/pt/es/fr/de) · GraniteDocling-258M (VLM pipeline) · SmolVLM (picture description) · Tesseract eng+por+spa+fra+deu+osd traineddata.

Requirements

• Docker 23+ (BuildKit) or Podman 4.6+
• An accelerator + its host runtime. The default build targets NVIDIA (CUDA 13): driver R580+ and the nvidia-container-toolkit, Turing (RTX 20xx / GTX 16xx) or newer. GeForce on R525–R57x: use the CUDA 12 variant (Troubleshooting → "Error 803"). AMD Radeon (ROCm) and Intel Arc (XPU) are supported through build variants; see "Other accelerators" below. No GPU? The CPU variant runs anywhere.
• Internet during the build only (~10 GB of wheels + weights). The resulting image needs none.

Build

./build.sh        # auto-detects docker/podman, builds, runs the offline GPU selftest
# or manually:
DOCKER_BUILDKIT=1 docker build --pull -t docling .
podman build --pull -t docling .

The Dockerfile is OCI-compliant: it uses no SHELL/HEALTHCHECK (Docker-format-only instructions that podman/buildah warn about and ignore), so it builds warning-free under both engines; every RUN is a POSIX-sh && chain.

Expect a ~15 GB final image (CUDA torch stack ≈ 7 GB, model weights ≈ 4 GB).

Build arguments:

ARG	Default	Purpose
BASE_IMAGE	nvidia/cuda:13.3.0-cudnn-runtime-ubuntu24.04	Accelerator base image (NVIDIA default; ROCm/Intel/CPU bases for other accelerators, see "Other accelerators")
UV_VERSION	0.11.20	Pinned uv release
TORCH_VERSION / TORCHVISION_VERSION	2.12.0 / 0.27.0	Pinned torch pair
TORCH_INSTALL_ARGS	(empty)	Redirect torch+torchvision to another accelerator wheel index: --torch-backend=cu126 (older NVIDIA) or --index-url …/whl/{cu126,rocm7.2,xpu,cpu} (NVIDIA-12 / AMD / Intel / CPU)
DOCLING_VERSION	2.107.0	Docling release
DOCLING_EXTRAS	vlm,easyocr,rapidocr	Docling extras to install
TESSERACT_LANGS	eng por spa fra deu osd	Extra tesseract-ocr-<lang> packages
EXTRA_SYSTEM_PACKAGES	(empty)	Extra apt packages for accelerator runtimes (e.g. Intel XPU: intel-opencl-icd libze1)
DOCLING_MODELS	layout … smolvlm	Model sets to prefetch (docling-tools models download --help)
EXTRA_HF_REPOS	docling-project/docling-layout-heron-101 docling-project/docling-layout-egret-xlarge	Additional HF repos to bake

Verify it is truly offline

docker run --rm --gpus all --network=none docling selftest --require-gpu
# Podman (GPU via CDI; run `sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml` once):
podman run --rm --device nvidia.com/gpu=all --security-opt=label=disable \
    --network=none docling selftest --require-gpu

Prints torch/CUDA/GPU, onnxruntime providers, tesseract, and checks every model folder (including both RapidOCR backend weight trees) exists. --network=none makes any network attempt fail instantly; HF_HUB_OFFLINE=1 / TRANSFORMERS_OFFLINE=1 are baked in as a second layer of protection.

Usage

Writing to the mounted directory: the image runs as root by default, which makes bind mounts Just Work: under rootless Podman, container-root is you (unprivileged on the host), so outputs come out owned by your user; under Docker, root writes succeed, and you can add --user "$(id -u):$(id -g)" if you want files owned by you. The image is hardened to run under any uid (all caches pinned to /tmp). On SELinux hosts mount with -v "$PWD:/data:z". A docling (uid 1001) account exists for opt-in --user docling setups (grant it the mount once with podman unshare chown -R 1001:1001 "$PWD").

# Docling CLI (default entrypoint), fully offline. Auto-OCR picks RapidOCR
# on onnxruntime (Docling's first-choice default) out of the box:
docker run --rm --gpus all --network=none \
    -v "$PWD:/data" docling mydoc.pdf

# Maximum-accuracy pipeline (heron-101 layout, TableFormer ACCURATE,
# GPU RapidOCR via the torch backend, code/formula enrichment):
docker run --rm --gpus all --network=none \
    -v "$PWD:/data" docling max mydoc.pdf

# GraniteDocling VLM pipeline (weights are baked in):
docker run --rm --gpus all --network=none \
    -v "$PWD:/data" docling --pipeline vlm --vlm-model granite_docling mydoc.pdf

# Interactive shell:
docker run --rm -it --gpus all --network=none \
    -v "$PWD:/data" docling shell

Command-line reference

The entrypoint dispatches on the first argument:

First argument	Runs	Purpose
max	convert.py	Maximum-accuracy GPU pipeline (this project)
selftest	selftest.py	Offline verification of GPU + baked models
shell	bash	Interactive shell inside the image
anything else	docling CLI	The full stock Docling command line

max mode: all flags

Flag	Values	Default	Effect
inputs (positional)	files / directories	required	Documents to convert; directories are scanned recursively for PDFs and EPUBs
-o, --output	path	alongside each input	Output directory; by default each result is written next to its source file
--device	gpu, cuda, cuda:N, xpu, cpu, auto	cuda	Compute device. gpu aliases cuda (which also drives AMD ROCm builds); xpu = Intel GPU; cuda:N picks GPU N on multi-GPU NVIDIA/ROCm hosts; unavailable device falls back to auto with a warning
--num-threads	int	OMP_NUM_THREADS (4)	CPU inference threads (AcceleratorOptions.num_threads); overrides the OMP_NUM_THREADS / DOCLING_NUM_THREADS env vars. Mainly affects CPU-bound runs
--layout	heron-101, egret-xlarge, heron	heron-101	Layout model; heron-101 is the most accurate per arXiv:2509.11720
--ocr	rapidocr, easyocr, tesseract, off	rapidocr	OCR engine (off disables OCR)
--ocr-backend	torch, onnxruntime	torch	RapidOCR backend: torch = GPU, onnxruntime = CPU (Docling's auto-OCR default)
--ocr-lang	L1,L2,...	per engine	rapidocr chinese default (en+zh); also english and latin sets; --ocr-lang de/fr/es/it/pt/nl works on GPU; easyocr en,pt,es,fr,de; tesseract eng,por,spa,fra,deu
--low-vram	flag	off	Fit ~6-8 GB GPUs: batch sizes 8/4/2, queue 32, enrichment batch 4; accuracy unchanged
--layout-batch-size	int	64 (8 w/ --low-vram)	Layout GPU batch size; explicit value overrides presets
--ocr-batch-size	int	16 (4 w/ --low-vram)	OCR batch size; explicit value overrides presets
--table-batch-size	int	4 (2 w/ --low-vram)	Table batch size; explicit value overrides presets
--images-scale	float	2.0	Page/picture render scale; 2.0 = 144 dpi (2× native) for maximum-fidelity crops, embedded images, and picture-description input. Lower to 1.0 for native-resolution speed
--describe-pictures	flag	off	Add SmolVLM picture descriptions (slower)
--no-enrich	flag	off	Disable code/formula/picture-classification enrichment
--export-json	flag	off	Also export lossless DoclingDocument JSON
--export-html	flag	off	Also export HTML
--no-compile	flag	off	Disable torch.compile of Docling's engines (one-shot or CPU runs)
--image-mode	embedded, referenced, placeholder	embedded	Pictures in MD/HTML: base64 data URIs (self-contained), PNG files in <name>_artifacts/, or <!-- image --> markers
-v / -vv (--verbose)	counting flag	warnings only	-v = verbose (info), -vv = debug; same semantics as the stock docling CLI

Exit codes: 0 all files converted · 1 at least one conversion failed or a named input was missing · 2 no usable input files (or bad output path / invalid arguments).

selftest: all flags

Flag	Default	Effect
--require-gpu	off	Exit non-zero when no CUDA device is visible (otherwise only a warning)

Exit codes: 0 everything present · 1 a check failed.

Stock docling CLI (default mode)

The complete upstream CLI ships unmodified: docker run --rm docling --help prints every option. The ones most relevant inside this image:

Flag	Notes in this image
--device [auto|cpu|cuda|mps|xpu]	Default auto; to pin a specific GPU, scope the container: --gpus '"device=N"' / -e CUDA_VISIBLE_DEVICES=N (podman: --device nvidia.com/gpu=N)
--ocr-engine, --ocr-lang	All engines baked; e.g. --ocr-engine easyocr --ocr-lang en,pt,es,fr,de
--pipeline, --vlm-model	--pipeline vlm --vlm-model granite_docling runs fully offline
--table-mode [fast|accurate]	Table structure mode
--enrich-code, --enrich-formula, --enrich-picture-classes, ...	Enrichment toggles; all models baked
--from, --to, --output	Input/output formats and output directory
--num-threads, --page-batch-size	CPU threads (env default OMP_NUM_THREADS=4) and page batching
--artifacts-path	Already set image-wide via DOCLING_ARTIFACTS_PATH
--abort-on-error, -v	Error handling and verbosity

Selecting the device (GPU / CPU)

# max mode: --device gpu|cuda|cuda:N|xpu|cpu|auto   (default: cuda)
docker run --rm --gpus all --network=none \
    -v "$PWD:/data" docling max --device cuda:1 mydoc.pdf   # second GPU
docker run --rm --network=none \
    -v "$PWD:/data" docling max --device cpu mydoc.pdf      # force CPU

# stock docling CLI: it has its own flag (auto|cpu|cuda|mps|xpu)
docker run --rm --gpus all --network=none \
    -v "$PWD:/data" docling --device cuda mydoc.pdf

# AMD ROCm / Intel XPU images use different passthrough flags and --device
# values; see "Other accelerators" (e.g. AMD: --device=/dev/kfd --device=/dev/dri
# --group-add video, then `max --device cuda`; Intel: --device=/dev/dri, `max --device xpu`).

gpu is an alias for cuda, which also drives AMD ROCm builds (torch exposes ROCm through the cuda device); xpu selects an Intel GPU. An unavailable device (or out-of-range cuda:N) falls back to auto with a warning instead of crashing. Only cuda takes an index (cuda:N); to pin a GPU on a multi-GPU host, scope what the container sees: -e CUDA_VISIBLE_DEVICES=1 (NVIDIA), -e HIP_VISIBLE_DEVICES=1 (AMD), or -e ZE_AFFINITY_MASK=1 (Intel), and the chosen card appears as device 0. (DOCLING_DEVICE is read only by code that builds AcceleratorOptions() without an explicit device; both CLIs here always pass one, so the env var is ignored.)

OCR language coverage

Engine	Default in max mode	Coverage (baked, offline)
rapidocr (default)	chinese set	three baked sets: english, chinese, and latin (latin recognizer covers de/fr/es/it/pt/nl/...). Left at the upstream chinese default; pass --ocr-lang de (etc.) to select the latin set on the GPU torch backend
easyocr	en,pt,es,fr,de	full requested set via the baked latin_g2 recognizer (GPU-capable)
tesseract	eng,por,spa,fra,deu	full requested set; osd (orientation/script detection) data also baked

For Latin-script European documents you now have two GPU paths: rapidocr's latin set (max --ocr-lang de mydoc.pdf, as of docling 2.103.0) or EasyOCR (max --ocr easyocr mydoc.pdf). An unrecognized --ocr-lang value for rapidocr prints a warning and falls back to the default set. The stock docling CLI equivalents: --ocr-engine rapidocr --ocr-lang de or --ocr-engine easyocr --ocr-lang en,pt,es,fr,de.

Low VRAM

docker run --rm --gpus all --network=none \
    -v "$PWD:/data" docling max --low-vram mydoc.pdf

--low-vram only shrinks batch sizes (layout 64→8, OCR 16→4, table 4→2), bounds the page queue, and reduces enrichment batching. The models (heron-101, TableFormer ACCURATE), image scale, and every accuracy feature stay identical. Quality is unchanged; throughput drops. Explicit --*-batch-size flags override the preset.

Images in exports and the EPUB "Image not available" case

If a plain-CLI conversion (e.g. of an EPUB) leaves <!-- 🖼️❌ Image not available ... --> comments in the Markdown, the cause is a mode mismatch: the stock CLI exports images embedded by default, but fetches none (--html-image-fetch defaults to none), so EPUB pictures carry no data. Two working paths in this image:

• max mode (recommended): EPUB inputs automatically extract their bundled images (local-archive only; --network=none stays intact), PDFs already render page pictures, and --image-mode controls the output form (embedded base64 by default, referenced PNG files, or placeholder).
• Stock CLI: in this image --html-image-fetch local is the default (set by the entrypoint). EPUB images embed out of the box; pass your own --html-image-fetch to override (last value wins). Previously you had to add --html-image-fetch local so images inside the EPUB/HTML are extracted; remote/all would need network access and are pointless under --network=none.

This exact scenario is regression-tested: a sample EPUB converted with stock defaults reproduces the error comment, while max produces ![Image](data:image/png;base64,...) in Markdown and HTML, and referenced mode writes the PNGs to <name>_artifacts/.

Default layout model: evaluated (heron-101)

max defaults to heron-101 on the strength of the primary source, the Docling layout paper (arXiv:2509.11720): Table 3 gives it the top mAP on canonical DocLayNet (0.780 vs heron 0.776, egret-m 0.765, egret-x 0.753) and the best score on the harder DocLayNet-v2 (0.758, with the best small-object AP), and the paper's conclusion names heron-101 its best model. Under Docling's own post-processed docling-eval (Table 4) it ties heron at 0.61, with egret-x at 0.60. On GPU the cost is negligible: ~28 ms/image at batch 200 (Table 5; the paper notes GPU performance is similar across the family). The docs' docstring suggestion of Egret for "complex documents" is not supported by these benchmark numbers. Caveat: on CPU heron-101 is the slowest of the family (0.988 s/img vs egret-x 0.808, egret-m 0.334). For CPU-bound runs prefer --layout egret-xlarge or heron; both stay baked and selectable.

When to use the VLM pipeline (GraniteDocling) vs. RapidOCR

max mode (and the stock standard pipeline) chains specialist models (heron-101 layout + TableFormer + an OCR engine), where RapidOCR/EasyOCR only ever read raster text. GraniteDocling (--pipeline vlm --vlm-model granite_docling, weights baked) replaces the whole chain with one 258M vision-language model that reads page images end-to-end and emits structured DocTags.

Prefer the standard pipeline (default) when documents are digital PDFs with embedded text (OCR is barely involved, conversion is fastest and deterministic), when table fidelity matters most (TableFormer ACCURATE is the strongest documented table path), when you need non-English Latin languages (pt/es/fr/de via EasyOCR/Tesseract; GraniteDocling is optimized for English/Latin-script with only experimental Japanese, Arabic, and Chinese), or when throughput per GPU matters.

Prefer GraniteDocling when pages are scanned or photographed and the classic OCR + layout assembly struggles (degraded scans, unusual layouts), when content is dense with equations and code blocks (its enhanced equation/inline-math and code recognition is a headline feature), or when you want a single model's end-to-end reading order and structure instead of stitched stages. It is slower and heavier per page than RapidOCR-based OCR; for sustained VLM throughput the Docling docs recommend serving it via vLLM rather than inline inference.

A practical rule: digital or lightly-scanned business documents → max (standard pipeline); messy scans, math/code-heavy papers → try GraniteDocling and compare on a sample.

ONNX Runtime: why docling[onnxruntime] is not used here

Evaluated against the Docling 2.107.0 dependency metadata and source:

Extra	Expands to	Installs on Linux
docling[rapidocr]	docling-slim[feat-ocr-rapidocr-onnx]	rapidocr + plain CPU onnxruntime
docling[onnxruntime]	docling-slim[models-onnxruntime]	onnxruntime-gpu<1.24 (a CUDA 12 build)

• The rapidocr extra already ships CPU onnxruntime, which is exactly what Docling's auto-OCR needs: its selection order tries RapidOCR on onnxruntime first (docling/models/stages/ocr/auto_ocr_model.py), so the stock docling CLI works in this image with no extra packages.
• docling[onnxruntime] exists to give Docling's ONNX inference engines (alternative layout / image-classification runtimes) a GPU path, but the pinned onnxruntime-gpu<1.24 is built for CUDA 12. On this CUDA 13 base its CUDAExecutionProvider cannot load (libcudart.so.12 etc. are absent), so it would silently fall back to CPU while adding ~250 MB, the exact problem reported against docling-serve's CUDA images (docling-serve#434). Stable PyPI onnxruntime-gpu did not ship CUDA-13 wheels until 1.27, and docling's <1.24 cap is far below that, so the extra only ever pulls a CUDA-12 build.
• Nothing accuracy-critical here needs ORT: layout (heron-101), TableFormer, enrichments, EasyOCR, RapidOCR-torch and the VLMs all run on torch CUDA.

Verdict: stay on CUDA 13 (the default here). PyTorch made cu130 the stable default and is pruning CUDA 12 builds: torch 2.12 removed cu128 wheels entirely (cu126 remains as the older-driver fallback), so a CUDA 12 default would pin this image to a deprecation track and forfeit the newest kernels, only to give ORT a GPU path that no accuracy-critical stage uses. CPU ORT (≈60 MB, already included) keeps the stock CLI's default OCR fully functional. Re-checked June 2026: onnxruntime-gpu 1.27 has since moved to CUDA-13 wheels (nvidia-*-cu13 deps), but docling's models-onnxruntime extra still pins <1.24 (a CUDA-12 build), so docling[onnxruntime] keeps installing a non-functional ORT-GPU on this CUDA 13 base regardless (ORT is also migrating its CUDA runtime to a plugin execution provider). Nothing changes the verdict: the extra cannot give a working GPU path here, and no accuracy-critical stage needs one.

Want ORT itself on the GPU anyway? Build the CUDA 12 variant, and don't mix the rapidocr and onnxruntime extras (plain onnxruntime and onnxruntime-gpu both install the onnxruntime module and collide):

docker build --pull -t docling:cu12-ort \
  --build-arg BASE_IMAGE=nvidia/cuda:12.9.1-cudnn-runtime-ubuntu24.04 \
  --build-arg TORCH_INSTALL_ARGS="--torch-backend=cu126" \
  --build-arg DOCLING_EXTRAS="vlm,easyocr,onnxruntime" .
# then add the RapidOCR library itself (without the CPU-ORT pin):
#   uv pip install "rapidocr>=3.8,<4"   # e.g. via a follow-up RUN layer

ONNX Runtime built for CUDA 12.x is compatible with any 12.x runtime, so onnxruntime-gpu 1.23 works against the 12.9 base above.

Older GPUs / drivers (< R580)

docker build --pull -t docling:cu12 \
  --build-arg BASE_IMAGE=nvidia/cuda:12.9.1-cudnn-runtime-ubuntu24.04 \
  --build-arg TORCH_INSTALL_ARGS="--torch-backend=cu126" .

torch 2.12.0 publishes cu130 (default) and cu126 wheels; cu128 was removed from its build matrix, so use cu126 for the CUDA 12 track. The cu126 build also keeps Volta (V100) support and works with older R525+ drivers via CUDA 12 minor-version compatibility; see https://pytorch.org/get-started/locally/ if you change TORCH_VERSION.

Other accelerators (AMD ROCm / Intel XPU / CPU)

The default image targets NVIDIA CUDA, but the same Dockerfile builds for every accelerator Docling supports; you change only BASE_IMAGE (an Ubuntu-24.04 base for that vendor) and TORCH_INSTALL_ARGS (the matching PyTorch wheel index). PyTorch wheels are single-backend, so each accelerator gets its own image; convert.py (max) and selftest are device-aware and work unchanged on whichever you build. NVIDIA is the tested path; the ROCm and Intel variants follow PyTorch's and Docling's official device guidance.

Accelerator	--build-arg BASE_IMAGE=	--build-arg TORCH_INSTALL_ARGS=	--device	Container GPU passthrough
NVIDIA CUDA 13 (default)	nvidia/cuda:13.3.0-cudnn-runtime-ubuntu24.04	(empty)	cuda	--gpus all (docker) / --device nvidia.com/gpu=all (podman)
NVIDIA CUDA 12	nvidia/cuda:12.9.1-cudnn-runtime-ubuntu24.04	--torch-backend=cu126	cuda	same as CUDA 13
AMD ROCm	rocm/dev-ubuntu-24.04	--index-url https://download.pytorch.org/whl/rocm7.2	cuda	--device=/dev/kfd --device=/dev/dri --group-add video
Intel Arc (XPU)	ubuntu:24.04 + EXTRA_SYSTEM_PACKAGES	--index-url https://download.pytorch.org/whl/xpu	xpu	--device=/dev/dri
CPU only	ubuntu:24.04	--index-url https://download.pytorch.org/whl/cpu	cpu	(none)

AMD Radeon / ROCm. PyTorch's ROCm wheels expose AMD GPUs through the same torch.cuda API, so Docling drives them with --device cuda (there is no separate rocm device). Match the torch ROCm wheel index to a ROCm your host amdgpu stack supports (torch 2.12 ships rocm7.2):

docker build --pull -t docling:rocm \
  --build-arg BASE_IMAGE=rocm/dev-ubuntu-24.04 \
  --build-arg TORCH_INSTALL_ARGS="--index-url https://download.pytorch.org/whl/rocm7.2" .

docker run --rm --device=/dev/kfd --device=/dev/dri --group-add video \
  --network=none docling:rocm selftest --require-gpu
docker run --rm --device=/dev/kfd --device=/dev/dri --group-add video \
  --network=none -v "$PWD:/data" docling:rocm max mydoc.pdf

The host needs the amdgpu kernel driver + ROCm and your user in the video/render groups. Scope multi-GPU hosts with -e HIP_VISIBLE_DEVICES=N. (The torch ROCm wheels bundle their own ROCm runtime, so the base image mainly supplies supporting userspace; the host driver is what must match.)

Intel Arc / XPU. Use the XPU wheel index and run with --device xpu:

docker build --pull -t docling:xpu \
  --build-arg BASE_IMAGE=ubuntu:24.04 \
  --build-arg TORCH_INSTALL_ARGS="--index-url https://download.pytorch.org/whl/xpu" \
  --build-arg EXTRA_SYSTEM_PACKAGES="intel-opencl-icd libze1" .

docker run --rm --device=/dev/dri --network=none docling:xpu selftest --require-gpu
docker run --rm --device=/dev/dri --network=none \
  -v "$PWD:/data" docling:xpu max --device xpu mydoc.pdf

The host needs Intel's GPU kernel driver (i915/xe) and compute runtime; see Intel's client-GPU install guide. Pass /dev/dri into the container; scope a specific GPU/tile with -e ZE_AFFINITY_MASK=N.

CPU only (no accelerator, runs anywhere):

docker build --pull -t docling:cpu \
  --build-arg BASE_IMAGE=ubuntu:24.04 \
  --build-arg TORCH_INSTALL_ARGS="--index-url https://download.pytorch.org/whl/cpu" .
docker run --rm --network=none -v "$PWD:/data" docling:cpu max --device cpu mydoc.pdf

Device-coverage caveat. The core layout/table/enrichment/VLM models run on the selected accelerator through torch; the layout model (heron-101) explicitly declares CPU/CUDA/XPU support. If a model does not support the requested device, Docling raises a clear AcceleratorDeviceNotAvailableError rather than degrading silently; --device auto sidesteps this by picking the best supported device per model, and --device cpu forces CPU. The OCR engines are the main XPU caveat: RapidOCR's torch backend and EasyOCR are exercised on CUDA (and ROCm through the cuda path) but don't advertise Intel-XPU support, so on an XPU build OCR may run on CPU or error depending on the engine; pass --ocr-backend onnxruntime (CPU) or --ocr off if so. selftest prints exactly which accelerator torch sees, so run it first on a new device.

Troubleshooting

Error: setting up CDI devices: unresolvable CDI devices nvidia.com/gpu=all (podman, at run time, typically right after a successful build): this is host-side: podman found no NVIDIA CDI specification in /etc/cdi or /var/run/cdi. The image is fine. One-time fix:

sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml   # needs nvidia-container-toolkit
nvidia-ctk cdi list                                          # must list nvidia.com/gpu=all

Then re-run ./build.sh (the build is cached) or just the verify command from "Verify it is truly offline". Regenerate the spec after NVIDIA driver upgrades or MIG-device changes; stale specs fail the same way. Rootless podman with a user-local spec: generate into ~/.config/cdi/ and either pass --cdi-spec-dir=$HOME/.config/cdi or add that directory to cdi_spec_dirs in containers.conf. build.sh now pre-flights this and prints these instructions instead of the raw error.

PermissionError: [Errno 13] Permission denied: '<output>.md' when converting into a mounted directory: the bind mount keeps the host's ownership. Current images run as root by default, which writes fine under both engines (rootless Podman maps container-root to you, so outputs are owned by your user). If you run with --user, ensure that uid can write the host directory; on SELinux hosts mount with -v "$PWD:/data:z". The image is hardened for any uid; caches are pinned to /tmp.

Error 803: system has unsupported display driver / cuda driver combination (torch; selftest shows CUDA available: False; in-container nvidia-smi misleadingly reports a newer CUDA version than your driver): the kernel driver can't natively run this image's CUDA major version, and the base image's bundled forward-compatibility libraries (/usr/local/cuda-*/compat) were being loaded; NVIDIA's container troubleshooting documents this exact failure, and forward compatibility is not supported on GeForce at all. Current images purge those compat libs, so a too-old driver now fails honestly instead. Fix: match the image to your driver: CUDA 13 needs R580+; for R525–R57x (e.g. driver 550 with a GTX 1650) build the CUDA 12 variant, which CUDA minor-version compatibility runs on any R525+ driver:

docker build --pull -t docling:cu12 \
  --build-arg BASE_IMAGE=nvidia/cuda:12.9.1-cudnn-runtime-ubuntu24.04 \
  --build-arg TORCH_INSTALL_ARGS="--torch-backend=cu126" .

Keep the ubuntu24.04 base (the image assumes noble's Python/tesseract paths). The groups: cannot find name for group ID ... message in shell mode is cosmetic: CDI injects host video/render groups that have no names inside the container.

Inconsistent number of pages: X!=Y, Failed to load page. (PdfiumError), Added N failed/skipped pages to document: the PDF itself is malformed: docling-parse and pdfium disagree on the page tree (common in rebuilt or scraped files). max keeps going by design: it converts every readable page, records one error per unreadable page, exits with a partial success warning listing them, and still writes the exports. To recover the missing pages, repair the file first and re-run: qpdf input.pdf repaired.pdf (rebuilds the xref; add --object-streams=generate for stubborn files), or mutool clean input.pdf repaired.pdf, or Ghostscript's pdfwrite.

CPU runs of scanned books are slow, and that is the floor, not a bug: on born-digital PDFs the text layer is used directly and OCR touches only bitmap regions (near-free). On scanned pages every page is a bitmap, so OCR runs in full, roughly tens of seconds per page on a desktop CPU, versus a GPU cutting that by an order of magnitude. The other large CPU cost is the code/formula VLM, ~30 s per detected formula image on CPU; pass --no-enrich to skip it for bulk CPU jobs, and --no-compile for one-shot runs. max never force-OCRs pages that have a text layer.

--pipeline vlm (GraniteDocling) and --device: earlier docling releases ignored --device in the stock VLM pipeline (the VLM branch built VlmPipelineOptions without the flag-derived accelerator_options). That was fixed upstream in docling 2.103.0 (#3599), so on 2.107.0 the stock --pipeline vlm --device cpu is honored directly; no workaround needed. If you run an older image, pass the device as a container env (docker run -e DOCLING_DEVICE=cpu … --pipeline vlm …) or hide the GPU with -e CUDA_VISIBLE_DEVICES=. max mode passes accelerator options into every stage explicitly and has always honored --device (verified by source and a device-propagation regression test).

CUDA available : False in the selftest (Docker): the container can't see the GPU: check --gpus all, the host driver (R580+ for this CUDA 13 image), and that nvidia-container-toolkit is configured for Docker.

max --device cuda on a host with no GPU: max resolves the device once and, if CUDA was requested but is unavailable, falls back to auto (best-available, typically CPU) with a warning; a batch never crashes mid-run. Note this is a deliberate divergence: since docling 2.104.0 (#3606) the stock CLI hard-fails an explicit unavailable --device with AcceleratorDeviceNotAvailableError. Want the strict behavior in max? Don't pass --device cuda on a CPU host; pass --device auto for best-effort or --device cpu to be explicit.

Performance & accuracy knobs

• OMP_NUM_THREADS (default 4): raise on big CPU hosts; feeds Docling's AcceleratorOptions.num_threads. In max mode, --num-threads N sets the same value from the CLI and overrides this env var.
• Pin a GPU at the container level: --gpus '"device=1"' / CUDA_VISIBLE_DEVICES=1 (podman: --device nvidia.com/gpu=1).
• DOCLING_PERF_PAGE_BATCH_SIZE, DOCLING_PERF_ELEMENTS_BATCH_SIZE: page / enrichment batching for the stock CLI (max mode has its own flags).
• torch.compile is ON by default (settings.inference.compile_torch_models) for Docling's VLM/engine stages; the image ships g++ so the inductor backend always has a host compiler. First call per model is slower (compilation), steady-state is faster. Disable with DOCLING_INFERENCE_COMPILE_TORCH_MODELS=false for one-shot jobs.
• DOCLING_CUDA_USE_FLASH_ATTENTION2=true exists upstream but requires the flash-attn package, which needs a CUDA devel toolchain to build, out of scope for this runtime image; leave it unset.
• PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True is preset to reduce fragmentation OOMs during batched inference.

License

This project uses MIT license.