DeepSeek-OCR vLLM Docker

A self-contained Dockerized FastAPI wrapper for DeepSeek-OCR running on vLLM 0.14.1.

This repository provides:

• A FastAPI service with image + PDF OCR endpoints.
• A sync batch endpoint for multi-file OCR.
• A Docker build based on vllm/vllm-openai:v0.14.1.

Model weights come from the official DeepSeek OCR repository:

• https://github.com/deepseek-ai/DeepSeek-OCR

Prerequisites

• NVIDIA GPU with drivers supporting CUDA >= 12.9
• Docker + NVIDIA Container Toolkit (GPU support)
• Optional: Hugging Face token for faster model downloads (HF_TOKEN)

Quickstart

One-command start:

./scripts/quickstart.sh

Requires Hugging Face CLI (hf/huggingface-cli). If you don’t have it, the script installs it into .venv. Set INSTALL_EXTRAS=1 to install optional host dependencies for goodies/scripts. Set INSTALL_DEPS=1 to install host dependencies for client scripts.

Or manual steps:

1. Install the Hugging Face CLI (optional helper)

Example using Hugging Face CLI:

mkdir -p models/deepseek-ai/DeepSeek-OCR
hf download deepseek-ai/DeepSeek-OCR \
  --local-dir models/deepseek-ai/DeepSeek-OCR

Or use the helper script:

# Optional: Pin a specific revision
# export MODEL_REVISION="..." 
./scripts/download_weights.sh

Notes:

• huggingface-cli download ... also works if you use the legacy CLI name.
• If hf/huggingface-cli isn’t installed, use a venv or pipx:
  • python3 -m venv .venv && . .venv/bin/activate && pip install -U huggingface_hub
  • pipx install huggingface_hub
• Set HF_TOKEN=... for faster downloads and higher rate limits.
• Optional helper install:

  make setup
  # or: ./scripts/install_hf.sh

2. Configure environment

cp .env.example .env
# edit .env if you need different paths or GPU selection

3. Build and run

docker compose up --build

The API will be available at http://localhost:8000 by default.

API Endpoints

• GET /health
• POST /ocr/image (multipart form: file, optional prompt)
• POST /ocr/pdf (multipart form: file, optional prompt)
• POST /ocr/batch (multipart form: repeated files, optional prompt)
  • Note: Aggregates all images/pages from all uploaded files into a single GPU batch for high throughput.

Example: OCR a PDF

curl -s -F "file=@examples/sample.pdf" http://localhost:8000/ocr/pdf

Optional: pipe to jq for pretty output.

Example: OCR an image

curl -s -F "file=@examples/sample.png" http://localhost:8000/ocr/image

Example: Python client (sync)

python3 -m venv .venv
. .venv/bin/activate
pip install -r requirements-extra.txt
./scripts/ocr_client.py examples/sample.pdf --url http://localhost:8000

Example: JavaScript client (sync)

node -e "const fs=require('fs'); const fd=new FormData(); fd.append('file', fs.createReadStream('examples/sample.pdf')); fetch('http://localhost:8000/ocr/pdf',{method:'POST',body:fd}).then(r=>r.json()).then(console.log).catch(console.error)"

Optional Extras

• Optional batch processing script and advanced options live under goodies/README.md.
• List all extras via:

  ./scripts/extra.sh

Known Limits

To prevent memory exhaustion and timeouts, the following soft limits are enforced by default (configurable via .env):

• PDF Pages: Max 50 pages per request (DEEPSEEK_OCR_MAX_PDF_PAGES).
• PDF Size: Max 50 MB (DEEPSEEK_OCR_MAX_PDF_BYTES).
• Image Size: Max 20 MB (DEEPSEEK_OCR_MAX_IMAGE_BYTES).

Note: Higher DEEPSEEK_OCR_PDF_DPI improves quality but increases latency and memory use.

Configuration

All configuration is via environment variables (see .env.example).

Highlights:

• MODEL_PATH (container path): must point to /app/models/deepseek-ai/DeepSeek-OCR.
• CUDA_VISIBLE_DEVICES: GPU selection.
• MAX_CONCURRENCY / MAX_NUM_SEQS: vLLM max concurrent sequences.
• MAX_MODEL_LEN: set a specific integer context length, or use auto / -1 for vLLM auto-detection.
• GPU_MEMORY_UTILIZATION: fraction of GPU memory to use.
• VLLM_WORKER_MULTIPROC_METHOD: multiprocessing start method (default spawn).
• Request limits: DEEPSEEK_OCR_MAX_PDF_PAGES, DEEPSEEK_OCR_MAX_PDF_BYTES, DEEPSEEK_OCR_MAX_IMAGE_BYTES.
• Prompt overrides:
  • DEEPSEEK_OCR_PROMPT (global override)
  • DEEPSEEK_OCR_PDF_PROMPT
  • DEEPSEEK_OCR_IMAGE_PROMPT

Outputs and Logs

• The service itself does not write OCR artifacts to disk. Output files are only written by the optional batch script.
• Logs go to stdout. Use docker compose logs -f to view them.

Extension Points

• Add async batch processing backed by Redis (or a DB) when you need long-running jobs, high throughput, or client polling without holding open requests.
• Add API auth or rate limiting if exposed publicly.
• Persist OCR results to object storage or a database.
• Tune prompts and max tokens for your document types.

Goodies (Optional)

All optional extras are indexed in goodies/README.md.

Notes

• This service is designed for a single GPU container.
• vLLM 0.14.1 is pinned by the base image; upgrade with care if you change the base.
• The base image provides PyTorch 2.9.1. If you run outside Docker, use torch 2.9.1.
• The default prompts are in deepseek_ocr_service/engine.py.

Smoke Test

make test

Slow GPUs or first-time model loads can take longer; set SMOKE_MAX_WAIT_SECONDS=... for more patience.

Or manually:

curl -s http://localhost:8000/health
curl -s -F "file=@examples/sample.pdf" http://localhost:8000/ocr/pdf

Or use the helper:

./scripts/smoke_test.sh

Windows Helper (Untested)

Windows helper lives at goodies/windows/build.bat.

Contributing

See CONTRIBUTING.md for development setup and guidelines.

License

This project is licensed under the MIT License - see the LICENSE file for details.