hart-thor-docker

Container packaging for HART (Hybrid Autoregressive Transformer) text-to-image generation, plus a compute-capability fix needed to build it at all on Thor/Blackwell-generation GPUs.

Host specs this was built and tested on: NVIDIA Thor · 124 GB unified memory · Driver 580.00 · CUDA 13.0 · Architecture: linux/arm64

Not affiliated with MIT HAN Lab. HART is by the MIT HAN Lab (MIT license): https://github.com/mit-han-lab/hart

---

Why this exists

HART had a public web demo; it went down at some point, and the upstream repo alone doesn't get you a running local setup — there's no Docker packaging, and (as of this writing) HART's own kernel build script doesn't know Blackwell- generation compute capabilities exist:

• hart/kernels/setup.py's SUPPORTED_ARCHS allowlist predates Blackwell and rejects TORCH_CUDA_ARCH_LIST=11.0 (Thor) outright.
• Even past that, its arch-string-to-nvcc-flag parser (capability[0] + capability[2]) only works for single-digit compute- capability majors — it silently produces "1." instead of "110" for "11.0", which would generate broken -gencode flags.

Both are fixed by patches/0001-thor-blackwell-sm110-arch-support.patch, applied automatically by build.sh. This isn't Thor-specific in principle — anything with a two-digit compute-capability major (RTX 50-series, other Blackwell-generation datacenter GPUs) hits the same wall.

Payoff of building natively instead of relying on PTX JIT from a Hopper (9.0+PTX) target: cold-start latency dropped ~63% (40.8s → 14.9s) and warm steady-state throughput improved ~11% (~14.1s → ~12.5s), measured end-to-end through a real generation request on Thor.

---

What this repo does and doesn't include

• Includes: Dockerfile (two-stage: compile kernels, then install the package + CLI), build.sh, the arch-compatibility patch, and run_hart_cli.py — a minimal CLI wrapper written for this packaging (see below for why it exists and what it deliberately omits).
• Does NOT include HART's source or any model weights. You clone HART and download weights yourself (below); build.sh applies the patch to your clone automatically — don't pre-patch it.

---

About run_hart_cli.py and the safety checker

Upstream HART's own sample.py gates generation behind a "shield model" safety classifier (google/shieldgemma-2b, prompted to judge whether a request violates a content policy). That gate wasn't practically available to us when building this, so run_hart_cli.py is a from-scratch minimal CLI that does not include a safety-checker gate at all — it's not upstream's script with the check patched out, it's a separate, simpler implementation that never had one. If you want content-policy filtering, you need to add it yourself — see upstream's sample.py and hart/utils/safety_check.py for the reference implementation and adapt it to whatever gate you have available.

---

Prerequisites

• Docker + the NVIDIA container runtime (runtime: nvidia or --gpus all; on Jetson this is the default).
• A clone of mit-han-lab/hart.
• Model weights (see upstream's README for the authoritative instructions):

  git clone https://huggingface.co/mit-han-lab/hart-0.7b-1024px
  git clone https://huggingface.co/mit-han-lab/Qwen2-VL-1.5B-Instruct

---

Quick start

git clone https://github.com/<you>/hart-thor-docker && cd hart-thor-docker
cp .env.example .env && $EDITOR .env   # set HART_SRC to your hart clone
./build.sh                             # stages the source, applies the patch,
                                        # builds hart-cli:thor, verifies SASS
docker run --rm --entrypoint which hart-cli:thor hart-txt2img

Run a generation directly (mount your model + text-encoder dirs and an output dir; adjust host paths to match where you cloned the weights above):

docker run --rm --gpus all --ipc host \
  -v /path/to/hart-0.7b-1024px/llm:/models/hart-llm:ro \
  -v /path/to/Qwen2-VL-1.5B-Instruct:/models/qwen:ro \
  -v /path/to/outputs:/workspace/outputs \
  hart-cli:thor \
  --model_path /models/hart-llm \
  --text_model_path /models/qwen \
  --prompt "a red bicycle leaning against a brick wall, golden hour lighting" \
  --seed 42 \
  --out_dir /workspace/outputs

This is a one-shot CLI, not a server — it loads the model, generates, saves a PNG, and exits. For a persistent HTTP API + browser UI on top of this image, see the companion hart-web project (a thin Nuxt/Nitro wrapper that spawns hart-txt2img per request).

---

Configuration (.env)

Var	Meaning
HART_SRC	Path to your mit-han-lab/hart clone (patched automatically by build.sh)
HART_CUDA_ARCH	TORCH_CUDA_ARCH_LIST value — 11.0 for Thor/Blackwell (default), 9.0 for Hopper

Check your GPU's compute capability with:

nvidia-smi --query-gpu=compute_cap --format=csv

---

CLI reference

hart-txt2img
  --model_path PATH         # required — .../hart-0.7b-1024px/llm
  --text_model_path PATH    # required — .../Qwen2-VL-1.5B-Instruct
  --prompt TEXT              # required
  --out_dir PATH             # default: /workspace/outputs/hart_samples
  --seed N                   # default: 42
  --cfg FLOAT                # default: 4.5
  --max_token_length N       # default: 300
  --no_embed_metadata        # disable PNG iTXt metadata (on by default)

Output filenames follow NNNNN-SEED-HASH.png (zero-padded counter, seed, 8-char prompt hash) and embed an Automatic1111-style parameters block in the PNG by default, for compatibility with SD-WebUI-style tooling.

---

Rebuild after changing HART_SRC or the patch

./build.sh

build.sh re-stages hart-src/ from HART_SRC and rebuilds from scratch every time — there's no incremental/cached mode. It re-applies the patch with patch -N, which is safe to run against an already-patched clone (already- applied hunks are skipped, not double-applied).

---

Rollback / uninstall

docker rmi hart-cli:thor
rm -rf hart-src   # staged build context; safe to delete, re-created by build.sh

---

Credits

• HART / MIT HAN Lab — the actual model and inference code: https://github.com/mit-han-lab/hart
• This repo: container packaging, the Thor/Blackwell arch-compatibility patch, and a minimal CLI wrapper. MIT (see LICENSE).