modly-trellis2

Microsoft TRELLIS.2 (4B image→3D with PBR materials, GLB output) packaged as a Modly extension. One codebase, three platforms:

Platform	Backend	Status
macOS (Apple Silicon)	MLX / Metal	tested (512³: ~94 s, 2.59M faces)
Linux + NVIDIA	upstream CUDA pipeline	packaged, untested here
Windows + NVIDIA	upstream CUDA pipeline	best-effort — upstream is Linux-tested and some CUDA deps build poorly natively; WSL2 is the supported fallback

Install

scripts/install.sh          # symlink into Modly's extensionsDir + build venv + verify
scripts/install.sh --copy   # detached copy instead of symlink

Requires Modly.app running (backend on 127.0.0.1:8765). The first setup builds an isolated venv inside the extension dir: ~2 GB torch download, two clones of the trellis2 repo, and on macOS four Metal source builds (Xcode command-line tools required) — expect 10–30 minutes.

If Modly is closed the script still installs the files; finish with the model's Repair button in Modly (runs extension/setup.py).

Weights (microsoft/TRELLIS.2-4B, ~15 GB net of skipped encoder checkpoints) download automatically on first load into Modly's models dir.

Hugging Face token (required)

The image conditioner facebook/dinov3-vitl16-pretrain-lvd1689m is gated: request access on huggingface.co and set your HF token in Modly Settings. load() preflights this and fails with instructions otherwise. Background removal uses the public ZhengPeng7/BiRefNet (the generator patches pipeline.json away from the gated briaai/RMBG-2.0 after every download).

How the venv gets the TRELLIS.2 code

The trellis2-apple fork (upstream CUDA pipeline preserved + MLX backend) is pip-installable; the venv installs it as a git requirement. o_voxel installs from the same repo (#subdirectory=o-voxel) in a --no-build-isolation pass, as do the Metal packages (their builds import torch). Baked-in fixes: huggingface-hub<1.0, mtlbvh (undeclared cumesh dep), kornia+timm (BiRefNet remote code), rembg swap above.

Overrides (set when running install.sh or Repair):

TRELLIS2_SRC=~/projects/trellis2-apple scripts/install.sh   # local checkout, editable
TRELLIS2_GIT_URL=https://github.com/bilgili/trellis2-apple.git scripts/install.sh

Params

pipeline_type 512 / 1024_cascade / 1536_cascade · seed · steps · texture_size 1024/2048/4096 · decimation_target (faces).

Notes

• Cancelling a generation kills the extension subprocess (Modly semantics); the model reloads on the next run.
• "Free Memory" in Modly fully reclaims Metal pages (process exit); plain model switches release what MLX/MPS caches allow.
• A Modly update that swaps its embedded Python bricks the venv (pyvenv.cfg points into the app bundle) — fix with Repair.
• Specs live in openspec/ (OpenSpec).