Skip to content

building-from-source.md

Latest commit

 

History

History
167 lines (111 loc) · 5.2 KB

building-from-source.md

File metadata and controls

167 lines (111 loc) · 5.2 KB
Raw
title Building from Source
sidebar-title Building from Source
description Build Dynamo from source for development and contributions

Build Dynamo from source when you want to contribute code, test features on the development branch, or customize the build. If you just want to run Dynamo, the Local Installation guide is faster.

This guide covers Ubuntu and macOS. For a containerized dev environment that handles all of this automatically, see DevContainer.

1. Install System Libraries

Ubuntu:

sudo apt install -y build-essential libhwloc-dev libudev-dev pkg-config libclang-dev protobuf-compiler python3-dev cmake

macOS:

# Install Homebrew if needed
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

brew install cmake protobuf

# Verify Metal is accessible
xcrun -sdk macosx metal

2. Install Rust

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env

3. Create a Python Virtual Environment

Install uv if you don't have it:

curl -LsSf https://astral.sh/uv/install.sh | sh

Create and activate a virtual environment:

uv venv .venv
source .venv/bin/activate

4. Install Build Tools

uv pip install pip 'maturin[patchelf]'

Maturin is the Rust-Python bindings build tool. The patchelf extra lets maturin patch native extension library paths during the build.

5. Build the Rust Bindings

cd lib/bindings/python
maturin develop --uv

6. Install GPU Memory Service

# Return to project root
cd "$(git rev-parse --show-toplevel)"
uv pip install -e lib/gpu_memory_service

7. Install the Wheel

Install Dynamo with a backend extra to pull the inference engine and its CUDA dependencies. Choose the backend you intend to run:

# Use .[vllm] or .[sglang] instead to install the relevant framework dependencies
uv pip install -e .

Note

The base uv pip install -e . installs only the Dynamo runtime and frontend. A backend extra ([vllm], or [sglang]) will install the relevant framework dependencies to run an inference worker. For the TensorRT-LLM backend, use the tensorrtllm-runtime container instead of installing via uv pip to ensure the right dependencies are installed. See Local Installation for more details.

8. Verify the Build

python3 -m dynamo.frontend --help

You should see the frontend command help output.

DevContainer

VSCode and Cursor users can skip manual setup using pre-configured development containers. The DevContainer installs all toolchains, builds the project, and sets up the Python environment automatically.

Framework-specific containers are available for vLLM, SGLang, and TensorRT-LLM. See the DevContainer README for setup instructions.

Set Up Pre-commit Hooks

Before submitting PRs, install the pre-commit hooks to ensure your code passes CI checks:

uv pip install pre-commit
pre-commit install

Run checks manually on all files:

pre-commit run --all-files

Troubleshooting

Missing system packages

If maturin develop fails with linker errors, verify all system dependencies are installed. On Ubuntu:

sudo apt install -y build-essential libhwloc-dev libudev-dev pkg-config libclang-dev protobuf-compiler python3-dev cmake

Virtual environment not activated

Maturin builds against the active Python interpreter. If you see errors about Python or site-packages, ensure your virtual environment is activated:

source .venv/bin/activate

Disk space

The Rust target/ directory can grow to 10+ GB during development. If builds fail with disk space errors, clean the build cache:

cargo clean

vLLM worker fails to start: FlashInfer sampler JIT and CUDA 13 wheels

When you run a vLLM worker from a CUDA 13 source install, the worker can abort during startup with a FlashInfer JIT error:

RuntimeError: Engine core initialization failed.
...
cuda/std/__cccl/cuda_toolkit.h:41: error: "CUDA compiler and CUDA toolkit headers are incompatible"

The CUDA wheels resolved for a CUDA 13 install can be version-skewed: torch pins the runtime headers to 13.0, while vLLM's tilelang dependency pulls nvidia-cuda-nvcc 13.2. FlashInfer compiles its sampler kernel with nvcc against those headers, and the version mismatch fails the build. This is tracked upstream at flashinfer#3493.

Set VLLM_USE_FLASHINFER_SAMPLER=0 so vLLM falls back to its native sampler:

export VLLM_USE_FLASHINFER_SAMPLER=0

Next Steps