Skip to content

Add option to limit ONNX Runtime build parallelism - #342

Open
LeC911 wants to merge 1 commit into
triton-inference-server:mainfrom
LeC911:add-ort-build-parallel
Open

Add option to limit ONNX Runtime build parallelism#342
LeC911 wants to merge 1 commit into
triton-inference-server:mainfrom
LeC911:add-ort-build-parallel

Conversation

@LeC911

@LeC911 LeC911 commented May 12, 2026

Copy link
Copy Markdown

Motivation

When building the ONNX Runtime backend through Triton build.py, the top-level --build-parallel option limits the outer CMake build, but the generated Dockerfile.ort still invokes ONNX Runtime build as:

./build.sh --parallel

without a job count.

On resource-constrained builders this can make the ORT build fail while building libonnxruntime.so, even when Triton is built with --build-parallel=1.

Change

This PR adds an optional CMake cache variable:

TRITON_BUILD_ONNXRUNTIME_PARALLEL

When set, it is forwarded to tools/gen_ort_dockerfile.py as:

--ort-build-parallel=<N>

and the generated Dockerfile.ort uses:

./build.sh --parallel <N>

Default behavior is unchanged. If the option is not set, the generated build continues to use:

./build.sh --parallel

The original issue was reproduced on r26.04 while building Triton 2.68.0/container 26.04 with ONNX Runtime backend, CPU-only, HTTP-only, GPU disabled.

Additional validation

Verified opt-in behavior using this PR branch through Triton build.py:

--extra-backend-cmake-arg=onnxruntime:TRITON_BUILD_ONNXRUNTIME_PARALLEL=2

The generated Dockerfile.ort contains:

ARG COMMON_BUILD_ARGS="--config ${ONNXRUNTIME_BUILD_CONFIG} --parallel 2 --skip_submodule_sync --build_shared_lib ..."

Also verified default behavior remains unchanged when TRITON_BUILD_ONNXRUNTIME_PARALLEL is not set. With only the outer build parallelism set, the generated Dockerfile.ort still contains:

ARG COMMON_BUILD_ARGS="--config ${ONNXRUNTIME_BUILD_CONFIG} --parallel --skip_submodule_sync --build_shared_lib ..."

@LeC911
LeC911 force-pushed the add-ort-build-parallel branch from e55062f to 756346c Compare May 12, 2026 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant