From 805145d8df2e31bf6e7c77cce094e8c9fb4c9646 Mon Sep 17 00:00:00 2001 From: Yingge He Date: Tue, 19 May 2026 18:14:17 -0700 Subject: [PATCH] Remove Windows --- README.md | 1 - docs/client_guide/in_process.rst | 11 +-- docs/customization_guide/build.md | 113 +++-------------------- docs/customization_guide/deploy.md | 12 --- docs/protocol/extension_shared_memory.md | 4 +- docs/user_guide/trace.md | 2 - 6 files changed, 18 insertions(+), 125 deletions(-) diff --git a/README.md b/README.md index b2f3d818b0..4db3a159c7 100644 --- a/README.md +++ b/README.md @@ -134,7 +134,6 @@ images. - [Install Triton Inference Server without Docker containers](docs/customization_guide/build.md#building-without-docker) - [Build a custom Triton Inference Server Docker container](docs/customization_guide/compose.md) - [Build Triton Inference Server from source](docs/customization_guide/build.md#building-on-unsupported-platforms) -- [Build Triton Inference Server for Windows 10](docs/customization_guide/build.md#building-for-windows-10) - Examples for deploying Triton Inference Server with Kubernetes and Helm on [GCP](deploy/gcp/README.md), [AWS](deploy/aws/README.md), and [NVIDIA FleetCommand](deploy/fleetcommand/README.md) - [Secure Deployment Considerations](docs/customization_guide/deploy.md) diff --git a/docs/client_guide/in_process.rst b/docs/client_guide/in_process.rst index 56ab778440..13d32af5e8 100644 --- a/docs/client_guide/in_process.rst +++ b/docs/client_guide/in_process.rst @@ -1,5 +1,5 @@ .. -.. Copyright 2024-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +.. Copyright 2024-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. .. .. Redistribution and use in source and binary forms, with or without .. modification, are permitted provided that the following conditions @@ -35,11 +35,10 @@ allows Triton to be linked directly into a C/C++/java/python application. This A is called the "Triton Server API" or just "Server API" for short. The API is implemented in the Triton shared library which is built from source contained in the `core -repository `__. On Linux -this library is libtritonserver.so and on Windows it is -tritonserver.dll. In the Triton Docker image the shared library is -found in /opt/tritonserver/lib. The header file that defines and -documents the Server API is +repository `__. On Linux this +library is libtritonserver.so. In the Triton Docker image the shared library is +found in /opt/tritonserver/lib. The header file that defines and documents the +Server API is `tritonserver.h `__. `Java bindings for In-Process Triton Server API <../customization_guide/inprocess_java_api.html#java-bindings-for-in-process-triton-server-api>`__ are built on top of `tritonserver.h` and can be used for Java applications that diff --git a/docs/customization_guide/build.md b/docs/customization_guide/build.md index 948fbafb06..48495c7211 100644 --- a/docs/customization_guide/build.md +++ b/docs/customization_guide/build.md @@ -59,8 +59,6 @@ to build Triton on a platform that is not listed here. * [Ubuntu 22.04, x86-64](#building-for-ubuntu-2204) -* [Windows 10, x86-64](#building-for-windows-10) - If you are developing or debugging Triton, see [Development and Incremental Builds](#development-and-incremental-builds) for information on how to perform incremental build. @@ -261,90 +259,6 @@ For a given version of Triton you can attempt to build with non-supported versions of TensorRT but you may have build or execution issues since non-supported versions are not tested. -## Building for Windows 10 - -For Windows 10, build.py supports both a Docker build and a non-Docker -build in a similar way as described for [Ubuntu](#building-for-ubuntu-2204). The primary -difference is that the minimal/base image used as the base of -Dockerfile.buildbase image can be built from the provided -[Dockerfile.win10.min](https://github.com/triton-inference-server/server/blob/main/Dockerfile.win10.min) -file as described in [Windows 10 "Min" Image](#windows-10-min-image). When running build.py -use the --image flag to specify the tag that you assigned to this -image. For example, --image=base,win10-py3-min. - -### Windows and Docker - -Depending on your version of Windows 10 and your version of Docker you -may need to perform these additional steps before any of the following -step. - -* Set your Docker to work with "Windows containers". Right click on - the whale icon in the lower-right status area and select "Switch to - Windows containers". - -### Windows 10 "Min" Image - -The "min" container describes the base dependencies needed to perform -the Windows build. The Windows min container is -[Dockerfile.win10.min](https://github.com/triton-inference-server/server/blob/main/Dockerfile.win10.min). - -Before building the min container you must download the appropriate -cuDNN and TensorRT versions and place them in the same directory as -Dockerfile.win10.min. - -* For cuDNN the CUDNN_VERSION and CUDNN_ZIP arguments defined in - Dockerfile.win10.min indicate the version of cuDNN that your should - download from https://developer.nvidia.com/rdp/cudnn-download. - -* For TensorRT the TENSORRT_VERSION and TENSORRT_ZIP arguments defined - in Dockerfile.win10.min indicate the version of TensorRT that your - should download from - https://developer.nvidia.com/nvidia-tensorrt-download. - -After downloading the zip files for cuDNN and TensorRT, you build the -min container using the following command. - -```bash -$ docker build -t win10-py3-min -f Dockerfile.win10.min . -``` - -### Build Triton Server - -Triton is built using the build.py script. The build system must have -Docker, Python3 (plus pip installed *docker* module) and git installed -so that it can execute build.py and perform a docker build. By -default, build.py does not enable any of Triton's optional features -and so you must enable them explicitly. The following build.py -invocation builds all features and backends available on windows. - -```bash -python build.py --cmake-dir=/build --build-dir=/tmp/citritonbuild --no-container-pull --image=base,win10-py3-min --enable-logging --enable-stats --enable-tracing --enable-gpu --endpoint=grpc --endpoint=http --repo-tag=common: --repo-tag=core: --repo-tag=backend: --repo-tag=thirdparty: --backend=ensemble --backend=tensorrt: --backend=onnxruntime: --backend=openvino: --backend=python: -``` - -If you are building on *main* branch then `` will -default to "main". If you are building on a release branch then -`` will default to the branch name. For example, if you -are building on the r24.12 branch, `` will default to -r24.12. Therefore, you typically do not need to provide `` at all (nor the preceding colon). You can use a different -`` for a component to instead use the corresponding -branch/tag in the build. For example, if you have a branch called -"mybranch" in the -[onnxruntime_backend](https://github.com/triton-inference-server/onnxruntime_backend) -repo that you want to use in the build, you would specify ---backend=onnxruntime:mybranch. - -### Extract Build Artifacts - -When build.py completes, a Docker image called *tritonserver* will -contain the built Triton Server executable, libraries and other -artifacts. Windows containers do not support GPU access so you likely -want to extract the necessary files from the tritonserver image and -run them directly on your host system. All the Triton artifacts can be -found in /opt/tritonserver directory of the tritonserver image. Your -host system will need to install the CUDA, cuDNN, TensorRT and other -dependencies that were used for the build. - ## Building on Unsupported Platforms Building for an unsupported OS and/or hardware platform is @@ -409,8 +323,8 @@ and cmake_build or the equivalent commands to perform a build. If you are [building without Docker](#building-without-docker) use the CMake invocation steps in cmake_build to invoke CMake to set-up a -build environment where you can invoke make/msbuild.exe to incremental -build the Triton core, a backend, or a repository agent. +build environment where you can invoke make to incremental build the Triton +core, a backend, or a repository agent. ### Development Builds With Docker @@ -423,8 +337,7 @@ agents. To perform an incremental build within the *tritonserver_buildbase* container, map your source into the container and then run the -appropriate CMake and `make` (or `msbuild.exe`) steps from cmake_build -within the container. +appropriate CMake and `make` steps from cmake_build within the container. #### Development Build of Triton Core @@ -450,10 +363,9 @@ CMakeLists.txt file and source: $ cmake /server ``` -Then you can change directory into the build directory and run `make` -(or `msbuild.exe`) as shown in cmake_build. As you make changes to the -source on your host system, you can perform incremental builds by -re-running `make` (or `msbuild.exe`). +Then you can change directory into the build directory and run `make` as shown +in cmake_build. As you make changes to the source on your host system, you can +perform incremental builds by re-running `make`. #### Development Build of Backend or Repository Agent @@ -467,10 +379,8 @@ incremental builds to test those changes. Your source code is in /home/me/tritonserver_backend. Run the *tritonserver_buildbase* container and map your TensorRT backend source directory into the container at /tensorrt_backend. Note that some backends will use -Docker as part of their build, and so the host's Docker registry must -be made available within the *tritonserver_buildbase* by mounting -docker.sock (on Windows use --v\\.\pipe\docker_engine:\\.\pipe\docker_engine). +Docker as part of their build, and so the host's Docker registry must be made +available within the *tritonserver_buildbase* by mounting docker.sock. ``` $ docker run -it --rm -v/var/run/docker.sock:/var/run/docker.sock -v/home/me/tensorrt_backend:/tensorrt_backend tritonserver_buildbase bash @@ -487,10 +397,9 @@ CMakeLists.txt file and source: $ cmake /tensorrt_backend ``` -Then you can change directory into the build directory and run `make` -(or `msbuild.exe`) as shown in cmake_build. As you make changes to the -source on your host system, you can perform incremental builds by -re-running `make` (or `msbuild.exe`). +Then you can change directory into the build directory and run `make` as shown +in cmake_build. As you make changes to the source on your host system, you can +perform incremental builds by re-running `make`. ### Building with Debug Symbols diff --git a/docs/customization_guide/deploy.md b/docs/customization_guide/deploy.md index 2a305de5d3..2a9553bb2f 100644 --- a/docs/customization_guide/deploy.md +++ b/docs/customization_guide/deploy.md @@ -279,18 +279,6 @@ Directory where cache shared libraries are found. > must be access controlled. Adding untrusted files > can lead to arbitrarty code execution. -##### `backend-config=,additional-dependency-dirs=` - -This is an optional Windows feature that enables Triton to search custom -dependency directories when loading a specific backend. The user can input -these directories as a string of semicolon-separated paths (including a -trailing semicolon). These directories are programmatically prepended to -the process's PATH and are removed when the backend is loaded successfully. -Windows will search PATH last in its search sequence, so be cautious that -no untrusted files of same name exist in a location of higher search priority -(e.g., System32). It is still recommended to add backend-specific dependencies -to their corresponding backend folder when possible. - # GRPC server options Triton Inference Server's gRPC inference handlers internally use states to manage inference requests and response queues. Each state consists of one inference request and one response queue. The response queue within a state can hold multiple response objects. These states remain allocated for reuse to optimize performance by minimizing dynamic allocations. diff --git a/docs/protocol/extension_shared_memory.md b/docs/protocol/extension_shared_memory.md index cb5d9ccc7f..d7ae7c045e 100644 --- a/docs/protocol/extension_shared_memory.md +++ b/docs/protocol/extension_shared_memory.md @@ -64,8 +64,8 @@ given Triton will return an error. > [!NOTE] > -> Shared memory is not supported on Windows. On Jetson, only system shared -> memory is supported (CUDA shared memory is not). +> On Jetson, only system shared memory is supported (CUDA shared memory is +> not). ## HTTP/REST diff --git a/docs/user_guide/trace.md b/docs/user_guide/trace.md index 2f3413385d..8db98e7705 100644 --- a/docs/user_guide/trace.md +++ b/docs/user_guide/trace.md @@ -641,8 +641,6 @@ spans are properly ended. ### Limitations -- OpenTelemetry trace mode is not supported on Windows systems. - - Triton supports only [OTLP/HTTP Exporter](https://opentelemetry.io/docs/specs/otlp/#otlphttp) and allows specification of only url for this exporter through