Description
Cuda SHM module errors out when accessed due to wrong cuda API used.
Triton Information
26.05 with the recommended cuda of 13.2 with it.
https://docs.nvidia.com/deeplearning/frameworks/support-matrix/index.html
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/introduction/compatibility.html#release-compatibility-matrix
Both recommend 13.2 or 13.1. All these version compatibility matrices have been wrongly reported because not all modules have been clearly tested - in this case Shared Memory.
Are you using the Triton container or did you build it yourself?
Built it myself
To Reproduce
Have cuda 13.2 or cuda 13.1 both have the same core cuda API changes, so it will error out when using SHM for both.
Describe the models (framework, inputs, outputs), ideally include the model configuration file (if using an ensemble include the model configuration file for that as well).
NA - will happen on any SHM model config.
Expected behavior
NA
The issue is with line:
|
cudaError_t err = cudaGetDriverEntryPoint(name, func_ptr, cudaEnableDefault); |
It can be replaced with
cudaError_t err = cudaGetDriverEntryPointByVersion(name, func_ptr, 13020, cudaEnableDefault, nullptr);
In a simplified manner.
CUDA Documentation for driver entrypoint getter
Here is the deprecation notice (as of 13.0) for the function you have used currently. It has been superceeded by the function replacement example I have provided.
Description
Cuda SHM module errors out when accessed due to wrong cuda API used.
Triton Information
26.05 with the recommended cuda of 13.2 with it.
https://docs.nvidia.com/deeplearning/frameworks/support-matrix/index.html
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/introduction/compatibility.html#release-compatibility-matrix
Both recommend 13.2 or 13.1. All these version compatibility matrices have been wrongly reported because not all modules have been clearly tested - in this case Shared Memory.
Are you using the Triton container or did you build it yourself?
Built it myself
To Reproduce
Have cuda 13.2 or cuda 13.1 both have the same core cuda API changes, so it will error out when using SHM for both.
Describe the models (framework, inputs, outputs), ideally include the model configuration file (if using an ensemble include the model configuration file for that as well).
NA - will happen on any SHM model config.
Expected behavior
NA
The issue is with line:
server/src/shared_memory_manager.cc
Line 277 in a510ae2
It can be replaced with
In a simplified manner.
CUDA Documentation for driver entrypoint getter
Here is the deprecation notice (as of 13.0) for the function you have used currently. It has been superceeded by the function replacement example I have provided.