Model Analyzer fails when profiling Triton model deployed with vLLM backend

[AmilkardelosMonteros]

Hi everyone. I am trying to profile a model deployed on Triton Inference Server using the vLLM backend, but Model Analyzer fails during execution.

It is unclear whether this is due to a configuration issue, lack of support for vLLM, or a bug in Model Analyzer.

Environment

• Triton Inference Server version: 26.03
• Model Analyzer version: 1.52.0
• GPU: RTX 5090
• CUDA version: 13.1

Steps to Reproduce

For easy reproduce, I'm using the instructions for deploy a vllm model on triton:

mkdir -p model_repository/vllm_model/1
wget -P model_repository/vllm_model/1 https://raw.githubusercontent.com/triton-inference-server/vllm_backend/r26.03/samples/model_repository/vllm_model/1/model.json
wget -P model_repository/vllm_model/ https://raw.githubusercontent.com/triton-inference-server/vllm_backend/r26.03/samples/model_repository/vllm_model/config.pbtxt

docker pull nvcr.io/nvidia/tritonserver:26.03-py3-sdk

cat < model_repository/tmp_config.yaml

run_config_search_mode: quick
model_type: LLM
client_protocol: grpc

genai_perf_flags:
backend: vllm
streaming: true
EOF

docker run -it --gpus all
-v /var/run/docker.sock:/var/run/docker.sock
-v $(pwd)/model_repository:/workspace/model_repository
--net=host
--entrypoint bash
nvcr.io/nvidia/tritonserver:26.03-py3-sdk
-c "
model-analyzer profile
--v
-f /workspace/model_repository/tmp_config.yaml
--model-repository /workspace/model_repository
--profile-models vllm_model --triton-launch-mode=docker
--output-model-repository-path output
--export-path profile_results
"

The error

02:52:45 [Model Analyzer] Initializing GPUDevice handles
02:52:47 [Model Analyzer] Using GPU 0 NVIDIA GeForce RTX 5090 with UUID GPU-ac4c1f94-bdc6-7ddc-2412-2aa63d0e7253
02:52:47 [Model Analyzer] Starting a Triton Server using docker
02:52:47 [Model Analyzer] No checkpoint file found, starting a fresh run.
02:52:47 [Model Analyzer] Profiling server only metrics...
02:52:47 [Model Analyzer] DEBUG: Triton Server started.
02:52:47 [Model Analyzer] DEBUG: Failed to connect to Triton server (attempt 1/50): [StatusCode.UNAVAILABLE] failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:8001: Failed to connect to remote host: connect: Connection refused (111)
02:53:04 [Model Analyzer] DEBUG: Stopped Triton Server.
02:53:04 [Model Analyzer]
02:53:04 [Model Analyzer] Starting quick mode search to find optimal configs
02:53:04 [Model Analyzer]
02:53:04 [Model Analyzer] Creating model config: vllm_model_config_default
02:53:04 [Model Analyzer]
02:53:05 [Model Analyzer] DEBUG: Triton Server started.
02:53:05 [Model Analyzer] DEBUG: Failed to connect to Triton server (attempt 1/50): [StatusCode.UNAVAILABLE] failed to connect to all addresses; last error: UNAVAILABLE: ipv4:127.0.0.1:8001: recvmsg:Connection reset by peer
02:53:09 [Model Analyzer] Model vllm_model_config_default load failed: [StatusCode.INTERNAL] failed to load 'vllm_model', failed to poll from model repository
02:53:13 [Model Analyzer] DEBUG: Stopped Triton Server.
02:53:13 [Model Analyzer] DEBUG: Measurement for [0]: None.
02:53:13 [Model Analyzer] Saved checkpoint to /workspace/checkpoints/0.ckpt
02:53:13 [Model Analyzer] Creating model config: vllm_model_config_0
02:53:13 [Model Analyzer] Setting instance_group to [{'count': 1, 'kind': 'KIND_GPU'}]
02:53:13 [Model Analyzer]
02:53:14 [Model Analyzer] DEBUG: Triton Server started.
02:53:14 [Model Analyzer] DEBUG: Failed to connect to Triton server (attempt 1/50): [StatusCode.UNAVAILABLE] failed to connect to all addresses; last error: UNAVAILABLE: ipv4:127.0.0.1:8001: recvmsg:Connection reset by peer
02:53:18 [Model Analyzer] Model vllm_model_config_0 load failed: [StatusCode.INTERNAL] failed to load 'vllm_model', failed to poll from model repository
02:53:24 [Model Analyzer] DEBUG: Stopped Triton Server.
02:53:24 [Model Analyzer] No changes made to analyzer data, no checkpoint saved.
Traceback (most recent call last):
File "/usr/local/bin/model-analyzer", line 8, in
sys.exit(main())
^^^^^^
File "/usr/local/lib/python3.12/dist-packages/model_analyzer/entrypoint.py", line 281, in main
analyzer.profile(
File "/usr/local/lib/python3.12/dist-packages/model_analyzer/analyzer.py", line 131, in profile
self._profile_models()
File "/usr/local/lib/python3.12/dist-packages/model_analyzer/analyzer.py", line 255, in _profile_models
self._model_manager.run_models(models=[model])
File "/usr/local/lib/python3.12/dist-packages/model_analyzer/model_manager.py", line 157, in run_models
self._stop_ma_if_no_valid_measurement_threshold_reached()
File "/usr/local/lib/python3.12/dist-packages/model_analyzer/model_manager.py", line 251, in _stop_ma_if_no_valid_measurement_threshold_reached
raise TritonModelAnalyzerException(
model_analyzer.model_analyzer_exceptions.TritonModelAnalyzerException: The first 2 attempts to acquire measurements have failed. Please examine the Tritonserver/PA error logs to determine what has gone wrong.

I also tried add the vllm backend but I found the same error.