I have tried to analyze my fine-tuned (faster whisper large v3) which is deployed on Triton Inference Server (server version: 2.62.0, and container image: nvcr.io/nvidia/tritonserver:25.10-pyt-python-py3). The following is my model and perf analyzer configuration files content:
config.pbtxt:
name: "faster-whisper-model"
backend: "python"
max_batch_size: 0 # Dynamic batching is complex with current faster-whisper
input [
{
name: "INPUT_AUDIO_FILE"
data_type: TYPE_FP32 # Or TYPE_FP32/TYPE_INT16 if you handle conversion
dims: [ -1 ]
}
]
output [
{
name: "OUTPUT_TEXT"
data_type: TYPE_STRING
dims: [ 1 ]
}
]
instance_group [
{
count: 1
kind: KIND_GPU
gpus: [ 0 ]
}
]
Perf analyzer command used:
perf_analyzer -m faster-whisper-model
-u my_triton_server_url
-i http
-f test_report.csv
--input-data=./audio_data.json
--input-tensor-format=json
--concurrency-range 1:8:1
--profile-export-file profile_data.json
--verbose-csv
audio_data.json file format:
But I got the following error:
Successfully read data for 1 stream/streams with 1 step/steps.
*** Measurement Settings ***
Batch size: 1
Service Kind: TRITON
Using "time_windows" mode for stabilization
Stabilizing using average latency and throughput
Measurement window: 5000 msec
Latency limit: 0 msec
Concurrency limit: 8 concurrent requests
Using synchronous calls for inference
Request concurrency: 1
Failed to maintain requested inference load. Worker thread(s) failed to generate concurrent requests.
Thread [0] had error: Unable to parse 'data': Shape does not match true shape of 'data' field
Please help me to deal with this problem.
My goal is to send multiple requests (audio files) into triton server and analyze my triton server performance.
I have tried to analyze my fine-tuned (faster whisper large v3) which is deployed on Triton Inference Server (server version: 2.62.0, and container image: nvcr.io/nvidia/tritonserver:25.10-pyt-python-py3). The following is my model and perf analyzer configuration files content:
config.pbtxt:
name: "faster-whisper-model"
backend: "python"
max_batch_size: 0 # Dynamic batching is complex with current faster-whisper
input [
{
name: "INPUT_AUDIO_FILE"
data_type: TYPE_FP32 # Or TYPE_FP32/TYPE_INT16 if you handle conversion
dims: [ -1 ]
}
]
output [
{
name: "OUTPUT_TEXT"
data_type: TYPE_STRING
dims: [ 1 ]
}
]
instance_group [
{
count: 1
kind: KIND_GPU
gpus: [ 0 ]
}
]
Perf analyzer command used:
perf_analyzer -m faster-whisper-model
-u my_triton_server_url
-i http
-f test_report.csv
--input-data=./audio_data.json
--input-tensor-format=json
--concurrency-range 1:8:1
--profile-export-file profile_data.json
--verbose-csv
audio_data.json file format:
But I got the following error:
Successfully read data for 1 stream/streams with 1 step/steps.
*** Measurement Settings ***
Batch size: 1
Service Kind: TRITON
Using "time_windows" mode for stabilization
Stabilizing using average latency and throughput
Measurement window: 5000 msec
Latency limit: 0 msec
Concurrency limit: 8 concurrent requests
Using synchronous calls for inference
Request concurrency: 1
Failed to maintain requested inference load. Worker thread(s) failed to generate concurrent requests.
Thread [0] had error: Unable to parse 'data': Shape does not match true shape of 'data' field
Please help me to deal with this problem.
My goal is to send multiple requests (audio files) into triton server and analyze my triton server performance.