Description
For PyTorch backend PT2/AOTI models, nv_inference_compute_infer_summary_us
can report extremely large values, e.g. around 18446744074s.
This appears to be caused by an argument ordering bug in the PT2 backend's
TRITONBACKEND_ModelInstanceReportStatistics call.
The API expects:
exec_start_ns, compute_start_ns, compute_end_ns, exec_end_ns
but pytorch_backend/src/pt2/model_instance_state.cc currently passes:
exec_start_ns, exec_end_ns, compute_start_ns, compute_end_ns
As a result, Triton core interprets exec_end_ns as compute_input_end_ns
and compute_start_ns as compute_output_start_ns. Then this subtraction
underflows as uint64_t (huge value 18446744074s).
Triton Information
What version of Triton are you using? 26.06
Are you using the Triton container or did you build it yourself? Container.
To Reproduce
Run PT2/AOTI pytorch model and observe nv_inference_compute_infer_summary_us.
Expected behavior
PT2/AOTI per-request statistics should pass timestamps in the same order as
legacy PyTorch backend:
TRITONBACKEND_ModelInstanceReportStatistics(
TritonModelInstance(), request, responses[r] != nullptr,
exec_start_ns, compute_start_ns, compute_end_ns, exec_end_ns)
For a PR, the actual fix is tiny in pytorch_backend/src/pt2/model_instance_state.cc: swap the last three timestamp args to match the API.
Description
For PyTorch backend PT2/AOTI models,
nv_inference_compute_infer_summary_uscan report extremely large values, e.g. around
18446744074s.This appears to be caused by an argument ordering bug in the PT2 backend's
TRITONBACKEND_ModelInstanceReportStatisticscall.The API expects:
exec_start_ns, compute_start_ns, compute_end_ns, exec_end_ns
but pytorch_backend/src/pt2/model_instance_state.cc currently passes:
exec_start_ns, exec_end_ns, compute_start_ns, compute_end_ns
As a result, Triton core interprets exec_end_ns as compute_input_end_ns
and compute_start_ns as compute_output_start_ns. Then this subtraction
underflows as uint64_t (huge value 18446744074s).
Triton Information
What version of Triton are you using? 26.06
Are you using the Triton container or did you build it yourself? Container.
To Reproduce
Run PT2/AOTI pytorch model and observe nv_inference_compute_infer_summary_us.
Expected behavior
PT2/AOTI per-request statistics should pass timestamps in the same order as
legacy PyTorch backend:
For a PR, the actual fix is tiny in
pytorch_backend/src/pt2/model_instance_state.cc: swap the last three timestamp args to match the API.