Skip to content

Fix swapped ReportStatistics timestamp args in PT2/AOTI backend - #203

Open
hg1112 wants to merge 1 commit into
triton-inference-server:mainfrom
hg1112:fix-pt2-report-statistics-arg-order
Open

Fix swapped ReportStatistics timestamp args in PT2/AOTI backend#203
hg1112 wants to merge 1 commit into
triton-inference-server:mainfrom
hg1112:fix-pt2-report-statistics-arg-order

Conversation

@hg1112

@hg1112 hg1112 commented Aug 2, 2026

Copy link
Copy Markdown

Summary

TRITONBACKEND_ModelInstanceReportStatistics expects timestamps in the order
(exec_start_ns, compute_start_ns, compute_end_ns, exec_end_ns), but
ModelInstanceState::ProcessRequests in the PT2/AOTI instance state passed
exec_end_ns before compute_start_ns/compute_end_ns. Triton core then
interprets exec_end_ns as compute_input_end_ns, and the resulting
subtraction underflows as an unsigned 64-bit value, so
nv_inference_compute_infer_summary_us reports an enormous bogus number
(~18446744074s) instead of the real compute latency.

This reorders the arguments to match the documented header signature and the
legacy (non-PT2) PyTorch backend's call convention, which already does this
correctly. The batch-level TRITONBACKEND_ModelInstanceReportBatchStatistics
call a few lines below was already in the correct order, so no other changes
are needed.

Ref: triton-inference-server/server#8874

Test plan

  • pre-commit run (clang-format, codespell, etc.) passes on the changed file
  • Verified new argument order against TRITONBACKEND_ModelInstanceReportStatistics's
    declaration in triton-inference-server/core's tritonbackend.h

TRITONBACKEND_ModelInstanceReportStatistics expects timestamps in the
order (exec_start_ns, compute_start_ns, compute_end_ns, exec_end_ns),
but ModelInstanceState::ProcessRequests passed exec_end_ns before
compute_start_ns/compute_end_ns. Triton core then interprets
exec_end_ns as compute_input_end_ns, and the resulting subtraction
underflows as an unsigned 64-bit value, reporting
nv_inference_compute_infer_summary_us as an enormous bogus number
(~18446744074s).

Reorders the arguments to match the documented signature and the
legacy PyTorch backend's call convention.

Ref: triton-inference-server/server#8874
@greptile-apps

greptile-apps Bot commented Aug 2, 2026

Copy link
Copy Markdown

Greptile Summary

Corrects the PT2/AOTI per-request statistics call so timestamps follow the expected execution and computation sequence.

  • Moves exec_end_ns after the compute timestamps.
  • Aligns PT2/AOTI reporting with the legacy backend and batch-statistics call convention.

Confidence Score: 5/5

The PR appears safe to merge because the corrected timestamp order is consistent with the established sibling statistics calls.

The change only reorders same-typed timestamp arguments and now aligns the PT2/AOTI per-request call with both the legacy backend and adjacent batch-statistics reporting.

Important Files Changed

Filename Overview
src/pt2/model_instance_state.cc Reorders the statistics timestamps to exec_start_ns, compute_start_ns, compute_end_ns, and exec_end_ns; no actionable issue found.

Reviews (1): Last reviewed commit: "Fix swapped ReportStatistics timestamp a..." | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant