test: Add CUDA graph tests for non-batching TensorRT models

[pskiran1]

What does the PR do?

Add CUDA graph coverage for non-batching TensorRT models (max_batch_size: 0). The tests verify that:

• A captured CUDA graph is launched for a fixed-shape model.
• A captured CUDA graph is launched for a dynamic-shape model with graph_spec.batch_size: 0.
• Inference does not fall back to regular TensorRT execution when the request shape matches the captured graph.

Checklist

• PR title reflects the change and is of format <commit_type>: <Title>
• Changes are described in the pull request.
• Related issues are referenced.
• Populated github labels field
• Added test plan and verified test passes.
• Verified that the PR passes existing CI.
• Verified copyright is correct on all changed files.
• Added succinct git squash message before merging ref.
• All template sections are filled out.
• Optional: Additional screenshots for behavior/output changes with before/after.

Commit Type:

Check the conventional commit type
box here and add the label to the github PR.

• build
• ci
• docs
• feat
• fix
• perf
• refactor
• revert
• style
• test

Related PRs:

Where should the reviewer start?

Test plan:

• CI Pipeline ID: 59749869

Caveats:

Background

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• closes GitHub issue: CUDA Graph not work  #7150

[greptile-apps]

Greptile Summary

This PR adds two new CUDA graph tests for non-batching TensorRT models: test_nobatch_fixed_shape (already present, now with a corrected expected graph-launch count of 1 instead of 0) and the new test_nobatch_dynamic_shape, which verifies that a variable-shape non-batching model uses CUDA graph execution when the request shape matches the graph_spec and falls back to regular execution for a non-matching shape.

• trt_cuda_graph_test.py: Adds test_nobatch_dynamic_shape with two inferences — shape (16,) expected to hit the captured graph, and shape (20,) expected to bypass it; sys.path.append(\"../common\") is correctly placed before the infer_util/test_util imports.
• test.sh: Adds a new test section that copies plan_nobatch_float32_float32_float32 from the variable model repository, appends an explicit graph_spec with batch_size: 0, and asserts log counts for graph capture (1), graph launch (1 on profile 6), and regular execution (1 on profile 6).

Confidence Score: 5/5

Safe to merge — the change is additive test infrastructure with no production code impact.

Both files add new test coverage for non-batching TRT CUDA graph paths. The logic in the Python test (two inferences exercising graph-hit and graph-miss paths) aligns correctly with the shell log assertions. The only omission is a profile-structure comment explaining why profile 6 is expected, which would aid future maintainers but does not affect correctness.

The test_nobatch_dynamic_shape block in test.sh (lines 344–403) hardcodes profile 6 [6] without documenting the plan_nobatch_float32_float32_float32 variable model's profile layout, unlike the analogous batching-model test sections.

Important Files Changed

Filename	Overview
qa/L0_cuda_graph/test.sh	Adds a test_nobatch_dynamic_shape shell test section that sets up a variable-shape non-batching TRT model, runs inference, and verifies log counts for CUDA graph launch vs. regular execution; also fixes test_nobatch_fixed_shape expected CUDA graph count from 0 to 1.
qa/L0_cuda_graph/trt_cuda_graph_test.py	Adds test_nobatch_dynamic_shape which exercises a graph-spec-matching inference (shape 16, uses CUDA graph) and a non-matching inference (shape 20, falls back to regular execution); import ordering correctly places sys.path.append before the ../common module imports.

Sequence Diagram

sequenceDiagram
    participant SH as test.sh
    participant SRV as Triton Server
    participant PY as trt_cuda_graph_test.py

    SH->>SRV: Start with plan_nobatch variable model + graph_spec (shape 16)
    SRV-->>SH: Server ready (captured 1 CUDA graph for shape 16)

    SH->>PY: Run test_nobatch_dynamic_shape
    PY->>SRV: "infer_exact(shape=(16,), batch_size=0)"
    SRV-->>PY: Response (CUDA graph launched — profile 6)
    PY->>SRV: "infer_exact(shape=(20,), batch_size=0)"
    SRV-->>PY: Response (regular execution — profile 6)
    PY-->>SH: Test pass

    SH->>SRV: grep log: 1x launching CUDA graph (profile 6)
    SH->>SRV: grep log: 1x being executed for (profile 6)
    SH->>SRV: grep log: 1x captured CUDA graph for
    SH->>SRV: Kill server

Loading

_{Reviews (2): Last reviewed commit: "Update" | Re-trigger Greptile}

[Vinya567]

LGTM