Skip to content

[https://nvbugs/6414760][fix] Rewrite the test with the PyTorch LLM API (LLM + SamplingParams(n=4…#16014

Closed
trtllm-agent wants to merge 1 commit into
NVIDIA:mainfrom
tensorrt-cicd:repair-bot-bug6414760
Closed

[https://nvbugs/6414760][fix] Rewrite the test with the PyTorch LLM API (LLM + SamplingParams(n=4…#16014
trtllm-agent wants to merge 1 commit into
NVIDIA:mainfrom
tensorrt-cicd:repair-bot-bug6414760

Conversation

@trtllm-agent

@trtllm-agent trtllm-agent commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Root cause: PR [TRTLLM-13781][chore] Remove legacy TensorRT examples #15763 (e5a05b2) removed examples/models/core/qwen/{requirements.txt,convert_checkpoint.py} as part of legacy TRT cleanup, but test_qwen_e2e_cpprunner_large_new_tokens still consumed the qwen_example_root fixture + convert_weights + trtllm-build + ModelRunnerCpp
  • Fix: Rewrite the test with the PyTorch LLM API (LLM + SamplingParams(n=4, best_of=4)) to preserve the RCCA invariant (no zero-length among 4 returned sequences) without touching deleted legacy files; keep the same test id and parametrize
  • Automated fix generated by repair-bot

Test plan

  • Verify fix on the same GPU type as the original failure
  • Check for regressions in related tests

Links

Summary by CodeRabbit

  • Tests
    • Simplified an end-to-end generation test to use a more direct inference flow.
    • Improved coverage for multi-sample text generation by checking that each returned completion contains tokens.
    • Reduced test setup requirements, making the test easier to run and maintain.

…LLM API

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The test test_qwen_e2e_cpprunner_large_new_tokens in test_e2e.py was rewritten to use tensorrt_llm.LLM and SamplingParams for generation instead of convert_weights, trtllm-build, and ModelRunnerCpp. The unused convert_weights import was removed and the test signature simplified.

Changes

Qwen e2e test rewrite

Layer / File(s) Summary
Simplified generation flow
tests/integration/defs/test_e2e.py
Removed convert_weights import; replaced the conversion/build/tokenizer/ModelRunnerCpp flow in test_qwen_e2e_cpprunner_large_new_tokens with direct tensorrt_llm.LLM/SamplingParams generation, asserting all completions have non-empty token_ids; test signature reduced to (model_name, model_path, llm_venv).

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: yibinl-nvidia, nv-guomingz

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title is specific and matches the main change: rewriting the Qwen test to use the PyTorch LLM API for bug 6414760.
Description check ✅ Passed The description clearly states the root cause, fix, test plan, and bug link, with only minor template-section differences.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/integration/defs/test_e2e.py (1)

75-92: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

llm_venv fixture is now unused.

The rewritten test runs generation in-process via LLM, so the llm_venv fixture is no longer exercised. Requesting it still triggers workspace creation and teardown (see llm_venv in conftest.py) for no purpose. Consider dropping it from the signature if parametrization/markers don't depend on it. Note: model_name should stay if it's supplied by a @pytest.mark.parametrize decorator (lines 31-74, not shown).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/defs/test_e2e.py` around lines 75 - 92, The test
test_qwen_e2e_cpprunner_large_new_tokens no longer uses the llm_venv fixture
because generation now runs directly through LLM in-process. Remove llm_venv
from the test function signature so pytest does not create and tear down the
unused workspace, and keep model_name only if it is still needed by the existing
parametrization in this test module.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/integration/defs/test_e2e.py`:
- Around line 75-92: The test test_qwen_e2e_cpprunner_large_new_tokens no longer
uses the llm_venv fixture because generation now runs directly through LLM
in-process. Remove llm_venv from the test function signature so pytest does not
create and tear down the unused workspace, and keep model_name only if it is
still needed by the existing parametrization in this test module.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a2c9155d-d12d-4cb0-b09b-7190fe8eea68

📥 Commits

Reviewing files that changed from the base of the PR and between a0c406f and 5aaaa5a.

📒 Files selected for processing (1)
  • tests/integration/defs/test_e2e.py

@Wanli-Jiang Wanli-Jiang closed this Jul 7, 2026
@Wanli-Jiang Wanli-Jiang reopened this Jul 7, 2026
@Wanli-Jiang

Copy link
Copy Markdown
Collaborator

I will include the fix in #15907 to save CI resources.

@Wanli-Jiang Wanli-Jiang closed this Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants