[TRTLLM-14474][chore] Remove legacy python relics and refresh docs after the backend removal#16612
Conversation
| "--output", type=str, help="Output json filename.", default="preprocessed_dataset.json" | ||
| ) | ||
| @click.option( | ||
| "--stdout", |
There was a problem hiding this comment.
Added tokenizer selection and --stdout output options to trtllm-bench prepare-dataset.
Added support for streaming prepared datasets directly to standard output.
Added thinking-budget processing exports for the LLM API.
Should we move the new features to a follow up PR?
There was a problem hiding this comment.
I removed --tokenizer; after dropping the standalone shim, nothing in-repo invokes prepare-dataset --tokenizer (all callers use trtllm-bench --model), so it was dead.
I kept --stdout because it's required parity, not a new feature: sample_performance_alignment.sh and the multigpu AD smoke test are migrated off python3 benchmarks/prepare_dataset.py --stdout, so removing the standalone requires the packaged CLI to support it.
And re: the "thinking-budget exports" summary line — that's a misread; the actual llmapi/init.py change just removes a duplicate DeepSeekV4SparseAttentionConfig entry in all.
WalkthroughThe PR migrates dataset preparation to ChangesBenchmark modernization
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (3)
tensorrt_llm/bench/dataset/utils.py (2)
105-109: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd type annotations to the modified utility.
Use precise annotations such as
Iterable[str],str | Path, and-> None; also document thatoutput_fileis ignored whenstd_out=True. As per coding guidelines, every Python function must be annotated.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tensorrt_llm/bench/dataset/utils.py` around lines 105 - 109, Update write_dataset_to_file with complete type annotations, using Iterable[str] for dataset_generator, str | Path for output_file, and -> None for the return type. Add documentation stating that output_file is ignored when std_out=True, while preserving the existing stdout behavior.Source: Coding guidelines
105-109: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winCover the new stdout branch with pytest.
Extend
tests/unittest/tools/test_prepare_dataset.pyto verify JSONL emission withstd_out=True, no output-file creation, and the existing file-output path. As per path instructions, keep this QA coverage undertests/unittest/and use pytest.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tensorrt_llm/bench/dataset/utils.py` around lines 105 - 109, Extend the pytest coverage in test_prepare_dataset.py for write_dataset_to_file: verify std_out=True emits the generated items as JSONL to stdout without creating an output file, and retain coverage for the existing file-writing path. Keep the tests under tests/unittest/ and use pytest conventions.Source: Path instructions
tensorrt_llm/bench/tuning/utils.py (1)
27-34: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd the required type annotation and contract documentation.
get_device_memoryis a new public helper without a return annotation or docstring. As per coding guidelines, annotate every function and document external interfaces; usedef get_device_memory() -> floatand describe the GiB result and NVML failure behavior.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tensorrt_llm/bench/tuning/utils.py` around lines 27 - 34, Update get_device_memory with the return annotation -> float and add a docstring documenting that it returns total device memory in GiB and describing the behavior when NVML initialization or queries fail.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@AGENTS.md`:
- Around line 58-64: Update the request-flow architecture text in AGENTS.md to
remove TensorRT from the Executor backend list, leaving only PyTorch and
AutoDeploy. Preserve the surrounding request-flow documentation and backend
table unchanged.
In `@docs/source/commands/trtllm-bench.rst`:
- Around line 23-26: Update the click directive’s :commands: list and opening
subcommand count in the trtllm-bench documentation to include prepare-dataset,
keeping the generated CLI command list consistent with the documented
subcommand.
In `@tensorrt_llm/bench/tuning/utils.py`:
- Around line 27-34: Update get_device_memory so the NVML device lookup and
memory read execute inside a try/finally block, ensuring pynvml.nvmlShutdown()
always runs after a successful pynvml.nvmlInit(), including when either NVML
call raises. Preserve the existing total_memory calculation and return behavior.
---
Nitpick comments:
In `@tensorrt_llm/bench/dataset/utils.py`:
- Around line 105-109: Update write_dataset_to_file with complete type
annotations, using Iterable[str] for dataset_generator, str | Path for
output_file, and -> None for the return type. Add documentation stating that
output_file is ignored when std_out=True, while preserving the existing stdout
behavior.
- Around line 105-109: Extend the pytest coverage in test_prepare_dataset.py for
write_dataset_to_file: verify std_out=True emits the generated items as JSONL to
stdout without creating an output file, and retain coverage for the existing
file-writing path. Keep the tests under tests/unittest/ and use pytest
conventions.
In `@tensorrt_llm/bench/tuning/utils.py`:
- Around line 27-34: Update get_device_memory with the return annotation ->
float and add a docstring documenting that it returns total device memory in GiB
and describing the behavior when NVML initialization or queries fail.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: d8f569d0-f54f-4d53-babd-54c165a61e03
📒 Files selected for processing (48)
.pre-commit-config.yamlAGENTS.mdbenchmarks/README.mdbenchmarks/prepare_dataset.pybenchmarks/utils/__init__.pybenchmarks/utils/convert_nemo_dataset.pybenchmarks/utils/generate_rand_loras.pybenchmarks/utils/prepare_real_data.pybenchmarks/utils/prepare_synthetic_data.pybenchmarks/utils/utils.pydocs/source/_ext/llmapi_config_telemetry.pydocs/source/commands/trtllm-bench.rstdocs/source/developer-guide/perf-overview.mddocs/source/developer-guide/telemetry.mddocs/source/examples/customization.mdexamples/auto_deploy/paragraf/create_standalone_package.pyexamples/layer_wise_benchmarks/sample_performance_alignment.shexamples/quantization/README.mdexamples/quantization/quantize.pylegacy-files.txtpyproject.tomlruff-legacy-baseline.jsonruff-legacy.tomlsetup.pytensorrt_llm/_deprecation.pytensorrt_llm/bench/benchmark/__init__.pytensorrt_llm/bench/benchmark/utils/general.pytensorrt_llm/bench/build/__init__.pytensorrt_llm/bench/dataset/prepare_dataset.pytensorrt_llm/bench/dataset/prepare_real_data.pytensorrt_llm/bench/dataset/prepare_synthetic_data.pytensorrt_llm/bench/dataset/utils.pytensorrt_llm/bench/tuning/__init__.pytensorrt_llm/bench/tuning/dataclasses.pytensorrt_llm/bench/tuning/heuristics.pytensorrt_llm/bench/tuning/settings.pytensorrt_llm/bench/tuning/utils.pytensorrt_llm/llmapi/__init__.pytensorrt_llm/models/convert_utils.pytensorrt_llm/models/unet/pp/__init__.pytensorrt_llm/quantization/__init__.pytensorrt_llm/quantization/image_processing.pytensorrt_llm/quantization/quantize_by_modelopt.pytensorrt_llm/serialization.pytests/unittest/auto_deploy/multigpu/smoke/test_ad_allreduce_strategies.pytests/unittest/auto_deploy/singlegpu/smoke/test_ad_trtllm_bench.pytests/unittest/others/test_quantize_calib_dataset.pytests/unittest/tools/test_prepare_dataset.py
💤 Files with no reviewable changes (17)
- benchmarks/prepare_dataset.py
- benchmarks/README.md
- benchmarks/utils/convert_nemo_dataset.py
- setup.py
- benchmarks/utils/generate_rand_loras.py
- tests/unittest/others/test_quantize_calib_dataset.py
- tensorrt_llm/quantization/image_processing.py
- benchmarks/utils/prepare_synthetic_data.py
- examples/auto_deploy/paragraf/create_standalone_package.py
- examples/quantization/quantize.py
- benchmarks/utils/utils.py
- benchmarks/utils/prepare_real_data.py
- tensorrt_llm/llmapi/init.py
- tensorrt_llm/models/convert_utils.py
- tensorrt_llm/serialization.py
- tensorrt_llm/quantization/quantize_by_modelopt.py
- tensorrt_llm/_deprecation.py
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 3
🧹 Nitpick comments (3)
tensorrt_llm/bench/dataset/utils.py (2)
105-109: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd type annotations to the modified utility.
Use precise annotations such as
Iterable[str],str | Path, and-> None; also document thatoutput_fileis ignored whenstd_out=True. As per coding guidelines, every Python function must be annotated.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tensorrt_llm/bench/dataset/utils.py` around lines 105 - 109, Update write_dataset_to_file with complete type annotations, using Iterable[str] for dataset_generator, str | Path for output_file, and -> None for the return type. Add documentation stating that output_file is ignored when std_out=True, while preserving the existing stdout behavior.Source: Coding guidelines
105-109: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winCover the new stdout branch with pytest.
Extend
tests/unittest/tools/test_prepare_dataset.pyto verify JSONL emission withstd_out=True, no output-file creation, and the existing file-output path. As per path instructions, keep this QA coverage undertests/unittest/and use pytest.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tensorrt_llm/bench/dataset/utils.py` around lines 105 - 109, Extend the pytest coverage in test_prepare_dataset.py for write_dataset_to_file: verify std_out=True emits the generated items as JSONL to stdout without creating an output file, and retain coverage for the existing file-writing path. Keep the tests under tests/unittest/ and use pytest conventions.Source: Path instructions
tensorrt_llm/bench/tuning/utils.py (1)
27-34: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd the required type annotation and contract documentation.
get_device_memoryis a new public helper without a return annotation or docstring. As per coding guidelines, annotate every function and document external interfaces; usedef get_device_memory() -> floatand describe the GiB result and NVML failure behavior.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tensorrt_llm/bench/tuning/utils.py` around lines 27 - 34, Update get_device_memory with the return annotation -> float and add a docstring documenting that it returns total device memory in GiB and describing the behavior when NVML initialization or queries fail.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@AGENTS.md`:
- Around line 58-64: Update the request-flow architecture text in AGENTS.md to
remove TensorRT from the Executor backend list, leaving only PyTorch and
AutoDeploy. Preserve the surrounding request-flow documentation and backend
table unchanged.
In `@docs/source/commands/trtllm-bench.rst`:
- Around line 23-26: Update the click directive’s :commands: list and opening
subcommand count in the trtllm-bench documentation to include prepare-dataset,
keeping the generated CLI command list consistent with the documented
subcommand.
In `@tensorrt_llm/bench/tuning/utils.py`:
- Around line 27-34: Update get_device_memory so the NVML device lookup and
memory read execute inside a try/finally block, ensuring pynvml.nvmlShutdown()
always runs after a successful pynvml.nvmlInit(), including when either NVML
call raises. Preserve the existing total_memory calculation and return behavior.
---
Nitpick comments:
In `@tensorrt_llm/bench/dataset/utils.py`:
- Around line 105-109: Update write_dataset_to_file with complete type
annotations, using Iterable[str] for dataset_generator, str | Path for
output_file, and -> None for the return type. Add documentation stating that
output_file is ignored when std_out=True, while preserving the existing stdout
behavior.
- Around line 105-109: Extend the pytest coverage in test_prepare_dataset.py for
write_dataset_to_file: verify std_out=True emits the generated items as JSONL to
stdout without creating an output file, and retain coverage for the existing
file-writing path. Keep the tests under tests/unittest/ and use pytest
conventions.
In `@tensorrt_llm/bench/tuning/utils.py`:
- Around line 27-34: Update get_device_memory with the return annotation ->
float and add a docstring documenting that it returns total device memory in GiB
and describing the behavior when NVML initialization or queries fail.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: d8f569d0-f54f-4d53-babd-54c165a61e03
📒 Files selected for processing (48)
.pre-commit-config.yamlAGENTS.mdbenchmarks/README.mdbenchmarks/prepare_dataset.pybenchmarks/utils/__init__.pybenchmarks/utils/convert_nemo_dataset.pybenchmarks/utils/generate_rand_loras.pybenchmarks/utils/prepare_real_data.pybenchmarks/utils/prepare_synthetic_data.pybenchmarks/utils/utils.pydocs/source/_ext/llmapi_config_telemetry.pydocs/source/commands/trtllm-bench.rstdocs/source/developer-guide/perf-overview.mddocs/source/developer-guide/telemetry.mddocs/source/examples/customization.mdexamples/auto_deploy/paragraf/create_standalone_package.pyexamples/layer_wise_benchmarks/sample_performance_alignment.shexamples/quantization/README.mdexamples/quantization/quantize.pylegacy-files.txtpyproject.tomlruff-legacy-baseline.jsonruff-legacy.tomlsetup.pytensorrt_llm/_deprecation.pytensorrt_llm/bench/benchmark/__init__.pytensorrt_llm/bench/benchmark/utils/general.pytensorrt_llm/bench/build/__init__.pytensorrt_llm/bench/dataset/prepare_dataset.pytensorrt_llm/bench/dataset/prepare_real_data.pytensorrt_llm/bench/dataset/prepare_synthetic_data.pytensorrt_llm/bench/dataset/utils.pytensorrt_llm/bench/tuning/__init__.pytensorrt_llm/bench/tuning/dataclasses.pytensorrt_llm/bench/tuning/heuristics.pytensorrt_llm/bench/tuning/settings.pytensorrt_llm/bench/tuning/utils.pytensorrt_llm/llmapi/__init__.pytensorrt_llm/models/convert_utils.pytensorrt_llm/models/unet/pp/__init__.pytensorrt_llm/quantization/__init__.pytensorrt_llm/quantization/image_processing.pytensorrt_llm/quantization/quantize_by_modelopt.pytensorrt_llm/serialization.pytests/unittest/auto_deploy/multigpu/smoke/test_ad_allreduce_strategies.pytests/unittest/auto_deploy/singlegpu/smoke/test_ad_trtllm_bench.pytests/unittest/others/test_quantize_calib_dataset.pytests/unittest/tools/test_prepare_dataset.py
💤 Files with no reviewable changes (17)
- benchmarks/prepare_dataset.py
- benchmarks/README.md
- benchmarks/utils/convert_nemo_dataset.py
- setup.py
- benchmarks/utils/generate_rand_loras.py
- tests/unittest/others/test_quantize_calib_dataset.py
- tensorrt_llm/quantization/image_processing.py
- benchmarks/utils/prepare_synthetic_data.py
- examples/auto_deploy/paragraf/create_standalone_package.py
- examples/quantization/quantize.py
- benchmarks/utils/utils.py
- benchmarks/utils/prepare_real_data.py
- tensorrt_llm/llmapi/init.py
- tensorrt_llm/models/convert_utils.py
- tensorrt_llm/serialization.py
- tensorrt_llm/quantization/quantize_by_modelopt.py
- tensorrt_llm/_deprecation.py
🛑 Comments failed to post (3)
AGENTS.md (1)
58-64: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Remove the stale TensorRT backend from the request-flow documentation.
The new backend table says only PyTorch and AutoDeploy remain, but Line 70 still says
Executor (PyTorch/AutoDeploy/TensorRT). Change it toExecutor (PyTorch/AutoDeploy)so the architecture guidance is consistent.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@AGENTS.md` around lines 58 - 64, Update the request-flow architecture text in AGENTS.md to remove TensorRT from the Executor backend list, leaving only PyTorch and AutoDeploy. Preserve the surrounding request-flow documentation and backend table unchanged.docs/source/commands/trtllm-bench.rst (1)
23-26: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Include
prepare-datasetin the generated command list.The click directive at Line 16 still lists only
throughput, latency, build, so the new command is absent from generated CLI syntax. Update:commands:and the opening subcommand count to includeprepare-dataset.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/source/commands/trtllm-bench.rst` around lines 23 - 26, Update the click directive’s :commands: list and opening subcommand count in the trtllm-bench documentation to include prepare-dataset, keeping the generated CLI command list consistent with the documented subcommand.tensorrt_llm/bench/tuning/utils.py (1)
27-34: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash set -euo pipefail git ls-files tensorrt_llm/bench/tuning/utils.py wc -l tensorrt_llm/bench/tuning/utils.py cat -n tensorrt_llm/bench/tuning/utils.py | sed -n '1,120p' rg -n "get_device_memory|nvmlInit|nvmlShutdown|pynvml" tensorrt_llm/bench -SRepository: NVIDIA/TensorRT-LLM
Length of output: 3132
🏁 Script executed:
#!/bin/bash set -euo pipefail sed -n '280,330p' tensorrt_llm/bench/dataclasses/reporting.py sed -n '1,140p' tensorrt_llm/bench/tuning/heuristics.py python3 - <<'PY' from pathlib import Path text = Path("tensorrt_llm/bench/tuning/utils.py").read_text() print("try/finally present:", "try:" in text and "finally:" in text) print("nvmlShutdown count:", text.count("nvmlShutdown")) PYRepository: NVIDIA/TensorRT-LLM
Length of output: 8583
Guard NVML teardown in
get_device_memory()
IfnvmlDeviceGetHandleByIndex()ornvmlDeviceGetMemoryInfo()raises afternvmlInit(),nvmlShutdown()is skipped and NVML stays initialized. Wrap the lookup/read intry/finally.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tensorrt_llm/bench/tuning/utils.py` around lines 27 - 34, Update get_device_memory so the NVML device lookup and memory read execute inside a try/finally block, ensuring pynvml.nvmlShutdown() always runs after a successful pynvml.nvmlInit(), including when either NVML call raises. Preserve the existing total_memory calculation and return behavior.
…ter the backend removal Trailing python-side cleanup after the legacy TensorRT backend removal (TRTLLM-14026, PR NVIDIA#16369): - Remove legacy quantization-export and plugin-gen relics, the engine-workflow serialization/deprecation shims, and other dead python modules (with their setup.py package_data entries). - Rename tensorrt_llm/bench/build to bench/tuning (build.py -> settings.py, tuning.py -> heuristics.py); the module no longer builds anything. The renamed files are also cleaned up to pass the standard ruff rules (docstrings, imports, quotes, line length) and removed from the legacy-lint allowlist. - Remove the top-level benchmarks/ folder entirely. benchmarks/prepare_dataset.py (already marked deprecated upstream) is deduplicated into the packaged tensorrt_llm/bench/dataset implementation behind trtllm-bench prepare-dataset, which gains --tokenizer and --stdout root options as a full replacement; all in-repo callers and docs are repointed. The remaining utils were orphaned legacy-engine helpers (generate_rand_loras.py wrote the removed .npy LoRA engine format; convert_nemo_dataset.py had no consumers). - Refresh docs and repo guidance to the current PyTorch/AutoDeploy state (customization.md rewrite, AGENTS.md backend section, examples/quantization/README.md, telemetry doc regeneration). - Prune stale legacy-lint entries (removed files) from legacy-files.txt, the generated ruff configs, and ruff-legacy-baseline.json. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
e823607 to
9843df9
Compare
|
/bot run --disable-fail-fast |
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/source/developer-guide/perf-overview.md`:
- Line 290: Update the relevant documentation wording from “In order to prepare
a dataset” to “To prepare a dataset,” preserving the surrounding meaning and
content.
In `@docs/source/examples/customization.md`:
- Line 5: Add the repository-standard NVIDIA copyright header with the current
copyright year at the very top of customization.md, before the existing document
title and content. Preserve all existing documentation unchanged.
- Line 17: Update the sentence describing SamplingParams to hyphenate the
compound adjective as “LLM-generated responses,” without changing the
surrounding wording or meaning.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 63b1b307-9955-4f6b-a25f-2d4a3a75b861
📒 Files selected for processing (25)
.pre-commit-config.yamlAGENTS.mdbenchmarks/README.mdbenchmarks/prepare_dataset.pybenchmarks/utils/__init__.pybenchmarks/utils/convert_nemo_dataset.pybenchmarks/utils/generate_rand_loras.pybenchmarks/utils/prepare_real_data.pybenchmarks/utils/prepare_synthetic_data.pybenchmarks/utils/utils.pydocs/source/_ext/llmapi_config_telemetry.pydocs/source/commands/trtllm-bench.rstdocs/source/developer-guide/perf-overview.mddocs/source/developer-guide/telemetry.mddocs/source/examples/customization.mdexamples/auto_deploy/paragraf/create_standalone_package.pyexamples/layer_wise_benchmarks/sample_performance_alignment.shexamples/quantization/README.mdexamples/quantization/quantize.pylegacy-files.txtpyproject.tomlruff-legacy-baseline.jsonruff-legacy.tomlsetup.pytensorrt_llm/_deprecation.py
💤 Files with no reviewable changes (15)
- setup.py
- benchmarks/utils/generate_rand_loras.py
- benchmarks/utils/convert_nemo_dataset.py
- tensorrt_llm/_deprecation.py
- examples/quantization/quantize.py
- examples/auto_deploy/paragraf/create_standalone_package.py
- benchmarks/prepare_dataset.py
- benchmarks/utils/prepare_synthetic_data.py
- benchmarks/README.md
- benchmarks/utils/prepare_real_data.py
- benchmarks/utils/utils.py
- ruff-legacy.toml
- pyproject.toml
- .pre-commit-config.yaml
- legacy-files.txt
🚧 Files skipped from review as they are similar to previous changes (4)
- docs/source/_ext/llmapi_config_telemetry.py
- examples/layer_wise_benchmarks/sample_performance_alignment.sh
- docs/source/commands/trtllm-bench.rst
- AGENTS.md
|
PR_Github #61189 [ run ] triggered by Bot. Commit: |
|
PR_Github #61189 [ run ] completed with state
|
|
/bot run --disable-fail-fast |
|
PR_Github #61285 [ run ] triggered by Bot. Commit: |
8 participants
Why this PR exists
Phase C removed the TensorRT engine backend. This PR removes the python
modules, packaging entries, and lint bookkeeping that existed only to
serve it, deduplicates the benchmark dataset tooling, and refreshes docs
that still described the removed workflow. It is authored on a file set
disjoint from the other Phase D PRs, so merge order does not matter.
Change groups
1. Legacy quantization-export and plugin-gen relics
tensorrt_llm/quantization/quantize_by_modelopt.py(1,332 lines)trtllm-build; the PyTorch backend quantizes via ModelOpt directlytensorrt_llm/quantization/image_processing.pyexamples/quantization/quantize.py+tests/unittest/others/test_quantize_calib_dataset.pytensorrt_llm/_deprecation.pytensorrt_llm/models/unet/pp/__init__.pysetup.pyentriespackage_dataglobs fortools/plugin_gen/templates/*andbench/build/benchmark_config.yml— both paths goneserialization.pywhitelist entry"tensorrt_llm.builder": ["BuildConfig"]— the whitelisted module is gonePlus the import /
__init__.pyfan-out intensorrt_llm/quantization/,llmapi/,models/convert_utils.py, and_utils.py.2.
tensorrt_llm/bench/build→bench/tuningrename (+ lint graduation)The module stopped building engines long ago; it only computes tuning
heuristics for
trtllm-bench. Renamed to say what it is:build.py → settings.py,tuning.py → heuristics.py,dataclasses.py/utils.pycarried over; importers inbench/benchmark/repointed. Nopublic API surface — nothing outside
tensorrt_llm.benchimports these.Per reviewer tburt-nv, the renamed files were graduated out of the
legacy-lint allowlist: cleaned up to pass the standard ruff rules
(docstrings, import order, quotes, line length) and removed from
legacy-files.txt/ruff-legacy.toml/ruff-legacy-baseline.json.Formatting-only, no logic change — verified by
ruff/ruff-formatandcheck-configs.3. Benchmark dataset tooling deduplicated;
benchmarks/folder removedbenchmarks/prepare_dataset.py(already@click.group(deprecated=True)upstream) and itsutils/are removed.The packaged
tensorrt_llm/bench/dataset/behindtrtllm-bench prepare-datasetis now the single entry point.--stdout(one JSON entry per line) — requiredparity for the in-repo callers migrated off
python3 benchmarks/prepare_dataset.py --stdout(
examples/layer_wise_benchmarks/sample_performance_alignment.shand themultigpu AD smoke test). It is encoded as a null output path in
RootArgs.output(Optional[str]), so only the group and a four-linebranch in
write_dataset_to_filechange —prepare_synthetic_data.pyand
prepare_real_data.pyare unchanged vs main.python3 benchmarks/prepare_dataset.pyscript is not kept;trtllm-bench prepare-datasetis the single supported invocation.sample_performance_alignment.sh,tests/unittest/tools/test_prepare_dataset.py).benchmarks/utils/files were orphaned legacy-enginehelpers (
generate_rand_loras.pywrote the removed.npyLoRA format;convert_nemo_dataset.pyhad no consumers).trtllm-bench --model <m> prepare-dataset --output <file> ...(file mode deliberately — see the stdout-banner note in Follow-ups).
4. Docs / repo guidance refresh
docs/source/examples/customization.mdrewritten (it imported theremoved
BuildConfigand documented engine building end-to-end).AGENTS.md: backend section trimmed to PyTorch / AutoDeploy; allreferences to the removed TensorRT backend dropped.
examples/quantization/README.md: rewritten around the currentpre-quantized-checkpoint / ModelOpt flow.
docs/source/_ext/llmapi_config_telemetry.pyno longer emits aTrtLlmArgssection; the now-single-element loop was inlined to handleTorchLlmArgsdirectly (per reviewer BowenFu; byte-identicalgenerator output verified).
telemetry.mdregenerated from the currentcommitted manifest (see Reviewer resolutions).
5. Legacy-lint bookkeeping (generated-config churn)
All Phase D lint pruning rides in this PR to keep it out of the other
three: stale
legacy-files.txtentries for removed files, the regeneratedruff-legacy.toml/pyproject.toml/.pre-commit-config.yamlmanagedblocks, and the stale
ruff-legacy-baseline.jsonkeys (_metatotalsupdated in the same change). Final baseline: 224 files / 2096 violations,
self-consistent and a strict subset of
legacy-files.txt(0 orphans).legacy_utils.py check-configspasses.Reviewer resolutions
for args_class in (...)loop was inlined;
telemetry.mdregenerated from the current manifest(
llm_args_golden_manifest.jsonalready holds onlyTorchLlmArgs, 269fields — untouched here). The checked-in doc was a stale build-time
snapshot; regenerating also drops main's vestigial 280-field
TrtLlmArgstable. No manifest change ⇒ no new telemetry/privacy CODEOWNER approval.
--tokenizerscope: a--tokenizeroption was initiallyadded for shim parity, then removed — no in-repo caller uses
prepare-dataset --tokenizer(they all passtrtllm-bench --model, sothe model resolves from
ctx.obj.modelexactly as on main). Deferred toa dedicated feature PR if standalone-tokenizer use is ever wanted.
legacy-lint allowlist (see change group 2).
Reviewer callouts
ParallelConfig/ executor surface is untouched — C++-side removalsare in the cpp-cleanup PR (TRTLLM-14475).
tests/unittest/api_stabilitysurface is touched;the deleted modules were not part of the committed LLM API.
llmapi/__init__.pyremoves a duplicate__all__entry(
DeepSeekV4SparseAttentionConfigwas listed twice on main); the symbolstays imported and exported. (A review tool mislabeled this as adding
"thinking-budget exports" — those lines are unchanged context.)
Validation
python3 -c "import tensorrt_llm"and the attention / bench importfan-out verified after each rename / deletion.
ruff,ruff-format, theruff-legacybaseline-gated lint, and the "verifylegacy lint configs are up to date" hook;
check-configspasses.a19410ebaa: parent isa19410ebaa, zerocpp/files in the diff.
Follow-ups (not in this PR)
import tensorrt_llmprints a version banner via abare
print()to stdout, soprepare-dataset --stdout > filepipingwould prepend the banner. Docs therefore use
--output <file>(filemode). Moving the banner to stderr is a possible separate
[None][infra]fix.
git history): local-directory dataset loading and the multimodal image
dump.
--tokenizerstandalone support, if ever wanted (see reviewerresolutions).
Dev Engineer Review
trtllm-bench prepare-dataset, adding--tokenizerand--stdoutsupport.ParallelConfigand executor surfaces are intentionally unchanged.QA Engineer Review
tests/unittest/auto_deploy/multigpu/smoke/test_ad_allreduce_strategies.py: updated dataset preparation invocation and working directory.tests/unittest/auto_deploy/singlegpu/smoke/test_ad_trtllm_bench.py: updated dataset preparation working directory.tests/unittest/tools/test_prepare_dataset.py: removed the obsolete script-path constant and updated CLI references.test_is_cnn_dailymail_local_repotest_get_calib_dataloader_local_cnn_dailymailtest-db/) or manual QA (qa/).Description
Test Coverage
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
To see a list of available CI bot commands, please comment
/bot help.