[TRTLLM-14022][feat] BREAKING: Remove python modules and tests for legacy TensorRT backend

[Wanli-Jiang]

Overview

This PR removes the legacy TensorRT execution backend from the tensorrt_llm Python
package so that import tensorrt_llm and the PyTorch backend run without the
tensorrt package installed. It is step 1.5 of the staged TensorRT-backend removal
(after the examples #15763, docs #15767, tests #15810, and Triton C++ backend #15907
removals). Dropping the tensorrt pip dependency itself is deferred to the C++/packaging
step; requirements.txt is unchanged here.

• 304 files: 205 deletions, 97 modifications, 2 additions (+1,211 / −92,411).
• Approach — gut-in-place, keep import paths stable: a module that only partly
  held TensorRT code is gutted in place — the TensorRT parts are deleted and the
  TensorRT-free symbols the PyTorch backend / VisualGen still need stay exactly where
  they were, so consumer import paths under _torch/** are unchanged and the diff stays
  removal-shaped.
• Test ownership rule: this PR deletes the symbols, so it owns every test that those
  deletions break — TRT-only tests are deleted, mixed suites are de-TRT'd (PyTorch parts
  kept, shared harness signatures unchanged), and all affected test-db/qa/waives
  entries are delisted (§ H–I).
• Validation: import tensorrt_llm green with tensorrt blocked; AST import-graph
  scan over the surviving package = 0 imports of any deleted module or symbol; all
  changed .py byte-compile; full test-list cross-reference (4k+ entries) = 0 dangling
  files/functions; pre-commit fully green (incl. ruff-legacy with the regenerated
  baseline and the test-list validators).
• Review history folded in: nv-guomingz (wording, _on_trt_backend removal,
  _ModelFormatKind removal), QiJune (plugin/ relocation; quantize_by_modelopt.py /
  functional.py full removals deferred to follow-ups), brb-nv (LoRA sign-off + comment
  style), tburt-nv (coverage: test_gather_generation_logits_cuda_graph ported, rest
  tracked as follow-ups) — see § K.
• One row per distinct reason — a file changed for two different reasons gets two rows.

---

A. Gutted-in-place core (TensorRT stripped, TRT-free symbols kept)

File	What changed	Why
functional.py	Massive strip (~−7,900): deleted import tensorrt as trt, graph_rewriting/network imports, _common.default_net/default_trtnet/precision, and all graph-op machinery (DimRange, the graph Tensor class and every functional op). Kept backend-agnostic enums (RotaryScalingType, PositionEmbeddingType, AttentionMaskType, AllReduceStrategy, AllReduceFusionOp), param classes (AllReduceParams, MoEAllReduceParams), and RopeEmbeddingUtils. The only added line is from torch import Tensor, repointing the retained Optional[Tensor] annotations off the deleted graph Tensor.	Deleted symbols build the tensorrt graph API and have no surviving consumer; the retained enums/params/RopeEmbeddingUtils are TRT-free and still imported by _torch / VisualGen at the same path. Full relocation (~55 consumer repoints) deferred per review (§ K).
models/modeling_utils.py	Trimmed 1,986→~580 lines to the config surface: kept SpeculativeDecodingMode, QuantConfig, LayerQuantConfig, PretrainedConfig, Gemma2/Gemma3ConfigGroup; removed the TRT model base classes (PretrainedModel, DecoderModelForCausalLM), per-arch helpers, and engine-build/load helpers.	Removed classes are TensorRT-engine-only; the LLM API and bench utilities only need the backend-agnostic Pydantic config types, kept here so import paths are unchanged.
quantization/functional.py	Gutted to a single retained function preprocess_weights_for_mixed_gemm; all tensorrt-based quant graph ops dropped.	The retained function is pure-torch weight interleaving still used by PyTorch weight loading; everything else built TRT graph ops.
_common.py	_init no longer loads the TensorRT plugin lib (only libth_common.so custom ops + MPI init); deleted default_net/default_trtnet/set_network/precision, engine (de)serialization helpers, and the net global. _is_building/_BuildingFlag kept with a TODO — they are the Python half of the IS_BUILDING Python↔C++ contract, removed together with the C++ isBuilding() in the C++ decouple step.	import tensorrt_llm must work without tensorrt; the graph-context machinery is TRT-only.
_common.py	_init re-dlopens libs/libtensorrt_llm.so with ctypes.RTLD_GLOBAL (non-Windows).	CI-failure fix (2026-07-09). The KV-cache transfer-agent wrappers (libtensorrt_llm_nixl_wrapper.so, libtensorrt_llm_ucx_wrapper.so) are dlopen'ed at runtime (transferAgent.cpp:44, RTLD_LAZY) without a DT_NEEDED on libtensorrt_llm.so (would be circular) — they resolve TllmException etc. from the process-global symbol table. Python extensions load RTLD_LOCAL; on main the promotion was a hidden side effect of _load_plugin_lib()'s CDLL(libnvinfer_plugin_tensorrt_llm.so, RTLD_GLOBAL) (plugin lib NEEDS libtensorrt_llm.so). Deleting the plugin load broke every CacheTransceiver/NIXL path (undefined symbol: _ZTI…TllmException, x86_64 + aarch64). Repro'd + fix validated locally via ctypes for both wrappers.
_utils.py	Removed import tensorrt as trt and all TRT dtype/version helpers (str_dtype_to_trt, trt_dtype_to_*, np_dtype_to_trt, torch_dtype_to_trt, dims_array, numpy_array, trt_version/trt_gte, is_same_dtype); pruned trt.DataType branches from TensorWrapper. Added CUASSERT + the cuda.bindings.runtime/cudart import guard, relocated from the deleted runtime/generation.py.	The dtype helpers wrap trt types with no surviving consumer; CUASSERT is a TRT-free CUDA-error checker still needed by four _torch consumers (§ D).
bench/build/build.py	Deleted the trtllm-bench build engine command (build_command, apply_build_mode_settings) and its BuildConfig/_tensorrt_engine imports; kept get_model_config/get_benchmark_engine_settings. DEFAULT_MAX_BATCH_SIZE/DEFAULT_MAX_NUM_TOKENS now read TorchLlmArgs.model_fields[...].default (was BuildConfig).	The engine builder dies with the backend; the kept heuristics are TRT-free and drive the PyTorch trtllm-bench auto-tuning. Defaults track the args class so they can't drift.

---

B. Package __init__ / re-exports / runtime relocation / packaging

File	What changed	Why
__init__.py	Dropped the TensorRT public surface: functional graph API (Tensor, constant, net_guard), Builder/BuilderConfig/build/BuildConfig, Network, Parameter, Module, PluginBase, str_dtype_to_trt/torch_dtype_to_trt, TrtLlmArgs; from ._common import _init only.	The graph/build API is removed; import tensorrt_llm works without tensorrt.
llmapi/__init__.py	Dropped TrtLlmArgs, BuildConfig, and the build_cache exports (BuildCacheConfig) from imports + __all__.	The TRT args/build surface and the build-cache feature are deleted.
models/__init__.py	All 139 TensorRT model-class imports deleted; now re-exports only the config surface (QuantConfig, LayerQuantConfig, PretrainedConfig, QuantAlgo, SpeculativeDecodingMode) from .modeling_utils; MODEL_MAP = {} kept for automodel importability.	Legacy model impls deleted; the PyTorch backend resolves models via tensorrt_llm._torch.models.
runtime/__init__.py	Removed the legacy runtime exports (Session/TensorInfo, GenerationSession, SamplingConfig, ModelRunner, ModelRunnerCpp, EncDecModelRunner, MultimodalModelRunner, KV cache manager v1, stopping/logits-processor lists); ModelConfig now imported from the new .model_config; kept PYTHON_BINDINGS + kv_cache_manager_v2.	Whole legacy runtime deleted; tensorrt_llm.runtime.ModelConfig import path preserved.
runtime/model_config.py (ADDED)	Backend-agnostic ModelConfig dataclass + from_model_config_cpp, relocated ~verbatim from the deleted runtime/generation.py. One deviation: language_adapter_config typed Optional[Any] (the concrete type lived in the deleted layers).	TensorRT-free home for the config consumed by the executor and _torch resource manager.
serialization.py	Removed TrtLlmArgs, plugin.plugin: PluginConfig, and builder: BuildConfig entries from the pickle allow-list.	Those classes/modules no longer exist and must not be deserializable.
setup.py	Removed the trtllm-build, trtllm-prune, trtllm-refit console scripts.	Their commands.{build,prune,refit} modules are deleted.

---

C. LLM API (llmapi/) + telemetry

File	What changed	Why
llmapi/llm.py	BaseLLM._get_llm_args_cls else-branch raises ValueError: Unknown backend … Supported backends are 'pytorch' and '_autodeploy'. (was llm_args_cls = TrtLlmArgs); the entire _TrtLLM class (workspace, save(), the TRT _build_model) deleted.	Behavior change: LLM(backend="tensorrt"/"trt") is rejected at construction; the engine-execution path no longer exists.
llmapi/llm.py	_on_trt_backend property removed (not kept as a False shim): workspace init collapsed to self._workspace = None (dead args.workspace + tempfile dropped), ctx-only guard collapsed to if is_ctx_only:.	Review (nv-guomingz): no dead always-False shims.
llmapi/llm.py	Dead _ModelInfo / _ModelRuntimeContext plumbing removed (never instantiated; runtime_context tokenizer read was a dead branch); _validate_args_for_torch_backend now validates against the TorchLlmArgs field set, with the comment/docstring reworded to describe allowed args without naming the removed backend.	Forced by TrtLlmArgs removal + review wording feedback.
llmapi/llm_args.py	Entire TrtLlmArgs class deleted (fields, validators, _load_config_from_engine/_ckpt); the backend telemetry categorical drops 'tensorrt'; update_llm_args_with_extra_dict strips all build_config handling; the whole _ModelFormatKind machinery removed (enum, get_model_format, model_format property + validator) since the format is always HF; TRT_LLMARGS_EXPLICIT_DOCSTRING/TRT_LLM_DOCSTRING removed outright.	The TRT config schema is removed; nothing branches on model format anymore (review: no dead compat aliases).
llmapi/llm_utils.py	Engine-build machinery deleted from ModelLoader/CachedModelLoader (build pipeline, engine cache staging, get_engine_dir, _build_model, multi-gpu build tasks); CachedModelLoader.__call__ is the pytorch/_autodeploy HF-download + quant-config path only; BuildConfig/BuildCacheConfig/_ModelInfo/_ModelRuntimeContext/model_format dropped from imports and __all__.	Removed capability: HF/ckpt→TRT-engine build + build cache.
llmapi/build_cache.py (DELETED)	Whole module deleted: BuildCache/CachedStage/CacheRecord and also BuildCacheConfig + get_build_cache_config_from_env.	The build-cache feature is dead post-removal (0 external refs; enable_build_cache was a TrtLlmArgs-only field, not in api_stability). Originally gutted, fully deleted per review.
usage/llmapi_config.py	golden_manifest() drops the "TrtLlmArgs": manifest_rows(TrtLlmArgs) entry + its import.	The class is deleted — functional change to the telemetry golden manifest, not cosmetic.
usage/usage_lib.py	_extract_architecture_class_name docstring/comments drop the deleted _ModelFormatKind.TLLM_CKPT mentions (logic unchanged).	The referenced symbol no longer exists.

---

D. Executor + _torch consumers

File	What changed	Why
executor/base_worker.py	Dropped builder.{Engine, ConfigEncoder, EngineConfig}; removed the in-memory-Engine executor branch, the engine-config→ModelConfig population, LoRA-plugin and prompt-adapter setup (attrs stay None); engine: Union[Path, Engine] → Path; ModelConfig now from ..runtime.	Workers no longer accept an in-memory TRT Engine.
executor/{executor,worker,ray_gpu_worker,rpc_worker}.py	Dropped from ..builder import Engine; narrowed the engine params to Path.	Same Engine-removal rewire.
_torch/pyexecutor/py_executor.py, _torch/disaggregation/native/transfer.py, _torch/disaggregation/native/bounce/{gather_scatter,impl}.py	from tensorrt_llm.runtime.generation import CUASSERT → from tensorrt_llm._utils import CUASSERT.	runtime/generation.py is deleted. The two bounce/ files arrived on main after the original scan; missing them was the root cause of every disaggregation failure in the first CI run (workers died on ModuleNotFoundError).
_torch/distributed/allreduce_helper.py (ADDED)	New home for CustomAllReduceHelper (workspace/IPC-buffer size computation) + force_all_reduce_deterministic, relocated ~verbatim from the deleted plugin/plugin.py.	Review (QiJune): don't keep a gutted plugin/ package — relocate the two TRT-free survivors and delete the package.
_torch/distributed/ops.py, _torch/custom_ops/torch_custom_ops.py, tests/microbenchmarks/{all_reduce,minimax_all_reduce}.py	Repointed from tensorrt_llm.plugin.plugin import CustomAllReduceHelper → the new allreduce_helper home.	The four consumers of the relocated helper.
_torch/models/checkpoints/hf/qwen3_moe_weight_mapper.py	Dropped the models.modeling_utils.DecoderModelForCausalLM import; model annotated nn.Module.	The TRT base class is gone; the PyTorch mapper only ever receives nn.Module.
_torch/auto_deploy/llm_args.py	Dropped the BuildConfig import and the vestigial build_config field + ensure_no_build_config validator.	Self-review fix: the field shadowed the deleted TrtLlmArgs.build_config; without this LLM(backend="_autodeploy") raised ImportError.

---

E. CLI + benchmarking + api_stability

File	What changed	Why
commands/serve.py	--backend → click.Choice(["pytorch", "_autodeploy"]); all tensorrt/trt branches, BuildConfig/TrtLlmArgs/_tensorrt_engine usage removed; the ChoiceWithAlias class deleted (its only purpose was the trt→tensorrt value alias); the no-op "build_config": None dict entry removed (defensive .pop("build_config") guards kept). CLI defaults (max_batch_size/max_num_tokens/max_beam_width) read TorchLlmArgs.model_fields[...].default; max_seq_len default → None (same value BuildConfig carried).	TensorRT serving path removed; defaults have a single source of truth.
tests/unittest/api_stability/references/trtllm_serve_cli.yaml	backend: type: narrowed Choice(['_autodeploy', 'pytorch', 'tensorrt']) → Choice(['_autodeploy', 'pytorch']).	The serve-CLI stability gate exists precisely to force this reviewer-visible diff for the surface change above.
commands/eval.py	--backend → click.Choice(["pytorch"]); _tensorrt_engine/BuildConfig + the tensorrt branch removed; defaults from TorchLlmArgs.model_fields.	PyTorch-only eval.
commands/bench.py	build_command import + registration removed.	Drops the trtllm-bench build subcommand.
bench/benchmark/__init__.py	_tensorrt_engine import dropped; default llm_cls → PyTorchLLM.	Benchmark defaults to the PyTorch LLM.
bench/benchmark/utils/asynchronous.py	from .._tensorrt_engine import LLM → from tensorrt_llm import LLM.	Public PyTorch LLM.
bench/benchmark/utils/general.py	ALL_SUPPORTED_BACKENDS drops "tensorrt".	No TRT bench backend.

---

F. Logging, profiling, LoRA, misc consumers, evaluate

File	What changed	Why
logger.py	The whole TRT-logger path deleted: trt_logger property, _trt_logger field, the set_level TRT branch, the module-level import tensorrt, and the TRT-severity column of severity_map (now [python_level] + optional polygraphy); _tensorrt_engine module abbreviation dropped. Zero tensorrt references remain — no lazy import / TYPE_CHECKING shim.	Its only consumer (profiler.check_gpt_mem_usage) is deleted (next row), so the earlier lazy-import gut was superseded by full deletion.
profiler.py	Deleted the dead check_gpt_mem_usage (legacy TRT-engine memory estimator; no in-repo callers even on main) + its now-unused _is_building/traceback imports. Zero tensorrt references remain.	Dead code whose only role was to force tensorrt/trt_logger dependencies into logger.py/profiler.py.
lora_manager.py	Deleted the dead TRT-model LoRA loaders load_hf_lora + load_nemo_lora (both mutate a TRT PretrainedModel in place) and the ColumnLinear/split_matrix_tp/pad_vocab_size imports. The PyTorch chain (load_torch_lora → load_torch_hf_lora/load_torch_nemo_lora, LoraManager runtime weight loading) is untouched.	Their only caller was lora_helper.use_lora, invoked exclusively by deleted TRT model classes — verified 0 live callers.
lora_helper.py	Deleted the dead use_lora wrapper; LoraConfig + the helpers the _torch path imports are untouched.	Same dead-chain removal.
quantization/quantize_by_modelopt.py	Removed the two TRT-checkpoint-export sub-paths (medusa-combine incl. combine_medusa_weight; the mllama config workaround) + 3 now-unused imports. Full file removal deferred (§ K).	They import the deleted models.medusa.weight / models.mllama.config.
serve/openai_server.py	_tensorrt_engine import + the isinstance(_tensorrt_engine.LLM) health branch dropped; LLM from llmapi.llm.	Module deleted; branch obsolete.
disaggregated_params.py	Module-level import tensorrt as trt (a tensorrt_libs preload hack) dropped.	Superseded by the single guarded preload in __init__.py (next row).
__init__.py	Added _preload_tensorrt_libs(): a try: import tensorrt / except ImportError: pass shim, run before the first tensorrt_llm.bindings import.	CI-failure fix (2026-07-09). The C++ bindings extension has a DT_NEEDED on libnvinfer.so.10 until the Phase C decouple; on main the module-level import tensorrt in _utils.py/disaggregated_params.py preloaded it from the tensorrt_libs wheel. Removing every such import broke import tensorrt_llm in CI (ImportError: libnvinfer.so.10), where TRT exists only as the pip wheel and not on the system loader path (local validation passed because the dev container has TRT installed system-wide). The guard keeps import working when the tensorrt Python package is blocked/absent in envs with system TRT libs.
evaluate/{lm_eval,mmlu,cnn_dailymail,covost2,json_mode_eval,longbench_v2}.py	Dropped from .._tensorrt_engine import LLM; Union[LLM, PyTorchLLM] hints narrowed to PyTorchLLM.	Only the PyTorch backend remains.
examples/apps/{chat,fastapi_server}.py	Repointed from tensorrt_llm._tensorrt_engine import LLM → from tensorrt_llm import LLM; BuildConfig usage replaced by direct LLM kwargs (max_batch_size/max_input_len/max_num_tokens/max_beam_width).	These kept LLM-API examples are exercised by active CI wrappers and imported the deleted symbols; BaseLlmArgs field defaults verified identical to the old BuildConfig defaults.

---

G. Docstring / comment style (review)

All prose added during the gut was trimmed per review: module docstrings that did not exist
on main were not (re)introduced, removal-narrative comments were dropped, and the added
comments/docstrings avoid double-backticks and em-dashes where the surrounding file style is
plain (lora_manager.py, models/__init__.py, models/modeling_utils.py,
runtime/model_config.py, _common.py) — brb-nv's uniformity nit, applied diff-wide.

---

H. Test-suite changes (this PR deletes the symbols, so it owns the breakage)

Deleted test files (9 of the 205 deletions)

File	Why
tests/integration/defs/accuracy/test_llm_api.py	TRT accuracy suite (module-level _tensorrt_engine import breaks pytest --co); PyTorch twin test_llm_api_pytorch.py carries the surviving coverage (§ K for the coverage-review disposition).
tests/integration/defs/llmapi/test_llm_e2e.py	TRT-only (builds engines; _tensorrt_engine.LLM + BuildConfig).
tests/integration/defs/disaggregated/test_configs/disagg_config{,_gen_only,_ctxtp2_gentp1}_trt_backend.yaml (3)	backend: trt disagg configs; their tests are deleted (below).
tests/unittest/llmapi/apps/_test_openai_multi_chat.py, _test_openai_consistent_chat.py	Fully TRT: build FP8 engines via _tensorrt_engine.LLM + BuildConfig and serve with --backend trt; their test_e2e.py wrappers also deleted; neither scheduled anywhere.
tests/unittest/tools/plugin_gen/ (2)	Orphaned helper importing the deleted tools.plugin_gen.core (its tests were removed in #15810).
tests/microbenchmarks/{build_time_benchmark,build_time_dashboard}.py + README.md (3)	Review-driven (chienchunhung, 2026-07-09). The TRT engine build-time microbenchmark: build_time_benchmark.py imports the deleted BuildConfig/build/AutoModelForCausalLM + tensorrt; build_time_dashboard.py is its only driver; the README documents only this benchmark. Executed by test_e2e.py::test_build_time_benchmark_sanity (also removed, with its l0_h100.yml post-merge-TRT entry — that stage is commented out in Jenkins, so this was latent, not an active failure).

Integration defs

File	What changed
disaggregated/test_disaggregated.py	Deleted test_disaggregated_{single_gpu,multi_gpu,benchmark_gen_only}_trt_backend + their config_map rows. The single-gpu one ran unwaived in pre-merge CI and would turn red the moment the backend rejection landed.
test_e2e.py	Deleted the TRT-only test_trtllm_bench_sanity / test_trtllm_bench_latency_sanity; dropped the TRT param from test_trtllm_bench_iteration_log and the trt_backend param from test_trtllm_bench_llmapi_launch; removed the [trt] param from the 5 openai/serve wrappers; removed the engine-build paths from trtllm_bench_prolog (now returns (model_path, dataset_path)) and BenchRunner (always --backend pytorch); rewrote test_trtllm_bench_request_rate_and_concurrency to pytorch (was --backend tensorrt, actively scheduled); dropped build --help from test_trtllm_bench_help_sanity; deleted the two TRT app-test wrappers.
llmapi/test_llm_api_qa.py + llmapi/_run_llmapi_llm.py	Deleted test_llm_args_type_tensorrt; test_llm_args_logging keeps only the pytorch half; the launcher script rewritten pytorch-only (no BuildConfig/engine save).
accuracy/accuracy_core.py, tests/unittest/utils/util.py	Shared harnesses: deleted-symbol imports + 6 dead TRT-only helpers dropped (0 consumers) so collection of surviving suites works.
accuracy/test_llm_api_pytorch.py	Added TestLlama3_1_8BInstruct::test_gather_generation_logits_cuda_graph (PyTorch port of the removed TRT test: gather_generation_logits=True + cuda_graph_config=CudaGraphConfig(); unique cuda-graph×gather-logits coverage; review follow-up, tburt-nv).

Unittest suites (de-TRT'd; pytest --co stays green)

File	What changed
llmapi/test_llm.py	2,886→1,358 lines; 47 TRT tests deleted (BuildConfig/TrtLlmArgs/engine-save/build-cache/TRT spec-dec/LoRA-prompt-adapter/…); 12 backend-neutral tests without pytorch twins trivially repointed (drop fast_build). All 15 cross-imported harness/fixture symbols kept with unchanged signatures (llm_test_harness, llm_get_stats*, llm_return_logprobs*, get_model_path, llama_model_path, prompts, cnn_dailymail_path, global_kvcache_config*, …) — the PyTorch twins (test_llm_pytorch.py, test_llm_multi_gpu_pytorch.py) and ~30 apps/_test_* files import them and run unchanged.
llmapi/test_llm_args.py	3,037→2,771 lines; TrtLlmArgs/BuildConfig/PluginConfig tests deleted; dual-parametrized tests narrowed to TorchLlmArgs; 2 backend-agnostic YAML tests converted to TorchLlmArgs; LoraConfig repointed to tensorrt_llm.lora_helper. Scheduled pre-merge.
llmapi/test_grpc.py	Repointed to the PyTorch LLM; fast_build=True dropped ×2 (gRPC service coverage kept).
llmapi/test_llm_quant.py	The 4 TRT in-flight-quantization tests deleted; the QuantConfig/ModelConfig parsing suite (~25 tests) kept, so its schedule entries stay valid.
llmapi/test_llm_download.py	Repointed to the PyTorch LLM; the deleted enable_build_cache arg dropped.
llmapi/test_llm_telemetry.py	TestTelemetryTRTBackend + the _tensorrt_engine import deleted; PyTorch telemetry classes untouched.
llmapi/test_llm_utils.py	Star-imports masked TRT usage: test_ModelLoader/test_CachedModelLoader (engine building, _ModelFormatKind) deleted; test_LlmArgs_default_gpus_per_node converted to TorchLlmArgs; explicit imports.
llmapi/test_memory_profiling.py	Module-level BuildConfig import (collection breaker in an active pre-merge block) replaced by explicit values; BaseLlmArgs defaults verified identical (2048/None/1/8192).
llmapi/run_llm.py, run_llm_with_postproc.py	Launchers rewritten pytorch-only (the postproc one is imported by a kept harness).
llmapi/apps/_test_openai_{chat,completions,misc,reasoning,multi_gpu}.py, _test_trtllm_serve_top_logprobs.py, _test_llm_chat.py, _test_llm_server.py	Backend fixtures narrowed to pytorch; if backend == "trt" branches/skips and TRT-only extra-options fixtures deleted; the two apps.{chat,fastapi_server} tests use LLM kwargs instead of BuildConfig.
usage/test_llmapi_config_capture.py	3 TrtLlmArgs-based capture tests deleted (quant_config is only a PrivateAttr on TorchLlmArgs — no pydantic-field carrier remains).
dynamo/test_imports.py	The ("tensorrt_llm.llmapi", "BuildConfig") import-assert tuple removed.
utils/test_logger.py	test_has_trt_logger deleted (asserted the deleted Logger.trt_logger).
conftest.py	The _maybe_force_ray TRT-LLM-class guard collapsed (permanently false once _tensorrt_engine is gone).

---

I. Test lists / CI metadata

File(s)	What changed
test-db/l0_{a10,a100,a30,dgx_h100,dgx_h200,gh200,h100,l40s}.yml	Delisted every entry whose test/param this PR deletes: the disagg *_trt_backend entries (incl. the pre-merge pytorch l0_a10 one), the [trt]/TRT-param e2e entries, test_trtllm_bench_sanity/latency_sanity, TestTelemetryTRTBackend, the accuracy/test_llm_api.py + llmapi/test_llm_e2e.py entries, and the medusa/multimodal entries in the still-active TRT-labelled blocks. Entries for de-TRT'd-but-kept files (e.g. test_llm_quant.py) stay.
qa/llm_function_core.txt, qa/llm_function_l20.txt	Same delisting (incl. the qa test_llm_api_qa tensorrt entry + a pre-existing duplicate line); added the ported test_gather_generation_logits_cuda_graph entry to llm_function_core.txt (was first added to the l20 list where the removed TRT entry lived, then moved per StanleySun639's review — L20 is a deprecated config slated for deletion).
waives.txt	Removed waives whose test/param no longer exists ([trt] openai waives, iteration_log[TRT-…], the orphaned multimodal tp:1-bs:1 waive) — the duplicated-waives and AST validators gate this.
.test_durations	9 stale duration keys for deleted tests removed.
ruff-legacy-baseline.json	Regenerated: the removal resolves ~660 baselined legacy-lint violations (439 of 929 legacy files no longer exist).

Dead backend: tensorrt test-db blocks whose Jenkins stages are commented out, and the
relocation of backend-neutral tests out of TRT-labelled CI stages + stage retirement in
L0_Test.groovy, are deliberately not in this PR — tracked in
trt-testlist-audit.md (Buckets 2–4) and plan.md.

---

J. Deleted files (205) — by area

Every deleted file serves the TensorRT execution/build path and has no surviving
consumer (AST import-graph + deleted-symbol scan over the surviving package, re-run
after each rebase and after the test sweep).

Area	Count	What / why
models/**	138	Legacy TensorRT model implementations (per-arch config.py/model.py/convert.py/weight.py, unet/**, enc_dec, multimodal encoders, generation_mixin, model_weights_loader) + package __init__s. Kept: models/__init__.py, models/modeling_utils.py (both gutted).
runtime/**	15	TensorRT Python runtime: session.py, generation.py (ModelConfig + CUASSERT relocated), model_runner{,_cpp}.py, enc_dec_model_runner.py, multimodal_model_runner.py, kv_cache_manager.py (v1), medusa_utils.py, redrafter_utils.py, memory_pools/**, processor_wrapper/**.
layers/**	15	TensorRT layer library.
tools/**	12	Plugin generator (plugin_gen/**), multimodal_builder.py, onnx_utils.py.
top-level	7	builder.py, network.py, graph_rewriting.py, module.py, parameter.py, python_plugin.py, top_model_mixin.py.
commands/**	3	build.py, prune.py, refit.py.
plugin/**	2	plugin.py + __init__.py — survivors relocated to _torch/distributed/allreduce_helper.py (§ D).
quantization/**	2	layers.py, quantize.py.
llmapi/build_cache.py	1	Whole build-cache feature (§ C).
_tensorrt_engine/	1	The TensorRT-engine LLM mirror.
tests	9	§ H table (2 integration files, 3 disagg TRT configs, 2 TRT app tests, tools/plugin_gen/ ×2).
Total	205

---

K. Review-driven changes (chronological)

1. Self-review fixes: AutoDeploy BuildConfig import (ImportError on backend="_autodeploy"); the logger.py lazy-TRT path (later superseded by full deletion of the trt_logger path once check_gpt_mem_usage was found dead).
2. nv-guomingz: error/comment wording without naming the removed backend; _on_trt_backend removed outright; the always-HF _ModelFormatKind machinery removed. His nvidia-modelopt requirements-drop suggestion rejected — modelopt is a core PyTorch-backend dependency (~9 non-quantize_by_modelopt modules import it).
3. QiJune: plugin/ package deleted with survivors relocated (done); full quantize_by_modelopt.py removal deferred to the examples-coordination PR; full functional.py relocation (~55 _torch repoints) deferred to a dedicated refactor.
4. Post-rebase re-verification: the two new bounce/ files' dangling runtime.generation imports caught and fixed — the root cause of every disaggregation failure in the first CI run (the run's 105 failures all mapped to this + the not-yet-pushed test sweep + the api_stability yaml; see trt-testlist-audit.md § CI-failure triage).
5. Test-suite sweep (§ H–I): motivated by the audit finding that live pre-merge entries (the disagg trt-backend test, test_llm_args.py, test_memory_profiling.py, the bench request-rate test) would break the moment the PR landed.
6. brb-nv (LoRA): approved; confirmed only the dead TRT LoRA chain was removed (the PyTorch load_torch_lora chain is untouched); comment-style nit applied diff-wide (§ G).
7. tburt-nv (coverage): test_gather_generation_logits_cuda_graph ported (§ H); test_logprobs already covered by pytorch unittest harnesses; W8A16 weight-only + fp8-rowwise are TRT in-flight-quantization feature gaps (the pytorch backend loads pre-quantized checkpoints only); aggregated Ulysses cp2 looks feasible but unvalidated — all tracked as follow-ups in plan.md.

Summary by CodeRabbit

• Refactor
  • Simplified the package to focus on PyTorch and AutoDeploy workflows.
  • Removed TensorRT-specific backend options, CLI commands, and several legacy exports.
  • Updated benchmark, serve, and eval defaults to align with the PyTorch configuration.
  • Streamlined model and layer availability across the public API.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[Wanli-Jiang]

/bot run --disable-fail-fast

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR removes the TensorRT engine-building backend from TensorRT-LLM. It deletes builder, layers, graph-rewriting, and per-architecture model modules; removes trtllm-build/prune/refit CLI commands and TrtLlmArgs/_TrtLLM; narrows executor/worker engine typing to Path; and updates bench/serve/eval CLIs, _common.py, _utils.py, and logger.py to drop TensorRT dependencies, leaving only PyTorch and autodeploy backends.

Changes

TensorRT backend removal

Layer / File(s)	Summary
CLI entry points and commands setup.py, tensorrt_llm/commands/bench.py, tensorrt_llm/commands/build.py, tensorrt_llm/commands/prune.py, tensorrt_llm/commands/refit.py, tensorrt_llm/commands/eval.py, tensorrt_llm/commands/serve.py, tensorrt_llm/bench/build/build.py, tensorrt_llm/bench/benchmark/utils/general.py	Removes trtllm-build/prune/refit console scripts and command modules; restricts --backend choices to pytorch/_autodeploy and sources CLI defaults from TorchLlmArgs.model_fields in bench/eval/serve.
LLM API args and llm.py backend validation tensorrt_llm/llmapi/__init__.py, tensorrt_llm/llmapi/llm_args.py, tensorrt_llm/llmapi/llm.py	Deletes TrtLlmArgs class and _TrtLLM implementation; backend selection/validation now only recognizes pytorch/_autodeploy and rejects other kwargs with a TensorRT-removed error.
Model loader and build cache removal tensorrt_llm/llmapi/llm_utils.py, tensorrt_llm/llmapi/build_cache.py	Removes HF/checkpoint loading, in-memory engine build, and engine persistence methods from ModelLoader/CachedModelLoader; get_engine_building_cache_stage now raises NotImplementedError.
Builder/graph-rewriting/layers module removal tensorrt_llm/builder.py, tensorrt_llm/graph_rewriting.py, tensorrt_llm/layers/*.py, tensorrt_llm/models/*	Deletes the builder, graph-rewriting pass framework, all layer implementations (linear, conv, moe, mlp, normalization, embedding, lora, recurrent, etc.), and per-architecture model config/convert/model files; empties MODEL_MAP.
Core init, _common, _utils TensorRT decoupling tensorrt_llm/__init__.py, tensorrt_llm/_common.py, tensorrt_llm/_utils.py, tensorrt_llm/logger.py, tensorrt_llm/disaggregated_params.py	Removes TensorRT plugin loading, TensorRT dtype conversion helpers, and engine serialization utilities; adds CUASSERT CUDA helper; Logger lazily loads the TensorRT logger only when accessed.
Executor/worker engine typing narrowed to Path tensorrt_llm/executor/*.py, tensorrt_llm/_torch/pyexecutor/py_executor.py, tensorrt_llm/_torch/disaggregation/native/transfer.py	Narrows engine parameter typing from Union[Path, Engine] to Path across executor/worker classes; removes Engine-based construction paths; updates CUASSERT import source.
Auto-deploy args, weight mapper, LoRA manager TensorRT cleanup tensorrt_llm/_torch/auto_deploy/llm_args.py, tensorrt_llm/_torch/models/checkpoints/hf/qwen3_moe_weight_mapper.py, tensorrt_llm/lora_manager.py	Removes build_config field/validator from auto_deploy LlmArgs, narrows weight mapper typing, and replaces TensorRT LoRA construction with NotImplementedError.
Benchmark and evaluation LLM class defaults tensorrt_llm/bench/benchmark/*, tensorrt_llm/evaluate/*.py	Switches default/annotated LLM class from _tensorrt_engine.LLM to PyTorchLLM across bench utilities and all evaluate scripts.

Estimated code review effort: 4 (Complex) | ~75 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#15814: Both PRs remove TensorRT/Triton TRT backend workflow surfaces from the codebase and its associated test/build tooling.

Suggested reviewers: tburt-nv, QiJune, xinhe-nv, yechank-nvidia, nv-guomingz, mlefeb01

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title includes the ticket, type, and correctly summarizes the main breaking removal of the legacy TensorRT backend.
Description check	✅ Passed	The description is detailed and covers scope, validation, and checklist items, though the dedicated Description and Test Coverage sections are sparse.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[tensorrt-cicd]

PR_Github #57955 [ run ] triggered by Bot. Commit: 6a4cdca Link to invocation

[Wanli-Jiang]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #58959 [ run ] triggered by Bot. Commit: 5f42d83 Link to invocation

[tensorrt-cicd]

PR_Github #58959 [ run ] completed with state SUCCESS. Commit: 5f42d83
/LLM/main/L0_MergeRequest_PR pipeline #47491 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[chienchunhung]

Thanks for checking the broader tree.

One item I'd like to flag: tests/unittest/others/test_leak.py calls APIs removed here, including Builder, net_guard, graph_rewriting, and LLaMAForCausalLM. Because those lookups occur inside the test body, collection succeeds, but running this test—or the documented full unit suite—fails at tllm.Builder(). Please remove the test if its legacy coverage is no longer applicable, or migrate it before merge. However, it seems like test_leak.py may not be exercised in regular CI; not sure why.

There may be other consumers as dormant examples, helpers, and legacy integration paths that are not currently selected by the checked-in CI configuration. This is not blocking this PR from merging, but would be great to track them in follow-up PRs.

[Wanli-Jiang]

/bot run --only-multi-gpu-test --disable-fail-fast

[tensorrt-cicd]

PR_Github #59083 [ run ] triggered by Bot. Commit: f18278b Link to invocation

[tensorrt-cicd]

PR_Github #59083 [ run ] completed with state SUCCESS. Commit: f18278b
/LLM/main/L0_MergeRequest_PR pipeline #47600 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[Wanli-Jiang]

Test conclusion

Pre-merge

• Singel X86: full passed at [TRTLLM-14022][feat] BREAKING: Remove python modules and tests for legacy TensorRT backend #15918 (comment)

• Single SBSA: full passed at [TRTLLM-14022][feat] BREAKING: Remove python modules and tests for legacy TensorRT backend #15918 (comment)

• Multi X86: most passed at [TRTLLM-14022][feat] BREAKING: Remove python modules and tests for legacy TensorRT backend #15918 (comment), except below 4 tests, which were passed locally.
  Test / DGX_B200-4_GPUs-PyTorch-2 / test_auto_dtype – DGX_B200-4_GPUs-PyTorch-2.accuracy.test_llm_api_pytorch.TestDeepSeekV4Flash
  Test / DGX_B200-4_GPUs-PyTorch-2 / test_nvfp4[adp4_cutedsl] – DGX_B200-4_GPUs-PyTorch-2.accuracy.test_llm_api_pytorch.TestQwen3_5_397B_A17B
  Test / DGX_B200-2_GPUs-PyTorch-1 / test_bart_pytorch_generate_encoder_decoder_end_to_end[bf16-kv-v1-cuda-graph-on-greedy-tp2-bart-large-cnn] – DGX_B200-2_GPUs-PyTorch-1.llmapi.test_llm_api_pytorch_bart
  Test / DGX_B200-4_GPUs-PyTorch-2 / test_openai_kv_cache_contamination – DGX_B200-4_GPUs-PyTorch-2.test_e2e

• Multi SBA: full passed at [TRTLLM-14022][feat] BREAKING: Remove python modules and tests for legacy TensorRT backend #15918 (comment)

Post-merge

• Singel X86: full passed at [TRTLLM-14022][feat] BREAKING: DON'T REVIEW Remove legacy TensorRT Python backend #16213 (comment)

• Single SBSA: full passed at [TRTLLM-14022][feat] BREAKING: DON'T REVIEW Remove legacy TensorRT Python backend #16213 (comment)

• Multi X86: most passed at [TRTLLM-14022][feat] BREAKING: DON'T REVIEW Remove legacy TensorRT Python backend #16213 (comment) the below 4 tests which can pass locally.
  Test / DGX_B200-8_GPUs-AutoDeploy-Post-Merge-1 / test_mtp[nvfp4_ws8_80gb-trtllm] – DGX_B200-8_GPUs-AutoDeploy-Post-Merge-1.accuracy.test_llm_api_autodeploy.TestNemotronSuperV3

  Test / RTXPro6000D-4_GPUs-PyTorch-Post-Merge-1 / test_nvfp4_4gpus[moe_backend=CUTLASS-mtp_nextn=2-tp4-fp8kv=True-attention_dp=True-cuda_graph=True-overlap_scheduler=True-low_precision_combine=False-torch_compile=False] – RTXPro6000D-4_GPUs-PyTorch-Post-Merge-1.accuracy.test_llm_api_pytorch.TestDeepSeekV3Lite

  Test / DGX_B200-4_GPUs-PyTorch-Post-Merge-1 / test_nvfp4_attn_multi_gpus – DGX_B200-4_GPUs-PyTorch-Post-Merge-1.accuracy.test_llm_api_pytorch.TestDeepSeekV32

  Test / DGX_B200-8_GPUs-PyTorch-VisualGen-PerfSanity-Post-Merge-1 / test_visual_gen_e2e[vg_upload-ltx2_blackwell-ltx2_2stage_bf16_i2v_cfg2_ulysses4_compile_on] – DGX_B200-8_GPUs-PyTorch-VisualGen-PerfSanity-Post-Merge-1.perf.test_visual_gen_perf_sanity

• Multi SBSA: most passed at [TRTLLM-14022][feat] BREAKING: DON'T REVIEW Remove legacy TensorRT Python backend #16213 (comment), except 7/8 were perf and confirmed no regression, and the last one were confirmed no related to our code changes.
  Test / GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-1 / test_e2e[disagg_upload-e2e-gb200_gpt-oss-120b-fp4_1k1k_con2048_ctx1_tp1_gen1_dep2_eplb0_mtp0_ccb-NIXL] – GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-1.perf.test_perf_sanity

Test / GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-2 / test_e2e[disagg_upload-gen_only-gb200_gpt-oss-120b-fp4_1k1k_con64_ctx1_tp1_gen1_tp4_eplb0_mtp0_ccb-NIXL] – GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-2.perf.test_perf_sanity

Test / GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-5 / test_e2e[disagg_upload-gen_only-gb200_gpt-oss-120b-fp4_8k1k_con4_ctx1_tp1_gen1_tp4_eplb0_mtp0_ccb-NIXL] – GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-5.perf.test_perf_sanity

Test / GB200-8_GPUs-2_Nodes-PyTorch-PerfSanity-Node2-GPU8-Post-Merge-2 / test_e2e[aggr_upload-ctx_only-gb200_deepseek-r1-fp4_128k8k_con1_ctx1_pp8_gen1_tep8_eplb0_mtp3_ccb-NIXL] – GB200-8_GPUs-2_Nodes-PyTorch-PerfSanity-Node2-GPU8-Post-Merge-2.perf.test_perf_sanity

Test / GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU2-GEN1-NODE2-GPU8-Post-Merge-5 / test_e2e[disagg_upload-gen_only-gb300_glm-5-fp4_8k1k_con1_ctx1_dep2_gen1_tep8_eplb0_mtp3_ccb-NIXL] – GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU2-GEN1-NODE2-GPU8-Post-Merge-5.perf.test_perf_sanity

Test / GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-1 / test_e2e[disagg_upload-e2e-gb300_deepseek-r1-fp4_128k8k_con256_ctx1_pp4_gen1_dep8_eplb0_mtp1_ccb-NIXL] – GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-1.perf.test_perf_sanity

Test / GB300-36_GPUs-9_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU2-GEN1-NODE8-GPU32-Post-Merge-2 / test_e2e[disagg_upload-gen_only-gb300_glm-5-fp4_8k1k_con512_ctx1_dep2_gen1_dep32_eplb0_mtp3_ccb-NIXL] – GB300-36_GPUs-9_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU2-GEN1-NODE8-GPU32-Post-Merge-2.perf.test_perf_sanity

Test / GB200-4_GPUs-PyTorch-Post-Merge-1 / test_nvfp4_4gpus_online_eplb[moe_backend=TRTLLM] – GB200-4_GPUs-PyTorch-Post-Merge-1.accuracy.test_llm_api_pytorch.TestNemotronV3Super

[Wanli-Jiang]

/bot skip --comment "full test analysis can be found in #15918 (comment) and we can regard it as passed"

[juney-nvidia]

Approve due to that this PR has already received the approvals before and just get invalidated due to the changes of Code owner file.

[tensorrt-cicd]

PR_Github #59167 [ skip ] triggered by Bot. Commit: 42ce5e0 Link to invocation

[tensorrt-cicd]

PR_Github #59167 [ skip ] completed with state SUCCESS. Commit: 42ce5e0
Skipping testing for commit 42ce5e0

Link to invocation