[TRTLLM-14026][feat] BREAKING: Remove C++ modules for legacy TRT backend

[Wanli-Jiang]

Overview

This PR severs the C++ tree's compile- and link-time dependency on TensorRT so the
shared core (runtime, batch manager, executor API, KV cache, sampling, kernels,
nanobind bridge) builds, links, and runs without the TensorRT library, and closes
the packaging gate (requirements.txt drops tensorrt). It is the C++ half of the
staged TensorRT-backend removal, following examples #15763, docs #15767, tests
#15810, the Triton C++ backend #15907, and the Python backend removal (TRTLLM-14022,
PR #15918, merged to main as 7e833f7220).

• 576 files (+2,703 / −79,322; 2 A / 302 M / 264 D / 8 R) at 845ab6559e (grew from the original 512 via
  the review/CI-driven amends recorded in §§ F–H: gtest fixes, MPI
  includes, --benchmarks sweep incl. docker/Dockerfile.multi, test-db lint
  fixes, benchmarks/cpp → benchmarks/ relocation, test_leak.py deletion,
  first-CI-run fixes, the two OSS-mode gtest regression fixes, two
  rebase-absorption respells of new-on-main nvinfer1 usages, and the 2026-07-16
  review-comment amends — § H last table — incl. the dormant Executor-binding
  client deletion).
• Approach — internal types + delete the engine path in one PR: the shared core
  used two nvinfer1 types as common currency (DataType, Dims).
  Those now live in common/tllmDataType.h as standalone tensorrt_llm:: types with
  identical enumerator values and layout (serialization stays byte-compatible),
  and every surviving file is respelled nvinfer1::X → tensorrt_llm::X. Because
  enum class types do not interconvert, the TensorRT-engine execution path (which
  must keep the real nvinfer1 types) cannot compile against the migrated core —
  so its removal is forced into the same PR, matching the shape of the original
  draft (Remove tensorrt backend QiJune/TensorRT-LLM#20, commit 59eb547a6d), re-derived on the live tree.
• Test ownership rule (same as Phase B): this PR deletes the C++ binaries and
  bindings, so it owns every test that drives them — engine-driven C++ gtests,
  their Python integration wrappers, and all test-db/waives/.test_durations
  entries are removed here.
• Validation: full build_wheel.py without --trt_root green; readelf -d
  on libtensorrt_llm.so/libth_common.so/bindings.*.so shows no libnvinfer
  NEEDED entry; import tensorrt_llm + bindings succeed with the tensorrt
  package hard-blocked; test_datatype_parity.py green (values 0–11 match legacy);
  pytest --co tests/unittest collects clean; quickstart e2e on H100 produces
  correct output; pre-commit fully green.
• One row per distinct reason — a file changed for two different reasons gets two rows.

---

A. Internal types (serialization-compatible)

File	What changed	Why
cpp/include/tensorrt_llm/common/tllmDataType.h (ADDED)	Standalone DataType (enum class : int32_t, values 0–11 = kFLOAT..kE8M0) and Dims (int32_t nbDims + int64_t d[8], MAX_DIMS=8), defined in tensorrt_llm::common (matching the other common/*.h headers, per review — VALLIS-NERIA) and hoisted into tensorrt_llm with using common::DataType; using common::Dims; so the tree-wide tensorrt_llm::X spelling works. (An ILogger shim was initially included and removed on review — its only would-be consumers are deleted by this PR; review thread BowenFu.) Includes only <cstdint> + common/config.h.	The two nvinfer1 types the shared core uses as common currency need a TensorRT-free home. Enumerator values intentionally mirror nvinfer1::DataType and the Dims layout mirrors TRT-10 Dims64 so previously-serialized executor configs and KV-cache metadata stay byte-compatible.
206 surviving files (the full list = every cpp/ file in the commit diff not otherwise named in this document)	Mechanical respell nvinfer1::DataType → tensorrt_llm::DataType (~2,000 uses), nvinfer1::Dims → tensorrt_llm::Dims (~60), 53 NvInfer*.h includes replaced by tllmDataType.h; 151 explicit tllmDataType.h includes added to files that had used the types only transitively (the new names are declared only in the new header).	Core migration. Breakdown: kernels 43, runtime 28, batch_manager 24, public cpp/include 23, thop 15, layers 9, nanobind 8, executor 6, common 6, tests 73, micro_benchmarks 1.
12 files with dead NvInfer*.h includes (runtime/{loraCache,worldConfig}.h, batch_manager/cacheFormatter.h, kernels/…/moe_util_kernels.h, kernels/weightOnlyBatchedGemv/cudaCoreGemm{,NVFP4}.h, runtime/loraManager.cpp, thop/{allgather,reducescatter,noAuxTc,IndexerTopK}Op.cpp, unit_tests/kernels/cudaCoreGemm/cudaCoreGemmKernelTest.cpp)	Include deleted (2 were already commented out).	They never referenced nvinfer1 at all; each verified to include its cuda headers explicitly.
cpp/include/tensorrt_llm/runtime/iTensor.h	Also dropped the dead namespace nvinfer1 { class IExecutionContext; } fwd-decl.	Its only real users (tllmRuntime.{h,cpp}) are deleted and included NvInferRuntime.h themselves. (The draft leaked this into its end state.)
cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu	File-local helper moeLoraNvInferType → moeLoraDataType (3 uses) + comment reword.	Identifier referenced nvinfer in name only.
cpp/tests/unit_tests/multi_gpu/kernels/allReduce/gemmAllReduceTest.cu	Dropped using namespace nvinfer1;.	All uses already qualified after the respell.
tests/unittest/bindings/test_datatype_parity.py (ADDED)	Asserts tensorrt_llm.bindings.DataType member set + integer values match the legacy nvinfer1::DataType values (FLOAT=0 … NVFP4=10).	Serialization/format-compatibility guard; auto-scheduled via the existing unittest/bindings entries in l0_a10/h100/gh200.yml.
cpp/tests/unit_tests/batch_manager/kvCacheManagerTest.cpp (2026-07-15 amend)	Respell of the 7 nvinfer1::DataType:: sites in FP4AttentionWithHalfRecurrentStatesPoolTest.	Rebase absorption: main #16304 (merged during an external rebase) added new nvinfer1 usages to a migrated file — one compile error fails the whole google-tests fixture build, which is why ALL H100/DGX CPP CI stages failed.
cpp/tensorrt_llm/kernels/rmsNormFp4QuantKernels.{h,cu} (2026-07-15 post-rebase amend)	#include <NvInferRuntime.h> → tllmDataType.h; 6 nvinfer1::DataType sites respelled.	Rebase absorption: main #14848 (RMSNorm nvfp4 quant fusion, new files) landed with nvinfer1 types; its thop caller already passes the migrated tensorrt_llm::DataType from TorchUtils::dataType. Caught by the mandatory post-rebase scan suite (check 1).

B. Engine execution path — deleted C++ sources

Area	Files	What / why
cpp/tensorrt_llm/plugins/**	131	The TensorRT plugin library (nvinfer_plugin_tensorrt_llm) — only consumed by TRT engines.
cpp/tensorrt_llm/batch_manager/	20	TRT model adapters (trtGptModel*, trtEncoderModel, trtGptModelFactory) + engine-only buffers/algos (runtimeBuffers, encoderBuffers, transformerBuffers, promptTuningBuffers, rnnStateBuffers, loraBuffers, handleContextLogits, handleGenerationLogits, logitsPostProcessor, makeDecodingBatchInputOutput, updateDecoderBuffers) — the PyTorch pyexecutor has its own Python implementations.
cpp/tensorrt_llm/runtime/	7	Engine runtime wrappers: tllmRuntime.{h,cpp}, tllmStreamReaders.{h,cpp}, layerProfiler.{h,cpp}, tllmLogger.cpp.
cpp/include/tensorrt_llm/	12	Public headers of the above + runtime/rawEngine.h, runtime/tllmLogger.h, executor/disaggServerUtil.h, plugins/api/tllmPlugin.h.
cpp/tensorrt_llm/executor/	5	executor.cpp, executorImpl.{h,cpp} (the engine Executor), disaggServerUtil.cpp, and model.h (0 remaining consumers — verified).
cpp/tensorrt_llm/executor_worker/	2	The executorWorker orchestrator binary (engine-mode only).
cpp/tensorrt_llm/nanobind/	4	The C++ Executor binding (executor/executor.{h,cpp}) and the ModelSpec binding (testing/modelSpecBinding.{h,cpp}) — 0 Python users each after the test deletions below; kvCacheManagerTestUtilBinding (live users) kept.
cpp/tensorrt_llm/testing/	3	modelSpec.{h,cpp} + the testing_src CMake target — engine test-data path helper, orphaned. kvCacheManagerTestUtil.h (header-only, live binding) kept in place.
cpp/cmake/modules/FindTensorRT.cmake	1	No find_package(TensorRT) remains.

C. Relocations (retained symbols out of deleted files)

File	What changed	Why
cpp/include/tensorrt_llm/executor/executor.h	KVCacheEvent constructor + executor::version(), relocated from the deleted executor.cpp/executorImpl — now inline in the header (version() returns kTensorRtLlmVersion from the public generated version.h, newly included; the ctor is defined in-class). An interim relocation file kvCacheEvent.cpp existed and was dropped on review (VALLIS-NERIA: trivial enough for the header).	Backend-agnostic; the retained KV-cache event path (kvCacheEventManager, serialization, nanobind) must keep linking.

D. Shared C++ files decoupled from the removed engine runtime

File	What changed	Why
batch_manager/utils/inflightBatchingUtils.{h,cpp}	Removed copyGenerationLogits (needs RuntimeBuffers::GenerationLogitsCache), findOutputTensor, copyAdditionalOutputs, and the CudaGraphExecutor/CudaGraphExecutorCache classes (drive TllmRuntime contexts). The PyTorch-used helpers stay.	Their parameter types are deleted with the engine path; PyTorch has its own cuda-graph and logits handling.
runtime/lookaheadBuffers.{h,cpp}	Removed the LookaheadRuntimeBuffers class (ctor takes TllmRuntime const&); kept LookaheadDecodingBuffers. layers/lookaheadDecodingUtils.h include → common/cudaUtils.h (for divUp).	Engine-side half of lookahead; the decoder-side half survives and is used by the shared decoder.
batch_manager/medusaBuffers.{h,cpp}	Removed the MedusaBuffers engine ctor (takes TllmRuntime const&, reads engine.getTensorDataType("medusa_logits")) + medusaModule/speculativeChoicesUtils includes; kept reshape/insertInputTensors.	Same split: engine ctor dies, shared methods stay.
batch_manager/dataTransceiver.cpp	Dropped the runtimeBuffers.h include.	Header deleted; nothing else used it.
nanobind/runtime/bindings.cpp	Removed the TllmRuntime class binding (~40 lines) + tllmRuntime.h include.	The Python TllmRuntime surface died with the Python backend; 0 remaining Python references (verified).
nanobind/batch_manager/algorithms.cpp	Removed the LogitsPostProcessor algorithm binding + include.	Its C++ impl is deleted; 0 Python users (pyexecutor uses its own Python logits path).
nanobind/executor/bindings.cpp	Dropped the deleted Executor class registration.	Config/type bindings (ExecutorConfig, request/response types) are untouched — the PyTorch backend keeps using them.
nanobind/bindings.cpp	Dropped the modelSpecBinding.h include + initBindings(mInternalTesting) call; kept initKvCacheTestUtilBindings.	ModelSpec binding deleted (§ B).
common/opUtils.h + common/attentionOp.cpp + tensorrt_llm/_common.py	Removed isBuilding() (env probe), the if (isBuilding()) return 0; gate in attentionOp::initialize, and the Python _BuildingFlag/_is_building writers of IS_BUILDING (+ now-unused wraps import).	The IS_BUILDING Python↔C++ contract only skipped NCCL setup during trtllm-build; no writer remains. Both halves deleted together (as the Phase B TODO in _common.py said).

E. Python executor layer (the engine path's last consumers)

File	What changed	Why
tensorrt_llm/executor/base_worker.py	Removed _create_engine() (constructed tllm.Executor(engine_path, ModelType.DECODER_ONLY, executor_config)); setup_engine now asserts llm_args is not None and always builds the PyExecutor; the three isinstance(self.engine, tllm.Executor) branches in fetch_stats/fetch_kv_cache_capacity/fetch_kv_cache_events collapsed to the PyTorch side.	The nanobind Executor class no longer exists — any tllm.Executor attribute access would raise at runtime. llm_args=None was unreachable from the LLM API since Phase B.
tensorrt_llm/executor/worker.py	block_subordinates drops the isinstance(self.engine, tllm.Executor) early-exit branch.	Same.
tensorrt_llm/__init__.py	Removed the _preload_tensorrt_libs() shim (import of the tensorrt package to preload libnvinfer for the bindings).	Its reason for existing — bindings.*.so's DT_NEEDED on libnvinfer.so.10 — is gone; verified by importing with tensorrt hard-blocked.

F. Build & packaging (the final gate)

File	What changed	Why
cpp/CMakeLists.txt	Removed find_package(TensorRT 10 REQUIRED COMPONENTS OnnxParser) + set(TRT_LIB TensorRT::NvInfer), the TensorRT::NvInfer include-dir injection, the BUILD_BENCHMARKS option/messages/add_subdirectory(benchmarks/cpp), and nvinfer_plugin_tensorrt_llm from the build-time measurement list.	Nothing links or includes TensorRT anymore.
cpp/tensorrt_llm/CMakeLists.txt	Removed add_subdirectory(plugins), add_subdirectory(executor_worker), add_subdirectory(testing), ${TRT_LIB} and testing_src from the link list.	Their targets are deleted.
cpp/tensorrt_llm/{batch_manager,runtime,executor}/CMakeLists.txt, nanobind/CMakeLists.txt	Deleted sources pruned from the SRCS lists; executor/executor.cpp + testing/modelSpecBinding.cpp dropped from the nanobind module.	Match the file deletions.
scripts/build_wheel.py	The build() function's trt_root param default → None; the "Ensure base TRT is installed" venv check removed; nvinfer_plugin_tensorrt_llm/executorWorker dropped from targets; the plugin .so/.dll and executorWorker install steps and the bin/ dir removed; the --benchmarks flag + target removed. The --trt_root CLI flag is kept (old default) as a [DEPRECATED] no-op for caller compatibility (§ H review fixes).	Building no longer needs TensorRT anywhere; C++ benchmarks are deleted.
setup.py	package_data drops bin/executorWorker, libs/libnvinfer_plugin_tensorrt_llm.so, libs/nvinfer_plugin_tensorrt_llm.dll.	Not built anymore.
requirements.txt	Drops tensorrt~=10.16.1 — the global packaging gate of the staged removal.	Python tree TRT-free since Phase B; C++ tree TRT-free as of this PR.
jenkins/Build.groovy	Dropped --benchmarks from the wheel build line, the x86 Debug build line (missed initially — caught by CI run L0_MergeRequest_PR #47240, unrecognized arguments: --benchmarks; fixed 2026-07-13 amend), the Step-4 benchmark packaging block (bertBenchmark/gptManagerBenchmark/disaggServerBenchmark/libnvinfer_plugin copies into the tarball), and the build_cpp_examples.py step.	Those binaries no longer exist; leaving the cp lines would fail every CI build.
jenkins/scripts/perf/local/slurm_install.sh (2026-07-13 amend)	Dropped --benchmarks + --trt_root /usr/local/tensorrt from its build_wheel call.	--benchmarks no longer parses; TRT root no longer needed.
docs/source/blogs/Best_perf_practice_on_DeepSeek-R1_in_TensorRT-LLM.md (2026-07-13 amend)	Build command updated: dropped --benchmarks, --trt_root, and --python_bindings (the last was already invalid on main).	Documented command must parse post-removal; actual benchmarking in the blog uses trtllm-bench, unaffected.
docker/Dockerfile.multi (2026-07-13 amend)	Wheel stage: BUILD_WHEEL_ARGS dropped --benchmarks (docker/Makefile derives its default from this line, so it inherits the fix). Release stage: dropped the cpp/build/benchmarks bind mount, the bertBenchmark/gptManagerBenchmark/disaggServerBenchmark copies and .cpp source removals, and the bin/ symlink + test -f bin/executorWorker + ldd bin/executorWorker checks; the lib sanity checks now use test -f lib/libtensorrt_llm.so and ldd lib/libth_common.so (DT_NEEDED on libtensorrt_llm.so verified).	The wheel build would reject --benchmarks; the binaries/sources no longer exist; the wheel no longer ships bin/. jenkins/current_image_tags.properties intentionally untouched — devel-image content is unchanged (TRT inside the pinned images is unused but harmless); stripping TRT from the devel images is a follow-up with its own image rebuild + tag bump.
scripts/get_wheel_from_package.py	Removed the benchmark-binary extraction loop + benchmarks_dir setup.	Tarball no longer carries them.
scripts/build_cpp_examples.py (DELETED) + examples/cpp/** (11 files)	The C++ executor examples (executorExampleBasic/Advanced/Disaggregated/…) all construct tle::Executor(<engine path>).	Engine-only examples die with the class; the Jenkins step that built them is removed above.

G. C++ tests — deleted (engine-driven) vs fixed (kept)

Deleted

File(s)	Why
cpp/tests/e2e_tests/batch_manager/{trtGptModelTest,trtGptModelRealDecoderTest,trtEncoderModelTest}.cpp	Drive the deleted TRT model adapters.
cpp/tests/e2e_tests/executor/** (executorTest.{h,cpp}, executorMockTest.cpp, encDecTest.cpp, disaggExecutor.h, disaggExecutorTest.cpp, CMakeLists.txt)	Drive the deleted C++ Executor with built engines.
cpp/tests/unit_tests/executor/executorTestSmall{,ArbitraryOutputTensors}.cpp	Same, via in-memory engines from tests/utils/engines.
cpp/tests/unit_tests/runtime/tllmRuntimeTest.cpp, cpp/tests/unit_tests/batch_manager/encDecBeamSearchTest.cpp	Test TllmRuntime / RuntimeBuffers::GenerationLogitsCache — both deleted.
cpp/tests/utils/{engines,executorUtils,common}.{h,cpp} + CMakeLists.txt (the testingUtils lib)	engines builds TRT engines in-memory (the only survivor-file user of TRT graph APIs); executorUtils drives executor::Executor; common holds engine paths/logits helpers. All consumers deleted → whole tests/utils/ dies.
cpp/tests/resources/scripts/build_*_engines.py + generate_expected_*_output.py + generate_hf_gpt_output.py + io_converter.py + build_engines_utils.py (22 files)	The engine-build machinery for the tests above (already broken since Phase B deleted the Python builder). Kept: generate_test_lora_weights.py — live user (lora_setup fixture feeds the surviving C++ LoRA unit tests, no engines involved).
benchmarks/cpp/{bertBenchmark,gptManagerBenchmark,disaggServerBenchmark}.cpp + utils/utils.{h,cpp} + CMakeLists.txt	Engine-driven C++ benchmarks (bertBenchmark uses TllmRuntime, the others the C++ Executor). Kept + relocated (2026-07-13 amend): prepare_dataset.py + the Python utils/ feed trtllm-bench and moved benchmarks/cpp/ → benchmarks/ (the cpp/ level was meaningless post-deletion); READMEs merged into one benchmarks/README.md (the old top-level one still recommended the deleted C++ benchmarks); references updated (test_prepare_dataset.py, test_perf.py, sample_performance_alignment.sh, perf docs ×4, create_standalone_package.py); legacy-files.txt renamed + gen-configs regenerated pyproject.toml/ruff-legacy.toml/.pre-commit-config.yaml + ruff-legacy-baseline.json re-keyed. Historical release-notes.md mention left as-is.
tests/unittest/others/test_leak.py (in the Phase B commit, 2026-07-13 amend, per PR #15918 review)	Collection-safe but its body calls Phase-B-deleted APIs (tllm.Builder, net_guard, graph_rewriting, LLaMAForCausalLM) — running it (or the documented full unit suite) fails at tllm.Builder(). Unscheduled in every test list; legacy TRT graph-building leak coverage with no PyTorch equivalent → deleted.

Fixed (kept tests that referenced deleted files — the draft missed all of these)

File	What changed	Why
unit_tests/{layers/baseSamplingLayerTest.h, layers/eagleLayerTest.cpp, layers/explicitDraftTokensLayerTest.cpp, kernels/sampling/samplingTest.h, kernels/routing/routingTest.h, kernels/eaglePackDataTest.cpp, runtime/samplingTest.cpp}	Removed the tllmLogger.h include, the mLogger = std::make_shared<TllmLogger>() line, and the member declaration.	The member was constructed but never consumed in all 7 files — vestigial; no replacement logger needed.
unit_tests/runtime/gptDecoderBatchedTest.cpp	The deleted MakeDecodingBatchInputOutput::createDecoderBatchInputs static (engine-free, ~50 lines) is inlined as a file-local helper; 4 call sites repointed.	Keeps real coverage of the surviving GptDecoderBatched (live in the PyTorch decoder path) without keeping a dead prod algorithm.
unit_tests/kernels/ropeTest.cu	plugins/gptAttentionCommon/gptAttentionCommon.h include → kernels/gptKernels.h.	It only needed the AttentionMaskType/PositionEmbeddingType enums, whose real home is gptKernels.h.
unit_tests/multi_gpu/mpiUtilsTest.cpp	Dropped the plugins/common/plugin.h include + the GlobalSessionHandle test (plugins::getCommSessionHandle() == &COMM_SESSION).	The only plugin-coupled assertion in an otherwise-kept shared-core test.
cpp/tests/CMakeLists.txt	TensorRT::OnnxParser dropped from the per-test link; add_subdirectory(utils) + the tests/utils include dir removed; nvinfer_plugin_tensorrt_llm dropped from the add_gtest link line; include_directories(${MPI_C_INCLUDE_DIRS}) added under ENABLE_MULTI_DEVICE (2026-07-13 post-rebase amends — tests previously got both the plugin link target and the MPI include dirs transitively via the plugin's PUBLIC includes; without them every gtest failed to link and MPI-touching tests failed on mpi.h).	No TRT link; testingUtils + plugins deleted; MPI includes made explicit.
unit_tests/batch_manager/cudaGraphExecutorCacheTest.cpp + its CMake registration	Deleted (2026-07-13 amend).	Its entire subject — CudaGraphExecutor/CudaGraphExecutorCache — was removed from inflightBatchingUtils in this PR (engine-path CUDA-graph runner; the PyTorch backend does CUDA graphs in Python).
unit_tests/kernels/ropeTest.cu (2026-07-13 amend, additional)	common/cudaUtils.h include + using namespace tensorrt_llm::common; added.	The removed plugin header transitively provided QuantMode/getSMVersion; first actual gtest compile surfaced it.
unit_tests/runtime/gptDecoderBatchedTest.cpp (2026-07-13 amend, additional)	batch_manager/llmRequest.h include added.	The inlined createDecoderBatchInputs helper dereferences LlmRequest, previously a complete type only via the deleted algorithm's header chain.
unit_tests/utils/ (utilsTest.cpp + CMakeLists + parent registration)	Deleted (2026-07-13 amend).	Tests the randomLogits helper from the deleted cpp/tests/utils/ (testingUtils) library — its subject is gone.
cpp/tensorrt_llm/thop/CMakeLists.txt (2026-07-13 amend)	th_utils now links ${SHARED_TARGET} PUBLIC.	thUtils.cpp genuinely references libtensorrt_llm symbols (ITensor::wrap, TllmException, fmtstr), but the dependency was never declared — thUtilsTest linked only because the plugin target's presence on the link line satisfied it. With the plugin gone, ld (--as-needed positional semantics) dropped the .so listed before libth_utils.a and the test failed to link. Declaring the real dependency makes CMake order the .so after the archive.

| cpp/tests/unit_tests/multi_gpu/kernels/allReduce/gemmAllReduceTest.cu (2026-07-15 amend) | Dropped the include of the deleted cpp/tests/utils/common.h; its sole consumed symbol (tensorrt_llm::testing::GpuTimer, a ~50-line CUDA-event timer) inlined into the test file. | Phase C regression, unmasked: on main this compiled only because every gtest linked nvinfer_plugin_tensorrt_llm, whose PUBLIC USING_OSS_CUTLASS_*_GEMM defines leaked into all tests. This PR removes that link, so the include went live and broke the shared fixture build. (An earlier triage note calling this "pre-existing / internal-CI-mode-only" was WRONG — there is no internal CI mode; CI builds gtests in OSS mode.) |
| cpp/tests/unit_tests/kernels/CMakeLists.txt (2026-07-15 amend) | mixtureOfExpertsInternalTest target creation gated on if(INTERNAL_CUTLASS_KERNELS_PATH). | Same unmasking: with the plugin's leaked PUBLIC define gone, remove_compile_definition finally took effect and the internal-path branch (#include "quantization.h", header not shipped in the OSS cutlass tarball) went live. On main the "internal" test had silently compiled the OSS path — a duplicate of mixtureOfExpertsTest, never real internal coverage — so gating it out in OSS builds loses nothing; internal-source builds now get a genuine internal-path test for the first time. |
| cpp/tests/{e2e_tests,e2e_tests/batch_manager,unit_tests/executor,unit_tests/runtime,unit_tests/batch_manager}/CMakeLists.txt | Registrations of every deleted test pruned (guidedDecoderTest stays — see below). | Match the deletions. |
| cpp/tests/e2e_tests/batch_manager/guidedDecoderTest.cpp | Migrated (respell + tllmDataType.h) and kept. | It lives under e2e_tests/ but tests the surviving GuidedDecoder with no engine coupling. Flagged for relocation out of e2e_tests/ in a follow-up. |

H. Python test suite + CI metadata

File(s)	What changed	Why
tests/integration/defs/cpp/test_e2e.py (DELETED)	Whole file: builds TRT engines and drives the deleted e2e_tests binaries (test_model, test_benchmarks).	Owns-the-breakage rule.
tests/integration/defs/cpp/test_multi_gpu.py	Pruned to the 5 shared-core gtest wrappers (test_mpi_utils, test_fused_gemm_allreduce, test_cache_transceiver, test_user_buffer, test_nccl_utils) — exactly the shared-core KEEP list. Deleted: test_llama_executor{,_logits_proc,_guided_decoding}, test_enc_dec, test_trt_gpt_real_decoder, TestDisagg (4 methods), their run_* helpers, and the engine-model fixtures (prepare_multi_gpu_model_tests, prepare_model_multi_gpu, gpt/llama_single_gpu_model, llama_multi_gpu_model, prepare_models_disagg, multi_gpu_model).	Per-function edit, not a file deletion (the file MIXES engine wrappers with shared-core wrappers).
tests/integration/defs/cpp/cpp_common.py	prepare_model_tests (engine builder driving the deleted build_*_engines.py) removed.	0 callers after the fixture pruning.
tests/integration/defs/cpp/conftest.py	prepare_model + build_benchmarks fixtures removed. Kept: lora_setup (generates numpy LoRA weights — no engines — for the surviving C++ LoRA unit tests).	Their targets/scripts are deleted.
tests/unittest/bindings/test_executor_bindings.py (DELETED)	All-engine trtllm.Executor tests (was already waived since TRTLLM-13781).	The binding is gone; closes the waive.
tests/unittest/bindings/binding_test_utils.py (DELETED)	Engine-run helpers; only consumer was test_executor_bindings.py.	Orphaned.
tests/unittest/utils/cpp_paths.py	Trimmed to the sole surviving fixture llm_root (consumed by tests/unittest/conftest.py + one auto_deploy test). Deleted: the ModelSpec import (module-level — broke collection of the whole bindings suite), get_base_model_spec, and the engine-path/results-path fixtures (0 users each, verified fixture-by-fixture).	The late straggler found by the parity-test run; the other apparent "users" (test_executor.py, test_base_worker.py) define their own local names.
tests/integration/defs/test_mlpf_results.py (DELETED)	Runs gptManagerBenchmark on a built engine; unscheduled in any test list.	Binary deleted.
tests/integration/test_lists/test-db/l0_{a10,a30,dgx_h100,h100}.yml	10 entries delisted: the cpp/test_e2e.py model/benchmark entries + the TestDisagg symmetric-executor entries.	Tests deleted.
tests/integration/test_lists/waives.txt	11 waives removed (cpp/test_e2e.py::*, TestDisagg::*, the test_executor_bindings.py waive).	Waived tests no longer exist (duplicate/AST validators gate this).
tests/integration/defs/.test_durations	35 stale duration keys removed.	Same.
benchmarks/cpp/README.md	Rewritten: documents the kept dataset tools (prepare_dataset.py, utils/*) and points benchmarking at trtllm-bench/trtllm-serve; all gptManagerBenchmark content removed.	The old doc was entirely about the deleted binaries.

First-CI-run fixes (2026-07-15 amends)

File(s)	What changed	Why
jenkins/L0_Test.groovy	Deleted the A10-CPP-1 and H100_PCIe-CPP-Post-Merge-1 stage definitions.	Our delisting emptied both stages; an empty stage hard-fails. Deleted rather than commented — the groovy stage parser does NOT skip // comments.
tests/integration/test_lists/test-db/l0_b200.yml	Removed unittest/disaggregated/test_router.py from the tensorrt post_merge chunk.	Dual-backend scheduling violation surfaced by the mapping test's shifted sampling window; coverage retained via the A10/H100 pre-merge pytorch entries.
tests/unittest/tools/test_test_to_stage_mapping.py	test_cli_functionality samples only stage-mapped tests (skips if none).	The CLI check sampled [:MAX_SAMPLES] of a glob-ordered list; our delisting shifted the window onto PRE-EXISTING unmapped test-db entries (violation sets byte-identical branch vs main).
tests/unittest/auto_deploy/singlegpu/smoke/test_ad_trtllm_bench.py, tests/unittest/auto_deploy/multigpu/custom_ops/test_ad_dist_strategies.py	script_dir = Path(root_dir, "benchmarks", "cpp") → Path(root_dir, "benchmarks") (+ dataset_tool path).	The benchmarks/cpp → benchmarks/ relocation (§ G) missed these cwd references; 6 AutoDeploy bench CI failures.
examples/auto_deploy/paragraf/create_standalone_package.py + tests/unittest/auto_deploy/standalone/test_standalone_test_export.py	The packager's replace-literal + the round-trip test updated to the new Path(root_dir, "benchmarks") string.	Same sweep. (Originally patched in llmc/create_standalone_package.py; main #16324 renamed the packager to paragraf/ — the one conflict at the 95f9f92a76 rebase, resolved by taking main's forwarding shim and re-applying the fix at the new location.)
tests/integration/defs/common.py	Deleted dead get_cpp_benchmark() (+ autoflake dropped the orphaned is_windows import).	Located deleted C++ benchmark binaries; zero callers.
tests/integration/defs/perf/test_perf.py, docs/source/commands/trtllm-bench.rst	prepare_dataset.py path/link → benchmarks/.	Same relocation sweep.

Review-comment fixes (2026-07-16 amends)

File(s)	What changed	Why
examples/disaggregated/slurm/benchmark/disaggr_torch.slurm, …/disaggr_torch_dwdp.slurm	Dropped --benchmarks from the build_wheel=true source-build command.	Review (chienchunhung): the flag is removed from build_wheel.py by this PR — argparse would reject it and fail the source build. --trt_root kept (still an accepted arg).
tests/integration/README.md	Removed the "build the cpp benchmark with --benchmarks" prerequisite sentence.	Same sweep; the C++ benchmarks no longer exist — perf tests use trtllm-bench.
.claude/skills/exec-local-compile/SKILL.md, .claude/skills/exec-slurm-compile/{SKILL.md,scripts/compile.sh}	Dropped --benchmarks from checked-in compile commands and flag tables.	Same sweep — these are tracked repo files whose build commands would now fail.
benchmarks/prepare_dataset.py	Two stale docstrings ("for gptManagerBenchmark") → benchmark-neutral wording.	The named binary is deleted by this PR; docs/source/legacy/ + release-notes.md mentions left as historical record.
cpp/include/tensorrt_llm/common/tllmDataType.h	Copyright 2022-2026 → 2026; ILogger/ILoggerSeverity shim deleted.	Review (Wanli-Jiang: new file gets current year; BowenFu: shim is dead — zero consumers, its only would-be implementer TllmLogger is deleted here).
scripts/build_wheel.py	--trt_root help text → "[DEPRECATED] No effect: TensorRT is no longer required to build. Accepted for backward compatibility and will be removed in a future release."	The flag is now a no-op (-DTensorRT_ROOT has zero CMake consumers after FindTensorRT.cmake's deletion); kept accepting it so existing checked-in build commands don't break — actual removal + caller sweep is a follow-up.
cpp/include/tensorrt_llm/executor/executor.h (OrchestratorConfig), cpp/tensorrt_llm/nanobind/executor/executorConfig.cpp	@deprecated doc comment on the class + comment at the nanobind binding site.	Orchestrator mode is non-functional after this PR (its executorWorker binary is deleted; sole remaining user is a dormant example). Class/binding retained for serialization + Python-binding compatibility; removal deferred to the dormant-consumer follow-up pending API-stability review.
legacy-files.txt, .pre-commit-config.yaml, pyproject.toml, ruff-legacy.toml	Removed the 29 entries naming files this PR deletes (engine-test resource scripts, build_cpp_examples.py, deleted test wrappers); configs regenerated via scripts/legacy_utils.py gen-configs, check-configs green.	Generalized sweep of chienchunhung's "callers of removed things" principle: the legacy-lint lists still named deleted paths. Only OUR 29 entries removed — the ~450 entries stale from earlier phases are main's pre-existing state (follow-up: legacy_utils.py prune-files). ruff-legacy-baseline.json had zero entries for our files.
cpp/include/tensorrt_llm/executor/executor.h	Deleted the class Executor declaration (all 4 engine-path ctors, the shared_ptr<Model> ctor, ~30 methods) and the now-orphaned class Model; fwd-decl.	Review (chienchunhung): executor.cpp's deletion removed every implementation, so the packaged header declared an API that compiles but cannot link. Zero users remain in the retained tree (grep: only comments/strings). Everything else in the header keeps its implementation — version()/KVCacheEvent ctor (now inline in the header, see the VALLIS-NERIA row below), KVCacheEventManager (executorKVCacheEventManager.cpp, bound + used by PyExecutor), ModelType (serialized, disagg), JsonSerialization, all config/request/response classes. Full google-tests build_wheel_targets rebuild green (846 targets).
cpp/include/tensorrt_llm/common/tllmDataType.h	DataType/Dims moved into namespace common with using common::DataType; using common::Dims; hoists into tensorrt_llm.	Review (VALLIS-NERIA): match the other common/*.h headers' namespace convention. The hoist keeps every existing tensorrt_llm::X spelling valid — zero call-site changes.
cpp/include/tensorrt_llm/executor/executor.h + cpp/tensorrt_llm/executor/{kvCacheEvent.cpp,CMakeLists.txt}	version() made inline (header now includes the public generated version.h) and the KVCacheEvent ctor defined in-class; the interim relocation file kvCacheEvent.cpp deleted and pruned from SRCS.	Review (VALLIS-NERIA): both bodies are trivial — no reason for a dedicated translation unit. One fewer new file in the PR; bindings.executor.__version__ unaffected.
examples/bindings/** (5 D), tests/integration/defs/deterministic/** (3 D), defs/common.py, defs/conftest.py, legacy-lint lists (legacy-files.txt, 3 generated configs, ruff-legacy-baseline.json)	Deleted the dormant Executor-binding clients: the 4 binding examples (+ README; the dir held nothing else), the unscheduled deterministic Mixtral test + driver + payload, the orphaned generate_deterministic_cmd helper and deterministic_test_root fixture; lint lists pruned.	Review: verified these fail with AttributeError: … no attribute 'Executor' (binding deleted by this PR). Nothing to migrate to — they demo the removed low-level Executor API; the LLM API is the replacement. Zero test-db/waives/durations/CODEOWNERS entries existed.

I. Review-driven scope change (chronological)

1. First cut (same day, superseded): tllmDataType.h was authored in a transitional
   alias form (using DataType = nvinfer1::DataType; …) so the 206-file respell could
   land as a standalone, provably-behavior-free PR while the plugin/engine path kept
   compiling, with the real-enum flip + engine deletion deferred to the next PR
   (the original two-step split of the removal plan).
2. Review direction: "remove nvinfer fully, not import nvinfer anymore" — the alias
   staging was dropped; the header was flipped to the real standalone types and the
   engine-path removal + packaging gate were folded into this PR (§ A–H above). The
   alias-form design record is kept in the internal design notes.
3. Stragglers beyond the draft (its base was ~3 weeks stale and its cpp/tests,
   benchmarks, examples/cpp, and Jenkins handling were incomplete): the 12
   deleted-header include stragglers (§ G "Fixed"), guidedDecoderTest classification,
   executor/model.h, ModelSpec/testing_src, benchmarks/cpp, examples/cpp,
   Build.groovy, cpp_paths.py — all found by systematic sweeps
   (deleted-header include scan, python deleted-symbol scan, fixture-consumer counts).
4. First-CI-run + OSS-mode fixes (2026-07-15): the § H "First-CI-run fixes" block,
   the kvCacheManagerTest rebase-absorption respell (§ A), and the two OSS-mode gtest
   regression fixes (§ G) — the latter overturned an earlier "pre-existing" misdiagnosis
   and produced the mandatory post-rebase scan suite: cpp-wide nvinfer grep incl.
   cpp/tests/, plugin-link grep,
   USING_OSS_CUTLASS/remove_compile_definition audit, full OSS-mode
   google-tests build — all four must be green before any force-push.
5. Rebase onto 95f9f92a76 (2026-07-15): one conflict (paragraf rename, § H) and
   one scan-suite catch (rmsNormFp4QuantKernels, § A) — both folded in.
6. Re-rebase onto a914fe4cd8 (2026-07-16): no conflicts; main's 24 new commits add
   zero TRT-surface references (checked the whole range diff). The rebase silently
   resurrected the removed ILogger shim (started from a pre-amend state) — caught by
   interdiffing the old and new heads against main's own range diff, re-applied.
7. Review-comment round (2026-07-16, § H last table): copyright year; --benchmarks
   caller sweep + its generalization (the removal-consistency sweep: declared-but-
   unimplemented API check via nm over the built libs, callers-of-removed-things
   checks → one further real find, the legacy-lint prune); dead class Executor
   declaration removed; --trt_root marked deprecated; OrchestratorConfig deprecation
   comments; DataType/Dims moved into tensorrt_llm::common (+using hoist);
   kvCacheEvent.cpp inlined away; dormant Executor-binding clients
   (examples/bindings/, defs/deterministic/) verified-broken and deleted.

J. Validation record

Current state (2026-07-16 10:45 UTC, 845ab6559e on a914fe4cd8):

• Mandatory post-rebase scan suite — checks 1–3 green at HEAD; check 4 pending for
  the last two header amends: (1) cpp-wide nvinfer grep incl. cpp/tests/ — 0 hits;
  (2) nvinfer_plugin CMake grep — 0 hits; (3) OSS-cutlass define audit — both test
  users self-define, internal target gated. (4) full OSS-mode
  ninja google-tests build_wheel_targets (sm90): 835 targets, zero failures at
  the rebased base, 846 targets, zero failures after the class Executor
  removal; the two final header amends (tensorrt_llm::common namespace move +
  kvCacheEvent.cpp inlining) and the python-only dormant-client deletion still
  need one rebuild pass — delegated to williamj, must be green before force-push.
  (Previous run at 393086c7f7/95f9f92a76: also 4/4, incl. the
  rmsNormFp4QuantKernels respell check 1 caught.)
• H100 execution run (4x H100 PCIe, 2026-07-15) —
  the ten CI tests' exact run phases against the CI-fixture-equivalent build:
  test_unit_tests[common-90] 28/28 PASS; test_unit_tests[kernels-90] 1138/1139
  (single failure MixtureOfExpertsTest.PermuteSweepNumTokens<fp8×int4> — the known
  MoE-permute GPU-contention flake family, isolation retest pending); layers/thop +
  the 6 multi-GPU tests in progress at time of writing. The CI failure mode itself
  (fixture build failure) is closed by scan-suite check 4.
• Earlier gates (2026-07-14, carried over — cpp/ respell content unchanged by the
  rebase): zero libnvinfer NEEDED in libtensorrt_llm.so + bindings + nixl/ucx
  wrappers; datatype parity test green; TinyLlama quickstart e2e coherent.

Superseded note: the 2026-07-14 record called mixtureOfExpertsInternalTest/
gemmAllReduceTest "pre-existing OSS-mode gaps" — WRONG, both were Phase C
regressions via the plugin PUBLIC-define leak; fixed in this PR (§ G).

Original validation record (2026-07-10, 53d64f3e74):

• python3 scripts/build_wheel.py --use_ccache --skip_building_wheel --cuda_architectures native — no --trt_root — exit 0.
  (Gotcha: BUILD_WHEEL_TARGETS is a CMake cache variable; the first incremental run
  kept the stale target list and failed at the final aggregate target — resolved with
  --configure_cmake.)
• readelf -d on libtensorrt_llm.so, libth_common.so, bindings.*.so:
  zero nvinfer/TensorRT NEEDED entries.
• import tensorrt_llm + tensorrt_llm.bindings with the tensorrt package
  hard-blocked via a sys.meta_path blocker: OK.
• tests/unittest/bindings/test_datatype_parity.py: 2 passed.
• pytest --co -q tests/unittest: exit 0 (collection clean).
• Quickstart e2e (H100, cuda graphs): coherent generation. (The MoE FP8-block-scale
  assert seen with one local checkpoint is a pre-existing Python weight-loader
  incompatibility with that checkpoint — path untouched by this PR.)
• pre-commit run on all 512 files: green (autoflake/clang-format/cmake-format
  reformats absorbed into the commit).

K. Follow-ups (deliberately NOT in this PR)

Consolidated 2026-07-14 by cross-checking the Phase B review threads. Summary of
what stays out:

• Test-list sweep (test-list audit Buckets 2+4) — ~60 dead
  backend: tensorrt test-db entries, inert TRT waives, the remaining reviewer-verified
  dormant deleted-module consumers under examples/ + integration helpers
  (chienchunhung thread — deferral accepted on the merged Phase B PR; the
  examples/bindings/ + defs/deterministic/ subset was pulled INTO this PR on
  review, § H), relocation of backend-neutral tests out of -TensorRT- labelled
  stages, and -TensorRT- stage retirement in L0_Test.groovy. Needs CI-owner +
  examples-owner ack.
• Python misc-cleanup pass (QiJune review deferrals, accepted as "step 4:
  trailing cleanup"): remove functional.py (relocate retained enums /
  AllReduceParams, ~55 _torch/ repoints), remove quantize_by_modelopt.py as a
  coordinated unit (NB: nvidia-modelopt STAYS in requirements — the PyTorch
  backend consumes modelopt checkpoints; only the legacy export path is dead),
  load_calib_dataset orphan (closes nvbugs/6215107), bench/build/ dissolve,
  tools/plugin_gen/, unet/pp/__init__.py straggler.
• QA coverage ports (tburt-nv threads; landed already:
  test_gather_generation_logits_cuda_graph → llm_function_core.txt):
  TestQwen2_7BInstruct::test_tp2 pytorch port (trivial), aggregated Ulysses cp2
  accuracy test (validate with CP owners first), test_fp8_rowwise port (blocked on
  provisioning a rowwise-quantized checkpoint), W8A16 = feature gap (no port).
• Post-merge validation commitment (brb-nv LoRA thread): include post-merge
  stages in this PR's /bot run (--extra-stage) + monitor post-merge boards after
  landing. LoRA surface here is respell-only + engine-buffer deletion; the three C++
  LoRA gtests are kept.
• C++ dead-code audit items (post-Phase-C audit): verified orphans
  (orchestratorUtils.h, promptTuningParams, requestWithId tangle), the
  dead-in-practice C++ spec-dec decoder stack (fold into the 1.4 TRTLLMSampler
  removal), stale gptJsonConfig.cpp:273 comment; transferAgent.cpp
  self-sufficient wrapper dlopen (replaces _common.py's RTLD_GLOBAL bridge, which
  is kept — it concerns our own lib's symbols, not TRT).
• Docker devel-image TRT strip (--tensorrt install line +
  install_tensorrt.sh) — the one follow-up requiring an image rebuild +
  jenkins/current_image_tags.properties bump (deliberately untouched here).
• prepare_dataset.py dedup — benchmarks/ standalone copy vs the drifted
  packaged twin tensorrt_llm/bench/dataset/ (trtllm-bench prepare-dataset).
• Relocate guidedDecoderTest.cpp out of cpp/tests/e2e_tests/ (unit test of a
  surviving component; kept in place to avoid churn).
• Perf-test config keys with runtime: "cpp" (bertBenchmark-era label) +
  test_perf.py's TRT branch + defs/perf/build.py: dead config surface, part of
  the test-list sweep above.

L. PR-split assessment (2026-07-15, updated 12:00 UTC)

Verdict: keep the single PR. Below: what the ~576 files actually are, then why
each bucket is chained to the next, then the splits that were considered.

L.1 What the files are (576: 2 A / 302 M / 264 D / 8 R)

#	Bucket	Files	Concretely	Nature
1	New internal types + guard	2 A	common/tllmDataType.h, test_datatype_parity.py	The only genuinely new code (~200 lines; the KVCacheEvent ctor + version() relocation is inline in executor.h per review)
2	Mechanical respell	~206 M	nvinfer1::{DataType,Dims,ILogger} → tensorrt_llm:: across kernels 43, runtime 28, batch_manager 24, cpp/include 23, thop 15, layers 9, nanobind 8, executor 6, common 6, tests 73, micro_benchmarks 1	Find-replace + include swaps; guarded by the parity test
3	Engine execution path	~185 D (of 264 D total)	plugins/** 131, TRT model adapters + engine buffers/algos in batch_manager/ 20, engine runtime wrappers in runtime/ 7, their public headers in cpp/include/ 12, executor{,_worker}/ 7, nanobind Executor/ModelSpec bindings 4, testing/ 3, FindTensorRT.cmake 1	Pure deletion
4	Engine-driven tests + their machinery	~40 D	cpp/tests/e2e_tests/** 10, unit_tests engine tests 7, cpp/tests/utils/ (in-memory engine builder) 7, cpp/tests/resources/scripts/build_*_engines.py etc. 23→(22 D + 1 kept)	Pure deletion
5	Engine-only C++ benchmarks + examples	~18 D, 8 R	benchmarks/cpp/{bert,gptManager,disaggServer}Benchmark.cpp + CMake 7, examples/cpp/** 11; the 8 renames = surviving python dataset tools benchmarks/cpp/ → benchmarks/	Deletion + moves
6	Shared-file decouplings	~15 M	inflightBatchingUtils, lookaheadBuffers, medusaBuffers, nanobind pruning, opUtils.h/attentionOp IS_BUILDING, python base_worker/worker/__init__/_common	Surgical edits — the highest-review-value bucket (§ C–E)
7	Kept tests fixed	~20 M	Vestigial TllmLogger members ×7, gptDecoderBatchedTest helper inline, ropeTest/mpiUtilsTest include repoints, cpp/tests/CMakeLists link/MPI fixes, GpuTimer inline + MoE-internal gate (§ G "Fixed")	Surgical edits
8	Build & packaging	~15 M	cpp/CMakeLists, per-dir CMake prunes, build_wheel.py, setup.py, requirements.txt, Build.groovy, Dockerfile.multi, slurm_install.sh, get_wheel_from_package.py	Must match #3/#5 exactly
9	Python wrappers + CI metadata	~33 M/D	defs/cpp/test_e2e.py D, test_multi_gpu.py pruned, test-db ymls, waives.txt, .test_durations, L0_Test.groovy stages, mapping-test fix; + the review-driven dormant-client deletions (examples/bindings/ 5 D, defs/deterministic/ 3 D, 2 helper edits)	Must match #4 exactly
10	Docs + lint configs	~12 M	perf docs, blog build cmd, trtllm-bench.rst, ruff-legacy configs/baseline (rename fallout)	Trailing edits

L.2 The dependency chain — why the buckets can't land separately

Each arrow is a hard breakage on main if the two sides are in different PRs:

1. Bump onnx from 1.12.0 to 1.13.0 #1 → Add static libraries for batch manager #2 (types force the respell). tllmDataType.h declares real
   enum class types. A file saying nvinfer1::DataType cannot receive a
   tensorrt_llm::DataType — every one of the ~2,000 uses in the 206 surviving files
   must flip in the same commit or those files stop compiling.
2. Add static libraries for batch manager #2 → Update TRT-LLM code #3 (the respell forces the engine deletion). The engine path
   (plugins/, trtGptModel*, tllmRuntime, C++ Executor) must keep the real
   nvinfer1 types to talk to TensorRT, but it consumes the shared core headers that
   are now respelled (iTensor.h, bufferManager.h, modelConfig.h, …). enum class
   doesn't interconvert → the engine path cannot compile against the migrated core.
   Splitting = a PR that leaves main unbuildable. (The only escape is the alias
   staging — designed as a transitional alias form, rejected in review: "remove nvinfer
   fully, not import nvinfer anymore", § I.2.)
3. Update TRT-LLM code #3 → update aarch64 libraries to main branch #6 (deletions force the decouplings). Surviving shared files reference
   deleted engine types: inflightBatchingUtils held CudaGraphExecutor (drives the
   deleted TllmRuntime), medusaBuffers/lookaheadBuffers have engine ctors taking
   TllmRuntime const&, nanobind registers the deleted Executor class,
   base_worker.py constructs tllm.Executor. Without the § C–E edits in the same
   commit, the library and the Python import both break.
4. Update TRT-LLM code #3 → Fix memory leak in falcon weight loader #8 (deletions force the packaging edits). cpp/CMakeLists.txt does
   find_package(TensorRT REQUIRED) and add_subdirectory(plugins); setup.py
   packages libnvinfer_plugin_*.so/bin/executorWorker; Build.groovy/Dockerfile
   copy the benchmark binaries. Any of these referencing a deleted target fails every
   build — same commit or nothing ships. requirements.txt dropping tensorrt is
   the point of the PR and is only correct once nothing links it.
5. Update TRT-LLM code #3/Kaiyu/update main #5 → Update TensorRT-LLM #4 (deleted binaries force deleting their tests). The e2e/executor
   gtests link the deleted Executor/TllmRuntime; cpp/tests/utils/engines builds
   TRT engines in-memory. Keeping the tests = keeping the code. (Ownership rule from
   Phase B: the PR that deletes a binary owns every test that drives it.)
6. Update TensorRT-LLM #4 → update aarch64 batch manager libraries to main #9 (deleted tests force the CI metadata). test-db entries for deleted
   tests crash the trt-test-db lint (AST validator + tests: None gotcha); stale
   waives fail the duplicate/AST validators; the two stages our delisting emptied
   (A10-CPP-1, H100_PCIe-CPP-Post-Merge-1) hard-fail as empty stages. CI turns red
   in the same pipeline run that tests the PR — these cannot trail.
7. update aarch64 libraries to release/0.5.0 branch #7 rides along by construction — each kept-test fix exists only because a
   dependency deleted in Update TRT-LLM code #3 (plugin PUBLIC defines/includes, testingUtils,
   tllmLogger.h) stopped papering over a latent gap. On pre-PR main these edits are
   no-ops at best, unbuildable at worst (e.g. the GpuTimer inline collides with the
   still-existing tests/utils/common.h definition; the MoE-internal gate would
   disable a then-working target).

L.3 Splits considered

Candidate split	Files saved	Why not / status
Alias-form respell precursor (#1+#2 alone, using DataType = nvinfer1::DataType)	~210	The one split that breaks chain-link 2. Designed (§ I.1), rejected by review direction — keeps NvInfer.h imports alive for another PR cycle.
benchmarks/ relocation precursor (#5's renames + reference sweep)	~25	Clean and independent — viable if reviewers ask. Pure churn reduction; touches none of the risky core.
Engine-test retirement precursor (#4 + #9)	~65	Viable with precedent (Phase A #15810 removed tests ahead of code). Cost: a main window where green engine tests are deleted for a not-yet-landed reason, plus a second CI-metadata review.
Everything else (#1 #2 #3 #6 #7 #8)	—	One atomic unit (~470 files) per chain-links 1–4 + 7.

Even taking both viable precursors, the core PR only shrinks ~576 → ~480 while adding
two more rebase-churn windows — and external rebases have already broken this branch
five times (#16304 + #14848 nvinfer stragglers, lost test-db fixes, the paragraf
conflict, the lost-ILogger-amend rebase), which is what the mandatory post-rebase scan
suite + interdiff check now guard. Review effort is better spent on the ~2,700 added
lines mapped by §§ A–H than on re-reviewing three PRs' worth of CI metadata.

M. Review guide — the files that actually need eyes (2026-07-15 12:20 UTC)

Of 576 files, ~35 carry review risk. The rest are mechanical respells (guarded by
test_datatype_parity.py + the full OSS google-tests build), pure deletions (review
the list, § B/G — not the content), or validator-gated metadata (test-db/waives/
durations lint). Suggested order below; "What to verify" is the actual review task.

M.1 New code — highest value per line (2 added files + 1 inlined relocation, ~250 lines)

File	What to verify	Risk if wrong
cpp/include/tensorrt_llm/common/tllmDataType.h (A)	Enumerator values 0–11 EXACTLY match nvinfer1::DataType (kFLOAT=0 … kE8M0=11); Dims layout = TRT-10 Dims64 (int32_t nbDims + int64_t d[8], MAX_DIMS=8); header stays dependency-free (<cstdint> + config.h only); types live in tensorrt_llm::common with using hoists into tensorrt_llm (both spellings name the same type).	Serialization corruption: executor configs / KV-cache metadata / disagg wire format written by old builds decode with shifted dtypes — silent data corruption, not a crash.
cpp/include/tensorrt_llm/executor/executor.h (inlined KVCacheEvent ctor + version())	Bodies equivalent to those deleted from executorImpl.cpp (relocation, not rewrite; inlined per review — VALLIS-NERIA).	KV-cache event stream (used by cache-aware routing) changes semantics.
tests/unittest/bindings/test_datatype_parity.py (A)	Covers the FULL member set with hard-coded integers (not read back from the enum — that would be a tautology).	The guard above guards nothing.

M.2 Shared production code, surgically edited (~12 files)

File	What to verify	Risk if wrong
batch_manager/utils/inflightBatchingUtils.{h,cpp}	ONLY copyGenerationLogits/findOutputTensor/copyAdditionalOutputs/CudaGraphExecutor{,Cache} removed; the PyTorch-used helpers (collectRequestIds, terminate/pause helpers…) byte-untouched.	PyTorch in-flight batching behavior change.
batch_manager/medusaBuffers.{h,cpp}, runtime/lookaheadBuffers.{h,cpp}	Only the engine ctor / LookaheadRuntimeBuffers half removed; the decoder-side halves (used by the shared decoder) intact; lookaheadDecodingUtils.h→cudaUtils.h include swap still provides divUp.	Medusa / lookahead speculative decoding breaks.
common/opUtils.h + common/attentionOp.cpp	Removing the if (isBuilding()) return 0; gate is safe because no writer of IS_BUILDING remains (Python _BuildingFlag deleted in the same commit) — i.e. the gate was already always-false at runtime.	attentionOp::initialize does NCCL setup in a context that previously skipped it.
nanobind/executor/bindings.cpp	Only the Executor class registration dropped; ExecutorConfig/request/response/KV-event types (live PyTorch API surface) all still registered.	import tensorrt_llm breaks or LLM-API types vanish.
nanobind/runtime/bindings.cpp, nanobind/batch_manager/algorithms.cpp, nanobind/bindings.cpp	The 0-Python-users claims (TllmRuntime, LogitsPostProcessor, ModelSpec) — grep the python tree yourself; initKvCacheTestUtilBindings kept.	Runtime AttributeError in whatever still referenced them.
tensorrt_llm/executor/base_worker.py	The collapsed isinstance(self.engine, tllm.Executor) branches keep PyExecutor semantics identical; the new assert self.llm_args is not None is genuinely unreachable-safe (LLM API always passes llm_args since Phase B).	Worker startup failure for every serve/LLM-API path.
tensorrt_llm/__init__.py	Dropping _preload_tensorrt_libs() is safe ⇔ no binding carries DT_NEEDED libnvinfer — cross-check the readelf validation in § J.	import tensorrt_llm ImportError in wheel-only envs.
cpp/include/tensorrt_llm/runtime/iTensor.h	Dropping the nvinfer1::IExecutionContext fwd-decl breaks no remaining user (its users are deleted).	Compile error — caught by build, zero silent risk.

M.3 Packaging & CI wiring — small diffs, big blast radius (~10 files)

File	What to verify	Risk if wrong
requirements.txt	The tensorrt~=10.16.1 drop — THE point of the PR. Confirm nothing at import time still needs it (§ J: import with tensorrt hard-blocked).	Every user install changes; a hidden dep breaks wheel-only envs.
scripts/build_wheel.py	--trt_root kept as a [DEPRECATED] no-op (old default retained) so existing build commands don't break; deleted targets gone from BUILD_WHEEL_TARGETS (CMake CACHE var — stale caches need --configure_cmake).	Every developer build.
jenkins/Build.groovy + docker/Dockerfile.multi	ALL benchmark/plugin copy/check lines gone (x86 Release AND Debug — Debug was the CI-caught miss); replacement sanity checks (test -f lib/libtensorrt_llm.so, ldd libth_common.so) actually gate what they claim. current_image_tags.properties intentionally NOT bumped (devel-image content unchanged).	CI build stage hard-fails post-merge.
jenkins/L0_Test.groovy	Only the two emptied stage defs deleted (A10-CPP-1, H100_PCIe-CPP-Post-Merge-1); no other stage's schedule touched.	Lost CI coverage or dangling stage refs.
cpp/CMakeLists.txt + cpp/tensorrt_llm/CMakeLists.txt	find_package(TensorRT) / TRT_LIB fully gone; no surviving target references a deleted one (scan-suite check 2).	Nothing builds — loud, not silent.
setup.py	Only bin/executorWorker + plugin .so/.dll dropped from package_data.	Wheel content regression.

M.4 Judgment calls in kept tests — worth a deliberate opinion (~8 files)

File	The call that was made	Push back if you disagree
cpp/tests/unit_tests/kernels/CMakeLists.txt	mixtureOfExpertsInternalTest only exists when INTERNAL_CUTLASS_KERNELS_PATH is set — on main it silently compiled the OSS path (plugin PUBLIC-define leak defeated remove_compile_definition), so OSS builds lose zero real coverage and internal builds gain a genuine internal test.	If you believe internal-mode CI builds gtests somewhere we didn't find — then this gate hides a regression there.
unit_tests/multi_gpu/kernels/allReduce/gemmAllReduceTest.cu	~50-line GpuTimer inlined (sole consumer of deleted tests/utils/common.h) instead of keeping a shared helper lib for one user.	Prefer a new tiny shared header if more users appear.
unit_tests/runtime/gptDecoderBatchedTest.cpp	The deleted prod algorithm's engine-free static (createDecoderBatchInputs, ~50 lines) inlined as a file-local test helper to keep real GptDecoderBatched coverage.	Alternative was deleting the test — losing live decoder coverage.
cpp/tensorrt_llm/thop/CMakeLists.txt	th_utils now declares its real ${SHARED_TARGET} dep PUBLIC (was satisfied accidentally by the plugin's presence on link lines).	None — this is a latent-bug fix, but confirm no dep cycle.
cpp/tests/CMakeLists.txt	Explicit MPI_C_INCLUDE_DIRS + no plugin on add_gtest link line (both previously leaked transitively from the plugin target).	Same.
tests/integration/defs/cpp/test_multi_gpu.py	Pruned per-function to exactly the 5 shared-core wrappers — NOT a file deletion; diff the kept functions.	Check no shared-core wrapper was over-deleted.
tests/unittest/tools/test_test_to_stage_mapping.py	CLI check now samples only stage-mapped tests: the old [:MAX_SAMPLES] window was hitting PRE-EXISTING unmapped test-db warts our delisting shifted into view (violation sets byte-identical branch vs main).	If you'd rather fix the ~28 pre-existing warts — that's the deferred test-list sweep.
unit_tests/batch_manager/kvCacheManagerTest.cpp, kernels/rmsNormFp4QuantKernels.{h,cu}	Rebase absorptions: new-on-main nvinfer1 code respelled to keep the tree consistent (#16304, #14848).	Mechanical; check nothing else in those PRs assumed nvinfer1.

M.5 What does NOT need line-by-line review

• The 206 respell files (§ A): spot-check 2–3; correctness is gated by the parity
  test + the full OSS google-tests builds (1008/835/846 targets at successive heads)
  • zero-libnvinfer-NEEDED readelf.
• The 264 deletions (§ B/G/H): review the lists against the stated criteria
  ("drives deleted engine path?"), not the deleted content.
• test-db / waives / .test_durations edits: pre-commit AST + duplicate validators
  gate exactly this class of error.
• Docs/blog/README + ruff-legacy config regen: rename fallout, no behavior.

@coderabbitai summary

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[Wanli-Jiang]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #59181 [ run ] triggered by Bot. Commit: d48f1b0 Link to invocation

[tensorrt-cicd]

PR_Github #59181 [ run ] completed with state SUCCESS. Commit: d48f1b0
/LLM/main/L0_MergeRequest_PR pipeline #47680 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[tensorrt-cicd]

PR_Github #60067 [ run ] triggered by Bot. Commit: 9f0a563 Link to invocation

[yuanjingx87]

Approved on oss compliance behalf

[tensorrt-cicd]

PR_Github #60067 [ run ] completed with state SUCCESS. Commit: 9f0a563
/LLM/main/L0_MergeRequest_PR pipeline #48455 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[Wanli-Jiang]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #60108 [ run ] triggered by Bot. Commit: 693ce19 Link to invocation

[tensorrt-cicd]

PR_Github #60108 [ run ] completed with state FAILURE. Commit: 693ce19
/LLM/main/L0_MergeRequest_PR pipeline #48493 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[Wanli-Jiang]

/bot skip --comment "all pre-merge/post-merge tests were passed in different runs"

[tensorrt-cicd]

PR_Github #60140 [ skip ] triggered by Bot. Commit: e2e81d0 Link to invocation

[Wanli-Jiang]

Test report

Pre-merge single SBSA

#16369 (comment) (all passed)

Pre-merge single x86

#16369 (comment) (all passed)

Pre-merge multi SBSA

#16370 (comment) (all passed)

Pre-merge multi x86 (passed with two runs, failed due to different reasons)

#16370 (comment) (two reasons)

• AD new added tests failed

• Test / DGX_B200-8_GPUs-PyTorch-2 / test_unittests_v2[unittest/_torch/visual_gen/multi_gpu/test_wan_transformer_parallel.py] – DGX_B200-8_GPUs-PyTorch-2.test_unittests

#16370 (comment) (3 flaky errors, passed at the above tests)

• Test / DGX_B200-8_GPUs-PyTorch-4 / test_nvfp4_nixl_python[cache_mgr_v1] DGX_B200-8_GPUs-PyTorch-4.accuracy.test_disaggregated_serving.TestGLM52NVFP4

• Test / DGX_B200-16_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-FUNCTIONAL-ONLY-CTX1-NODE1-GPU4-GEN1-NODE1-GPU8-1 / test_e2e[disagg_upload-gen_only-b200_deepseek-r1-fp4_8k1k_con1536_ctx1_dep4_gen1_dep8_eplb0_mtp1_ccb-NIXL] – DGX_B200-16_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-FUNCTIONAL-ONLY-CTX1-NODE1-GPU4-GEN1-NODE1-GPU8-1.perf.test_perf_sanity

• Test / DGX_B200-8_GPUs-PyTorch-3 / test_auto_dtype[mtp_nextn=0-block_reuse=False-use_py_transceiver=True] – DGX_B200-8_GPUs-PyTorch-3.accuracy.test_disaggregated_serving.TestNemotron3Super120B

Post-merge single SBSA

#16476 (comment) (all passed except GH200 package sanity, infra issue)
#16476 (comment) （one GB300 failed, while GH200 package sanity passed)

Post-merge single x86 (two runs with different failures) => regard as pass.

#16476 (comment) (2 flaky errors)

• unittest/_torch/attention/test_attention_backends.py::test_attention_backend[gemma3_1b_mqa_hd256_swa512-ctx-bf16-HND-p32-v1]
• kv_cache/test_kv_cache_v2_scheduler.py::TestKVCacheV2LoRA::test_lora_eviction
• cpp moduel fixed and passed locally, also passed at the following CI runs

#16476 (comment) (6 failed)

• unittest/_torch/attention/test_attention_backends.py::test_attention_backend[gpt_oss_gqa_hd64_gptj_swa128-ctx-fp8-HND-p32-v1]
• accuracy/test_llm_api_autodeploy.py::TestNemotronSuperV3::test_accuracy[nvfp4-1-attn_dp_off-trtllm]
• kv_cache/test_kv_cache_v2_scheduler.py::TestKVCacheV2Llama::test_token_budget_limited
• unittest/_torch/misc/test_autotuner.py::test_cutedsl_nvfp4_heuristic_matches_full_sweep
• test_e2e.py::test_trtllm_bench_iteration_log[PyTorch-streaming-meta-llama/Llama-3.1-8B-llama-3.1-model/Meta-Llama-3.1-8B]
• accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_fp8_block_scales[mtp=disable-fp8kv=True-attention_dp=False-cuda_graph=False-overlap_scheduler=False-torch_compile=False]

Post-merge multi SBSA => Run two times, 5 of sanity perf check failed but accpetable)

#16477 (comment) (10 perf sanity errors)

Test / GB200-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-6 / test_e2e[disagg_upload-gen_only-gb200_deepseek-v32-fp4_32k4k_con1_ctx1_dep4_gen1_tep8_eplb0_mtp3_ccb-NIXL] – GB200-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-6.perf.test_perf_sanity

Test / GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-2 / test_e2e[disagg_upload-e2e-gb300_kimi-k25-thinking-fp4_1k1k_con4096_ctx1_dep4_gen1_dep8_eplb0_mtp0_ccb-NIXL] – GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-2.perf.test_perf_sanity

Test / GB300-20_GPUs-5_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE4-GPU16-Post-Merge-2 / test_e2e[disagg_upload-e2e-gb300_kimi-k25-thinking-fp4_8k1k_con4096_ctx1_dep4_gen1_dep16_eplb0_mtp0_ccb-NIXL] – GB300-20_GPUs-5_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE4-GPU16-Post-Merge-2.perf.test_perf_sanity

Test / GB300-4_GPUs-PyTorch-PerfSanity-Post-Merge-1 / test_e2e[aggr_upload-ctx_only-gb300_deepseek-r1-fp4_128k8k_con256_ctx1_pp4_gen1_dep8_eplb0_mtp1_ccb-NIXL] – GB300-4_GPUs-PyTorch-PerfSanity-Post-Merge-1.perf.test_perf_sanity

Test / GB300-20_GPUs-5_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE4-GPU16-Post-Merge-4 / test_e2e[disagg_upload-gen_only-gb300_kimi-k25-thinking-fp4_8k1k_con4096_ctx1_dep4_gen1_dep16_eplb0_mtp0_ccb-NIXL] – GB300-20_GPUs-5_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE4-GPU16-Post-Merge-4.perf.test_perf_sanity

Test / GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU2-GEN1-NODE2-GPU8-Post-Merge-2 / test_e2e[disagg_upload-e2e-gb300_glm-5-fp4_8k1k_con1024_ctx1_dep2_gen1_dep8_eplb256_mtp1_ccb-NIXL] – GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU2-GEN1-NODE2-GPU8-Post-Merge-2.perf.test_perf_sanity

Test / GB300-8_GPUs-2_Nodes-PyTorch-PerfSanity-Node2-GPU8-Post-Merge-2 / test_e2e[aggr_upload-deepseek_r1_fp4_v2_2_nodes_grace_blackwell-r1_fp4_v2_dep8_mtp1_8k1k] – GB300-8_GPUs-2_Nodes-PyTorch-PerfSanity-Node2-GPU8-Post-Merge-2.perf.test_perf_sanity

Test / GB300-36_GPUs-9_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE8-GPU32-Post-Merge-2 / test_e2e[disagg_upload-gen_only-gb300_kimi-k25-thinking-fp4_8k1k_con1024_ctx1_dep4_gen1_dep32_eplb416_mtp3_ccb-NIXL] – GB300-36_GPUs-9_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE8-GPU32-Post-Merge-2.perf.test_perf_sanity

Test / GB200-8_GPUs-2_Nodes-PyTorch-PerfSanity-Node2-GPU8-Post-Merge-2 / test_e2e[aggr_upload-ctx_only-gb200_deepseek-r1-fp4_128k8k_con1_ctx1_pp8_gen1_tep8_eplb0_mtp3_ccb-NIXL] – GB200-8_GPUs-2_Nodes-PyTorch-PerfSanity-Node2-GPU8-Post-Merge-2.perf.test_perf_sanity

Test / GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-1 / test_e2e[disagg_upload-e2e-gb300_deepseek-r1-fp4_128k8k_con256_ctx1_pp4_gen1_dep8_eplb0_mtp1_ccb-NIXL] – GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-1.perf.test_perf_sanity

#16477 (comment) (5 perf sanity errors)

Test / GB200-8_GPUs-2_Nodes-PyTorch-PerfSanity-Node2-GPU8-Post-Merge-2 / test_e2e[aggr_upload-ctx_only-gb200_deepseek-r1-fp4_128k8k_con1_ctx1_pp8_gen1_tep8_eplb0_mtp3_ccb-NIXL] – GB200-8_GPUs-2_Nodes-PyTorch-PerfSanity-Node2-GPU8-Post-Merge-2.perf.test_perf_sanity

Test / GB300-36_GPUs-9_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE8-GPU32-Post-Merge-1 / test_e2e[disagg_upload-gen_only-gb300_kimi-k25-thinking-fp4_8k1k_con1024_ctx1_dep4_gen1_dep32_eplb416_mtp3_ccb-NIXL] – GB300-36_GPUs-9_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE8-GPU32-Post-Merge-1.perf.test_perf_sanity

Test / GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-1 / test_e2e[disagg_upload-e2e-gb300_deepseek-r1-fp4_128k8k_con256_ctx1_pp4_gen1_dep8_eplb0_mtp1_ccb-NIXL] – GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-1.perf.test_perf_sanity

Test / GB300-20_GPUs-5_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE4-GPU16-Post-Merge-2 / test_e2e[disagg_upload-e2e-gb300_kimi-k25-thinking-fp4_8k1k_con4096_ctx1_dep4_gen1_dep16_eplb0_mtp0_ccb-NIXL] – GB300-20_GPUs-5_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE4-GPU16-Post-Merge-2.perf.test_perf_sanity

Test / GB300-20_GPUs-5_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE4-GPU16-Post-Merge-4 / test_e2e[disagg_upload-gen_only-gb300_kimi-k25-thinking-fp4_8k1k_con4096_ctx1_dep4_gen1_dep16_eplb0_mtp0_ccb-NIXL] – GB300-20_GPUs-5_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE4-GPU16-Post-Merge-4.perf.test_perf_sanity

Post-merge multi x86 => can regarded as pass with two runs

#16477 (comment) (2 perf sanity errors, 1 flaky error)

accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus[moe_backend=TRTLLM-mtp_nextn=0-tp4-fp8kv=False-attention_dp=False-cuda_graph=False-overlap_scheduler=False-low_precision_combine=False-torch_compile=True]

perf/test_perf_sanity.py::test_e2e[aggr_upload-llama_v3_3_70b_instruct_fp4_blackwell-llama70b_fp4_tp4_512_32]

perf/test_perf_sanity.py::test_e2e[aggr_upload-dynamo_qwen3_32b_fp8_hopper-qwen3_32b_fp8_tp2_6k1k]

#16477 (comment) （2 perf sanity errors, 3 flaky errors)

Test / DGX_B200-8_GPUs-PyTorch-PerfSanity-Post-Merge-3 / test_e2e[aggr_upload-qwen3_5_397b_fp4_blackwell-qwen3_5_397b_fp4_tep4_mtp3_8k1k] – DGX_B200-8_GPUs-PyTorch-PerfSanity-Post-Merge-3.perf.test_perf_sanity

Test / DGX_H200-4_GPUs-PyTorch-Post-Merge-1 / test_skip_softmax_attention_4gpus[target_sparsity_0.5-fp8kv=False] – DGX_H200-4_GPUs-PyTorch-Post-Merge-1.accuracy.test_llm_api_pytorch.TestQwen3_30B_A3B_Instruct_2507
1s
Test / DGX_H200-8_GPUs-PyTorch-PerfSanity-Post-Merge-1 / test_e2e[aggr_upload-dynamo_qwen3_32b_fp8_hopper-qwen3_32b_fp8_tp2_6k1k] – DGX_H200-8_GPUs-PyTorch-PerfSanity-Post-Merge-1.perf.test_perf_sanity

Test / DGX_B300-4_GPUs-PyTorch-Post-Merge-1 / test_nvfp4_4gpus[moe_backend=CUTLASS-mtp_nextn=0-tp4-fp8kv=True-attention_dp=True-cuda_graph=True-overlap_scheduler=True-low_precision_combine=False-torch_compile=True] – DGX_B300-4_GPUs-PyTorch-Post-Merge-1.accuracy.test_llm_api_pytorch.TestDeepSeekV3Lite

Test / DGX_B200-4_GPUs-PyTorch-Post-Merge-2 / test_nvfp4_4gpus[moe_backend=TRTLLM-mtp_nextn=0-tp4-fp8kv=False-attention_dp=False-cuda_graph=False-overlap_scheduler=False-low_precision_combine=False-torch_compile=True] – DGX_B200-4_GPUs-PyTorch-Post-Merge-2.accuracy.test_llm_api_pytorch.TestDeepSeekV3Lite

[tensorrt-cicd]

PR_Github #60140 [ skip ] completed with state SUCCESS. Commit: e2e81d0
Skipping testing for commit e2e81d0

Link to invocation

[Wanli-Jiang]

/bot skip --comment "all pre-merge/post-merge tests were passed in different runs"

[tensorrt-cicd]

PR_Github #60153 [ skip ] triggered by Bot. Commit: e2e81d0 Link to invocation

[tensorrt-cicd]

PR_Github #60153 [ skip ] completed with state SUCCESS. Commit: e2e81d0
Skipping testing for commit e2e81d0

Link to invocation