fix(trtllm): expose env var to force engine side conversation dp assignment

[GuanLuo]

Signed-off-by: Guan Luo 41310872+GuanLuo@users.noreply.github.com

Overview

Adds --conversation-affinity / DYN_ENGINE_CONV_AFFINITY to let operators force engine-side conversation-affinity attention-DP routing for TRT-LLM workers, without relying on engine auto-detection. Useful when the engine reports no conversation-affinity capability at handler construction time (e.g. older TRT-LLM, disagg scenarios where the engine is not yet initialized).

Details

• backend_args.py: registers --conversation-affinity (negatable bool, env DYN_ENGINE_CONV_AFFINITY, default False) and adds conversation_affinity: bool to DynamoTrtllmConfig.
• llm_worker.py: passes config.conversation_affinity into RequestHandlerConfig.
• handler_base.py: stores the override as _engine_conversation_affinity_override and ORs it with the engine-detected _conversation_affinity flag when deciding whether to use ConversationParams vs. SchedulingParams.

Where should the reviewer start?

• components/src/dynamo/trtllm/request_handlers/handler_base.py — the routing decision at conv_affinity = self._conversation_affinity or self._engine_conversation_affinity_override and the conv_kwargs path below it.
• components/src/dynamo/trtllm/backend_args.py — new flag registration.

Related Issues

🚫 This PR is NOT linked to an issue:

• Confirmed — no related issue

[coderabbitai]

Walkthrough

Changes

Conversation affinity routing

Layer / File(s)	Summary
Configuration and handler wiring components/src/dynamo/trtllm/backend_args.py, components/src/dynamo/trtllm/workers/llm_worker.py	Adds the --conversation-affinity setting and passes it into RequestHandlerConfig.
Request handler routing override components/src/dynamo/trtllm/request_handlers/handler_base.py	Stores the override and enables conversation-based attention-DP parameters when configured or detected.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: exposing a setting to force engine-side conversation DP assignment in TRT-LLM.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

components/src/dynamo/trtllm/request_handlers/handler_base.py (1)

1154-1179: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Use the combined affinity flag when forwarding conversation_params. In components/src/dynamo/trtllm/request_handlers/handler_base.py:1154-1178, the config override path builds conversation_params but still drops it at the call site because conv_kwargs is gated only by conv_affinity. That leaves override-only requests with neither conversation_params nor scheduling_params. Reuse the combined flag here and add a regression test for override enabled + engine detection disabled.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@components/src/dynamo/trtllm/request_handlers/handler_base.py` around lines
1154 - 1179, The conversation_params forwarding logic near conv_kwargs
incorrectly checks only conv_affinity; gate it with the combined affinity
condition that also includes self._engine_conversation_affinity_override, so
override-only requests receive conversation_params while preserving scheduling
behavior. Add a regression test covering the override-enabled,
engine-detection-disabled case and verify the generated request includes
conversation_params.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@components/src/dynamo/trtllm/request_handlers/handler_base.py`:
- Around line 1154-1179: The conversation_params forwarding logic near
conv_kwargs incorrectly checks only conv_affinity; gate it with the combined
affinity condition that also includes
self._engine_conversation_affinity_override, so override-only requests receive
conversation_params while preserving scheduling behavior. Add a regression test
covering the override-enabled, engine-detection-disabled case and verify the
generated request includes conversation_params.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7b686791-5243-4844-8e88-76c8ba84a149

📥 Commits

Reviewing files that changed from the base of the PR and between b0b7786 and 266ab73.

📒 Files selected for processing (3)

• components/src/dynamo/trtllm/backend_args.py
• components/src/dynamo/trtllm/request_handlers/handler_base.py
• components/src/dynamo/trtllm/workers/llm_worker.py

[datadog-official]

🎯 Code Coverage (details)
• Patch Coverage: 95.65%
• Overall Coverage: 29.85% (-17.96%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: cee306a | Docs | Give us feedback!}