[None][fix] Stop forcing Mixed hybrid manager for PYTHON transceiver runtime#3
Open
lingjiew wants to merge 1 commit into
Conversation
…runtime The transceiver_runtime=='PYTHON' -> MixedMambaHybridCacheManager rule predates V2 disagg support: KvCacheTransceiverV2 now drives V2MambaHybridCacheManager natively (d19e249). With block reuse enabled the rule was already dead code (the reuse branch returns the V2 manager earlier); with reuse disabled it silently downgraded PYTHON-runtime deployments to the Mixed manager instead of the default V2 manager. Remove it — Mixed remains available via TLLM_MAMBA_MANAGER_PREFERENCE=MIXED. The V2 manager + KvCacheTransceiverV2 pairing is validated end-to-end on Qwen3.5-397B-A17B GB300 disagg (37-point ctx-only + 52-point gen-only sweeps, realized KV reuse 93-94%). Signed-off-by: Lingjie Wu <lingjiew@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
In
_util.pyhybrid manager selection,transceiver_runtime == 'PYTHON'force-selectsMixedMambaHybridCacheManager. The rule predates this branch's V2 disagg support (d19e249):KvCacheTransceiverV2now drivesV2MambaHybridCacheManagernatively.Effect today:
enable_block_reuse=Truethe rule is dead code (the reuse branch returns the V2 manager a few lines earlier);transceiver_runtime: PYTHONentirely and reach the python transceiver via the V2-manager isinstance re-route).Fix
Remove the override;
PYTHONruntime now selects managers exactly likeCPP/unset does. Mixed remains available viaTLLM_MAMBA_MANAGER_PREFERENCE=MIXED.Validation
The resulting pairing (V2MambaHybridCacheManager + KvCacheTransceiverV2) is the one validated end-to-end on Qwen3.5-397B-A17B-NVFP4 GB300 disagg: 37-point ctx-only sweep + 52-point gen-only sweep, KV reuse realized 93-94%, conversation-affinity ADP routing and MTP forced-AL all green (2026-07-06).
Related: #2 (ADP dummy-slot guard), and a sibling PR adding logging/validation to the transceiver factory.
🤖 Generated with Claude Code