ENH: Support Qwen3.5 recurrent-state PD handoff

doc/source/locale/de/LC_MESSAGES/user_guide/pd_separation.po

@@ -149,14 +149,8 @@ msgid "Hybrid/recurrent attention limitation"
 msgstr ""
 
 #: source/user_guide/pd_separation.rst:136
-msgid ""
-"Xavier PD does not support hybrid/recurrent attention models such as "
-"Qwen3.5. Deploy these models without PD, or choose a full-attention model "
-"such as Qwen3 for PD."
-msgstr ""
-"Xavier PD unterstützt keine hybriden/rekurrenten Aufmerksamkeitsmodelle wie "
-"Qwen3.5. Stellen Sie diese Modelle ohne PD bereit oder wählen Sie für PD ein"
-" Modell mit vollständiger Aufmerksamkeit wie Qwen3."
+msgid "Qwen3.5 text requests are supported with Xavier and native NIXL. Set ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` and ``async_scheduling=False`` on both replicas. Use TP=1 and PP=1. Xavier history caching is not available for these hybrid/recurrent attention models; their configured history budget is not allocated. Image/video inputs and speculative decoding are not supported in this configuration. Native NIXL requires vLLM >= 0.22.0 for this mode."
+msgstr "Xavier und natives NIXL unterstützen Qwen3.5-Textanfragen. Setzen Sie auf beiden Replikaten ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` und ``async_scheduling=False``. Verwenden Sie TP=1 und PP=1. Der Xavier-Verlaufscache ist für diese Modelle mit hybrider/rekurrenter Attention nicht verfügbar; sein konfiguriertes Budget wird nicht reserviert. Bild-/Videoeingaben und spekulatives Decoding werden in dieser Konfiguration nicht unterstützt. Natives NIXL erfordert für diesen Modus vLLM >= 0.22.0."
 
 #: source/user_guide/pd_separation.rst:143
 msgid "Xavier requirements and memory"

doc/source/locale/es/LC_MESSAGES/user_guide/pd_separation.po

@@ -149,14 +149,8 @@ msgid "Hybrid/recurrent attention limitation"
 msgstr ""
 
 #: source/user_guide/pd_separation.rst:136
-msgid ""
-"Xavier PD does not support hybrid/recurrent attention models such as "
-"Qwen3.5. Deploy these models without PD, or choose a full-attention model "
-"such as Qwen3 for PD."
-msgstr ""
-"Xavier PD no admite modelos de atención híbrida/recurrente como Qwen3.5. "
-"Despliegue estos modelos sin PD o elija un modelo de atención completa como "
-"Qwen3 para PD."
+msgid "Qwen3.5 text requests are supported with Xavier and native NIXL. Set ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` and ``async_scheduling=False`` on both replicas. Use TP=1 and PP=1. Xavier history caching is not available for these hybrid/recurrent attention models; their configured history budget is not allocated. Image/video inputs and speculative decoding are not supported in this configuration. Native NIXL requires vLLM >= 0.22.0 for this mode."
+msgstr "Xavier y NIXL nativo admiten solicitudes de texto de Qwen3.5. Configure ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` y ``async_scheduling=False`` en ambas réplicas. Use TP=1 y PP=1. La caché histórica de Xavier no está disponible para estos modelos de atención híbrida/recurrente; no se asigna su presupuesto configurado. Esta configuración no admite imágenes, vídeos ni decodificación especulativa. NIXL nativo requiere vLLM >= 0.22.0 para este modo."
 
 #: source/user_guide/pd_separation.rst:143
 msgid "Xavier requirements and memory"

doc/source/locale/fr/LC_MESSAGES/user_guide/pd_separation.po

@@ -149,14 +149,8 @@ msgid "Hybrid/recurrent attention limitation"
 msgstr ""
 
 #: source/user_guide/pd_separation.rst:136
-msgid ""
-"Xavier PD does not support hybrid/recurrent attention models such as "
-"Qwen3.5. Deploy these models without PD, or choose a full-attention model "
-"such as Qwen3 for PD."
-msgstr ""
-"Xavier PD ne prend pas en charge les modèles à attention hybride/récurrente "
-"comme Qwen3.5. Déployez ces modèles sans PD ou choisissez un modèle à "
-"attention complète comme Qwen3 pour le PD."
+msgid "Qwen3.5 text requests are supported with Xavier and native NIXL. Set ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` and ``async_scheduling=False`` on both replicas. Use TP=1 and PP=1. Xavier history caching is not available for these hybrid/recurrent attention models; their configured history budget is not allocated. Image/video inputs and speculative decoding are not supported in this configuration. Native NIXL requires vLLM >= 0.22.0 for this mode."
+msgstr "Xavier et NIXL natif prennent en charge les requêtes textuelles Qwen3.5. Définissez ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` et ``async_scheduling=False`` sur les deux réplicas. Utilisez TP=1 et PP=1. Le cache historique Xavier est indisponible pour ces modèles à attention hybride/récurrente ; son budget configuré n’est pas alloué. Les entrées image/vidéo et le décodage spéculatif ne sont pas pris en charge dans cette configuration. Dans ce mode, NIXL natif nécessite vLLM >= 0.22.0."
 
 #: source/user_guide/pd_separation.rst:143
 msgid "Xavier requirements and memory"

doc/source/locale/it/LC_MESSAGES/user_guide/pd_separation.po

@@ -149,14 +149,8 @@ msgid "Hybrid/recurrent attention limitation"
 msgstr ""
 
 #: source/user_guide/pd_separation.rst:136
-msgid ""
-"Xavier PD does not support hybrid/recurrent attention models such as "
-"Qwen3.5. Deploy these models without PD, or choose a full-attention model "
-"such as Qwen3 for PD."
-msgstr ""
-"Xavier PD non supporta modelli con attenzione ibrida/ricorrente come "
-"Qwen3.5. Distribuisci questi modelli senza PD oppure scegli un modello con "
-"attenzione completa come Qwen3 per il PD."
+msgid "Qwen3.5 text requests are supported with Xavier and native NIXL. Set ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` and ``async_scheduling=False`` on both replicas. Use TP=1 and PP=1. Xavier history caching is not available for these hybrid/recurrent attention models; their configured history budget is not allocated. Image/video inputs and speculative decoding are not supported in this configuration. Native NIXL requires vLLM >= 0.22.0 for this mode."
+msgstr "Xavier e NIXL nativo supportano le richieste di testo Qwen3.5. Imposta ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` e ``async_scheduling=False`` su entrambe le repliche. Usa TP=1 e PP=1. La cache storica Xavier non è disponibile per questi modelli con attenzione ibrida/ricorrente; il budget configurato non viene allocato. Questa configurazione non supporta immagini, video o decodifica speculativa. In questa modalità NIXL nativo richiede vLLM >= 0.22.0."
 
 #: source/user_guide/pd_separation.rst:143
 msgid "Xavier requirements and memory"

doc/source/locale/ja/LC_MESSAGES/user_guide/pd_separation.po

@@ -143,13 +143,8 @@ msgid "Hybrid/recurrent attention limitation"
 msgstr ""
 
 #: source/user_guide/pd_separation.rst:136
-msgid ""
-"Xavier PD does not support hybrid/recurrent attention models such as "
-"Qwen3.5. Deploy these models without PD, or choose a full-attention model "
-"such as Qwen3 for PD."
-msgstr ""
-"Xavier PD は Qwen3.5 などのハイブリッド／再帰型アテンションモデルをサポートしません。これらは PD なしでデプロイするか、PD には"
-" Qwen3 などのフルアテンションモデルを選択してください。"
+msgid "Qwen3.5 text requests are supported with Xavier and native NIXL. Set ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` and ``async_scheduling=False`` on both replicas. Use TP=1 and PP=1. Xavier history caching is not available for these hybrid/recurrent attention models; their configured history budget is not allocated. Image/video inputs and speculative decoding are not supported in this configuration. Native NIXL requires vLLM >= 0.22.0 for this mode."
+msgstr "Xavier とネイティブ NIXL は Qwen3.5 のテキストリクエストに対応しています。両方のレプリカで ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` および ``async_scheduling=False`` を設定し、TP=1、PP=1 を使用してください。これらのハイブリッド／再帰型アテンションモデルでは Xavier の履歴キャッシュは利用できず、設定された履歴キャッシュ容量は確保されません。この構成は画像・動画入力と投機的デコードに対応していません。 このモードでネイティブ NIXL を使うには vLLM >= 0.22.0 が必要です。"
 
 #: source/user_guide/pd_separation.rst:143
 msgid "Xavier requirements and memory"

doc/source/locale/ko/LC_MESSAGES/user_guide/pd_separation.po

@@ -143,13 +143,8 @@ msgid "Hybrid/recurrent attention limitation"
 msgstr ""
 
 #: source/user_guide/pd_separation.rst:136
-msgid ""
-"Xavier PD does not support hybrid/recurrent attention models such as "
-"Qwen3.5. Deploy these models without PD, or choose a full-attention model "
-"such as Qwen3 for PD."
-msgstr ""
-"Xavier PD는 Qwen3.5 같은 하이브리드/순환 어텐션 모델을 지원하지 않습니다. 이러한 모델은 PD 없이 배포하거나, PD에는 "
-"Qwen3 같은 전체 어텐션 모델을 선택하세요."
+msgid "Qwen3.5 text requests are supported with Xavier and native NIXL. Set ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` and ``async_scheduling=False`` on both replicas. Use TP=1 and PP=1. Xavier history caching is not available for these hybrid/recurrent attention models; their configured history budget is not allocated. Image/video inputs and speculative decoding are not supported in this configuration. Native NIXL requires vLLM >= 0.22.0 for this mode."
+msgstr "Xavier와 네이티브 NIXL은 Qwen3.5 텍스트 요청을 지원합니다. 두 복제본 모두에서 ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` 및 ``async_scheduling=False`` 를 설정하고 TP=1, PP=1을 사용하세요. 이러한 하이브리드/순환 어텐션 모델에서는 Xavier 기록 캐시를 사용할 수 없으므로 설정된 기록 캐시 용량은 할당되지 않습니다. 이 구성은 이미지·동영상 입력과 추측 디코딩을 지원하지 않습니다. 이 모드에서 네이티브 NIXL을 사용하려면 vLLM >= 0.22.0이 필요합니다."
 
 #: source/user_guide/pd_separation.rst:143
 msgid "Xavier requirements and memory"

doc/source/locale/pt_BR/LC_MESSAGES/user_guide/pd_separation.po

@@ -149,14 +149,8 @@ msgid "Hybrid/recurrent attention limitation"
 msgstr ""
 
 #: source/user_guide/pd_separation.rst:136
-msgid ""
-"Xavier PD does not support hybrid/recurrent attention models such as "
-"Qwen3.5. Deploy these models without PD, or choose a full-attention model "
-"such as Qwen3 for PD."
-msgstr ""
-"O Xavier PD não suporta modelos de atenção híbrida/recorrente como Qwen3.5. "
-"Implante esses modelos sem PD ou escolha um modelo de atenção completa como "
-"Qwen3 para PD."
+msgid "Qwen3.5 text requests are supported with Xavier and native NIXL. Set ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` and ``async_scheduling=False`` on both replicas. Use TP=1 and PP=1. Xavier history caching is not available for these hybrid/recurrent attention models; their configured history budget is not allocated. Image/video inputs and speculative decoding are not supported in this configuration. Native NIXL requires vLLM >= 0.22.0 for this mode."
+msgstr "Xavier e NIXL nativo suportam solicitações de texto do Qwen3.5. Defina ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` e ``async_scheduling=False`` nas duas réplicas. Use TP=1 e PP=1. O cache histórico do Xavier não está disponível para esses modelos de atenção híbrida/recorrente; seu orçamento configurado não é alocado. Esta configuração não suporta imagens, vídeos ou decodificação especulativa. Nesse modo, o NIXL nativo requer vLLM >= 0.22.0."
 
 #: source/user_guide/pd_separation.rst:143
 msgid "Xavier requirements and memory"

doc/source/locale/zh_CN/LC_MESSAGES/user_guide/pd_separation.po

@@ -148,13 +148,8 @@ msgid "Hybrid/recurrent attention limitation"
 msgstr "混合注意力与循环注意力的限制"
 
 #: source/user_guide/pd_separation.rst:136
-msgid ""
-"Xavier PD does not support hybrid/recurrent attention models such as "
-"Qwen3.5. Deploy these models without PD, or choose a full-attention model "
-"such as Qwen3 for PD."
-msgstr ""
-"Xavier PD 不支持 Qwen3.5 等混合或循环注意力模型。请在不启用 PD 的情况下部署这些模型，或选择 Qwen3 等全注意力模型使用 "
-"PD。"
+msgid "Qwen3.5 text requests are supported with Xavier and native NIXL. Set ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` and ``async_scheduling=False`` on both replicas. Use TP=1 and PP=1. Xavier history caching is not available for these hybrid/recurrent attention models; their configured history budget is not allocated. Image/video inputs and speculative decoding are not supported in this configuration. Native NIXL requires vLLM >= 0.22.0 for this mode."
+msgstr "Xavier 和原生 NIXL 支持 Qwen3.5 纯文本请求。两个副本均需设置 ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` 和 ``async_scheduling=False``，并使用 TP=1、PP=1。此类混合或循环注意力模型暂不支持 Xavier 历史缓存，因此不会分配配置的历史缓存预算。该配置不支持图片、视频输入或推测解码。 此模式下原生 NIXL 需要 vLLM >= 0.22.0。"
 
 #: source/user_guide/pd_separation.rst:143
 msgid "Xavier requirements and memory"

doc/source/locale/zh_TW/LC_MESSAGES/user_guide/pd_separation.po

@@ -143,13 +143,8 @@ msgid "Hybrid/recurrent attention limitation"
 msgstr ""
 
 #: source/user_guide/pd_separation.rst:136
-msgid ""
-"Xavier PD does not support hybrid/recurrent attention models such as "
-"Qwen3.5. Deploy these models without PD, or choose a full-attention model "
-"such as Qwen3 for PD."
-msgstr ""
-"Xavier PD 不支援 Qwen3.5 等混合或循環注意力模型。請在不啟用 PD 的情況下部署這些模型，或選擇 Qwen3 等全注意力模型使用 "
-"PD。"
+msgid "Qwen3.5 text requests are supported with Xavier and native NIXL. Set ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` and ``async_scheduling=False`` on both replicas. Use TP=1 and PP=1. Xavier history caching is not available for these hybrid/recurrent attention models; their configured history budget is not allocated. Image/video inputs and speculative decoding are not supported in this configuration. Native NIXL requires vLLM >= 0.22.0 for this mode."
+msgstr "Xavier 和原生 NIXL 支援 Qwen3.5 純文字請求。兩個副本均需設定 ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode=\"none\"``, ``disable_hybrid_kv_cache_manager=False`` 和 ``async_scheduling=False``，並使用 TP=1、PP=1。此類混合或循環注意力模型暫不支援 Xavier 歷史快取，因此不會配置設定的歷史快取預算。此設定不支援圖片、影片輸入或推測解碼。 此模式下原生 NIXL 需要 vLLM >= 0.22.0。"
 
 #: source/user_guide/pd_separation.rst:143
 msgid "Xavier requirements and memory"

doc/source/user_guide/pd_separation.rst

@@ -120,7 +120,7 @@ After a worker restart, relaunch the native NIXL PD deployment.
 Hybrid/recurrent attention limitation
 -------------------------------------
 
-Xavier PD does not support hybrid/recurrent attention models such as Qwen3.5. Deploy these models without PD, or choose a full-attention model such as Qwen3 for PD.
+Qwen3.5 text requests are supported with Xavier and native NIXL. Set ``language_model_only=True``, ``enable_prefix_caching=False``, ``mamba_cache_mode="none"``, ``disable_hybrid_kv_cache_manager=False`` and ``async_scheduling=False`` on both replicas. Use TP=1 and PP=1. Xavier history caching is not available for these hybrid/recurrent attention models; their configured history budget is not allocated. Image/video inputs and speculative decoding are not supported in this configuration. Native NIXL requires vLLM >= 0.22.0 for this mode.
 
 Xavier requirements and memory
 ------------------------------

xinference/model/llm/vllm/core.py

@@ -796,15 +796,23 @@ def load(self):
         if self._nixl_config is not None:
             from .pd import configure_nixl_engine
 
-            if isinstance(self, VLLMMultiModel):
-                raise ValueError("Native NIXL PD currently supports text-only models")
+            if isinstance(self, VLLMMultiModel) and not self._model_config.get(
+                "language_model_only", False
+            ):
+                raise ValueError(
+                    "Native NIXL PD requires text-only models or language_model_only=True"
+                )
+            if isinstance(self, VLLMMultiModel) and VLLM_VERSION < version.parse(
+                "0.22.0"
+            ):
+                raise ValueError(
+                    "Native NIXL PD for language_model_only models requires vLLM >= 0.22.0"
+                )
             configure_nixl_engine(self._model_config, VLLM_VERSION, enable_lora)
         use_native_mp, native_mp_reason = self._native_mp_route()
         if self._xavier_config is not None:
             from .xavier.engine import XavierEngine
 
-            # Enabling Xavier means that `enable_prefix_caching` is enabled by default.
-            self._model_config.setdefault("enable_prefix_caching", True)
             xavier_transfer_block_num = self._model_config.pop(
                 "xavier_transfer_block_num", 512
             )
@@ -2951,7 +2959,13 @@ async def async_chat(
             )
             assert not isinstance(c, AsyncGenerator)
             if tools:
-                return self._post_process_completion(
+                result = self._post_process_completion(
                     self.model_family, self.model_uid, c
                 )
-            return self._to_chat_completion(c, self.reasoning_parser)
+            else:
+                result = self._to_chat_completion(c, self.reasoning_parser)
+            if "_pd_kv_transfer_params" in c:
+                result["_pd_kv_transfer_params"] = cast(Dict[str, Any], c)[
+                    "_pd_kv_transfer_params"
+                ]
+            return result

xinference/model/llm/vllm/pd.py

@@ -41,6 +41,12 @@ def resolve_nixl_host(host: str, host_key: str = "VLLM_NIXL_SIDE_CHANNEL_HOST")
 def configure_nixl_environment(env: Dict[str, str], worker_address: str) -> None:
     from xoscar.utils import get_next_port
 
+    # Native NIXL's recurrent-state transfer requires dimension-first conv
+    # states. Select the layout before vLLM initializes model/cache classes.
+    env.setdefault(
+        "VLLM_SSM_CONV_STATE_LAYOUT",
+        os.environ.get("VLLM_SSM_CONV_STATE_LAYOUT", "DS"),
+    )
     # Each model subprocess needs its own listener, including same-host replicas.
     # Reallocate on recovery as the old listener may not have exited yet.
     host_key = "VLLM_NIXL_SIDE_CHANNEL_HOST"