fix json bug

[MiXaiLL76]

What happened + What you expected to happen

Deploying a Ray Serve LLM (ray.serve.llm.build_openai_app) with a nested
dataclass field in engine_kwargs — e.g. structured_outputs_config — crashes
the vLLM engine on startup with:

AttributeError: 'dict' object has no attribute 'reasoning_parser'

or, depending on which field is accessed first:

AttributeError: 'dict' object has no attribute 'compute_hash'

Expected: the engine starts successfully and structured_outputs_config is
applied to the model, exactly as it would with vLLM's own CLI
(--structured-outputs-config.enable_in_reasoning=True).

Why this matters

This blocks the documented upstream vLLM fix for broken structured outputs on
Qwen3 Coder + reasoning models
(vllm-project/vllm#18819),
which requires setting structured_outputs_config.enable_in_reasoning=True.
Ray Serve LLM users deploying via YAML (serveConfigV2 / LLMConfig.engine_kwargs)
have no working way to set this flag — the only way to express a nested vLLM
config field from YAML is as a plain dict, and that dict reaches
AsyncEngineArgs unconverted.

Root cause

_get_vllm_engine_config() in
python/ray/llm/_internal/serve/engines/vllm/vllm_engine.py builds the engine
config like this:

async_engine_args = vllm.engine.arg_utils.AsyncEngineArgs(
    **engine_config.get_initialization_kwargs()
)
vllm_engine_config = async_engine_args.create_engine_config(
    usage_context=UsageContext.OPENAI_API_SERVER
)

engine_config.get_initialization_kwargs() returns a plain dict for every
engine_kwargs entry, including nested ones like structured_outputs_config.
vllm.engine.arg_utils.EngineArgs.__post_init__ only coerces a fixed allowlist
of fields from dict to their dataclass type (compilation_config,
attention_config, mamba_config, kernel_config, eplb_config,
weight_transfer_config, ir_op_priority). structured_outputs_config and
reasoning_config are not in that list, so they stay as plain dict
objects on the AsyncEngineArgs instance.

create_engine_config() then does (vLLM engine/arg_utils.py):

if self.reasoning_parser:
    self.structured_outputs_config.reasoning_parser = self.reasoning_parser

— which crashes immediately with AttributeError: 'dict' object has no attribute 'reasoning_parser' whenever a top-level engine_kwargs.reasoning_parser
is also set (a very common config for Qwen3 reasoning models). Even without
that top-level field, VllmConfig.compute_hash() later calls
self.structured_outputs_config.compute_hash(), which crashes the same way.

There already is a fix for a similar symptom — but it doesn't cover this path

#60380 ([LLM] Fix nested dict to Namespace conversion in vLLM engine initialization, merged) introduced a helper, _convert_config_dicts(), that
converts known AsyncEngineArgs dict fields into their proper dataclass
instances using field type hints. It is called in VLLMEngine.start():

merged = vllm_frontend_args.__dict__ | vllm_engine_args.__dict__
merged = _convert_config_dicts(merged)
args = _dict_to_namespace(merged)
self._vllm_args = args

This fixes AttributeError in the serving/frontend layer
(build_asgi_app() / init_app_state(), which reads
args.structured_outputs_config.backend etc. to configure the OpenAI-compatible
endpoint). But self._vllm_args is only built after the engine itself has
already been started (self._start_async_llm_engine(vllm_engine_args, ...)),
using the args that came out of _get_vllm_engine_config(). That function
builds AsyncEngineArgs and calls create_engine_config() before
_convert_config_dicts() ever runs — so the crash described above still
reproduces on top of #60380.

Reproduction

from ray.llm._internal.serve.core.configs.llm_config import LLMConfig, ModelLoadingConfig
from ray.llm._internal.serve.engines.vllm.vllm_engine import _get_vllm_engine_config

llm_config = LLMConfig(
    model_loading_config=ModelLoadingConfig(
        model_id="qwen-test", model_source="Qwen/Qwen3-4B"
    ),
    engine_kwargs={
        "reasoning_parser": "qwen3",
        "structured_outputs_config": {"enable_in_reasoning": True},
    },
)
_get_vllm_engine_config(llm_config)
# AttributeError: 'dict' object has no attribute 'reasoning_parser'

Or via a RayService serveConfigV2:

applications:
  - name: llms
    import_path: ray.serve.llm:build_openai_app
    route_prefix: "/"
    args:
      llm_configs:
        - model_loading_config:
            model_id: qwen3-4b-thinking
            model_source: /models/Qwen3-4B
          engine_kwargs:
            tensor_parallel_size: 2
            reasoning_parser: qwen3
            structured_outputs_config:
              backend: xgrammar
              enable_in_reasoning: true
          deployment_config:
            autoscaling_config:
              min_replicas: 1
              max_replicas: 1

The vLLM engine actor crashes on startup (CrashLoopBackOff under KubeRay).

Related

• [Bug]: Broken Structured Output (Guided Decoding) with Qwen3 models when enable_thinking=False vllm-project/vllm#18819 — the upstream fix this blocks (enable_in_reasoning)
• [LLM] Fix nested dict to Namespace conversion in vLLM engine initialization #60380 — fixes the same class of bug in the serving/frontend layer, but not this path
• [Serve][LLM] Add regression test for nested dict in engine_kwargs #60430 — regression test for [LLM] Fix nested dict to Namespace conversion in vLLM engine initialization #60380

[gemini-code-assist]

Code Review

This pull request updates _get_vllm_engine_config in vllm_engine.py to preprocess initialization arguments using _convert_config_dicts before initializing AsyncEngineArgs. The review feedback points out a potential fragility in the _convert_config_dicts helper function when handling Union type hints, as it might resolve to a dict instead of a dataclass, and suggests a more robust implementation to prevent future regressions.

[gemini-code-assist]

While calling _convert_config_dicts here correctly resolves the initialization crash, the implementation of _convert_config_dicts (on lines 140-143) has a potential fragility when handling Union type hints.

Currently, it resolves Union types by picking the first non-None type:

hint = next((a for a in args if a is not type(None)), hint)

If a type hint is defined as Union[dict, StructuredOutputsConfig] or Union[None, dict, StructuredOutputsConfig], this logic will pick dict instead of the dataclass, causing the conversion to be skipped and leading to the same AttributeError crash.

To make this more robust, we should search the Union arguments specifically for a dataclass type:

        if origin is Union:
            args = typing.get_args(hint)
            hint = next((a for a in args if dataclasses.is_dataclass(a)), hint)

Since _convert_config_dicts is defined outside the current diff hunk, this cannot be applied via an automated code suggestion, but it is highly recommended to update its definition to prevent future regressions if vLLM's type annotations change.