Skip to content

[roundtrip-sweep] ModelMessagesTypeAdapter: NativeToolReturnPart placed in ModelRequest.parts silently fails JSON round-trip (different gap fr [Content truncated due to length] #8234

Description

@github-actions

Impact

A NativeToolReturnPart placed inside a ModelRequest.parts cannot survive a JSON round-trip through ModelMessagesTypeAdapter:

  • dump_json succeeds (with PydanticSerializationUnexpectedValue warnings, because the request-side union doesn't list the part type).
  • The resulting JSON carries part_kind: 'builtin-tool-return' and 'timestamp': ..., and the content/tool-id/provider_name all serialize.
  • validate_json then raises pydantic_core.ValidationError: 1 validation error for list[tagged-union[ModelRequest,ModelResponse]] — 1.request.parts.0 — Input tag 'builtin-tool-return' found using _model_request_part_discriminator() does not match any of the expected tags.

Affected paths that produce ModelRequest(parts=[NativeToolReturnPart(...)]) in real code:

  • pydantic_ai/ui/ag_ui/_adapter.py:533 — when an AG-UI client sends a ToolMessage whose toolCallId carries the BUILTIN_TOOL_CALL_ID_PREFIX (i.e. a server-side native tool return), the adapter builds a NativeToolReturnPart and adds it to a ModelRequest. Anyone who then reads AgentRun.all_messages_json() / new_messages_json() and re-loads (e.g. logfire exporters, Temporal payload converters, downstream consumers) hits this gap.
  • Any user code that constructs such a history directly and tries to feed it back via result.new_messages() / message_history=.

The response-side path is fine: ModelResponse(parts=[NativeToolReturnPart(...)]) round-trips cleanly today (the tag is registered at messages.py:2756). Only the request-side path is broken.

Boundary & Code Path

  • Serializer/Deserializer: ModelMessagesTypeAdapter (defined at pydantic_ai_slim/pydantic_ai/messages.py:2996).
  • Union missing the tag: ModelRequestPart at pydantic_ai_slim/pydantic_ai/messages.py:2698 — its 8 members are system-prompt | user-prompt | speech | tool-search-return | capability-load-return | tool-return | retry-prompt | tool-availability-delta. The builtin-tool-return tag is not among them.
  • Mirror union (correct one): ModelResponsePart at messages.py:2756 does include Annotated[NativeToolReturnPart, pydantic.Tag('builtin-tool-return')] — this is the pattern the request side is missing.
  • Maintainer acknowledgment of the intent: NativeToolReturnPart.narrow_type's docstring at messages.py:1725-1729 literally says "keeping it on a base part would break a ModelMessagesTypeAdapter round-trip" — the runtime already wants this to round-trip; the discriminator just isn't wired up symmetrically.

Reproduction

Drop this into tests/test_messages.py — it fails today on the JSON round-trip with the same ValidationError documented above:

def test_native_tool_return_part_in_model_request_roundtrips():
    """`NativeToolReturnPart` is registered in `ModelResponsePart` (tag `builtin-tool-return`) but
    omitted from `ModelRequestPart`. Real AG-UI clients can build a `ModelRequest` carrying it —
    see `ag_ui/_adapter.py:533` — so dumping such a history (e.g. `all_messages_json()`) and
    re-validating raises `ValidationError: Input tag 'builtin-tool-return' does not match any
    of the expected tags` instead of round-tripping cleanly. Construction and response-side
    round-trips work; only the request path is broken.

    See: `pydantic_ai/messages.py:2698` for `ModelRequestPart` (gap) and `:2756` for
    `ModelResponsePart` (registration).
    """
    messages: list[ModelMessage] = [
        ModelResponse(
            parts=[
                NativeToolCallPart(
                    tool_name='web_search',
                    args={'query': 'pydantic-ai'},
                    tool_call_id='ws_1',
                    provider_name='openai',
                )
            ]
        ),
        ModelRequest(
            parts=[
                NativeToolReturnPart(
                    tool_name='web_search',
                    content={'results': [{'title': 'Pydantic AI', 'url': 'https://ai.pydantic.dev'}]},
                    tool_call_id='ws_1',
                    provider_name='openai',
                )
            ]
        ),
    ]

    with warnings.catch_warnings():
        warnings.simplefilter('ignore', category=UserWarning)
        json_bytes = ModelMessagesTypeAdapter.dump_json(messages)

    json_roundtripped = ModelMessagesTypeAdapter.validate_json(json_bytes)
    assert json_roundtripped == messages

Run:

.venv/bin/python -m pytest tests/test_messages.py::test_native_tool_return_part_in_model_request_roundtrips -x

Captured output (full failure):

pydantic_core._pydantic_core.ValidationError: 1 validation error for list[tagged-union[ModelRequest,ModelResponse]]
1.request.parts.0
  Input tag 'builtin-tool-return' found using _model_request_part_discriminator() does not match
  any of the expected tags: 'system-prompt', 'user-prompt', 'speech', 'tool-search-return',
  'capability-load-return', 'tool-return', 'retry-prompt', 'tool-availability-delta'
  [type=union_tag_invalid, input_value={'tool_name': 'web_search', ...,
                                          'part_kind': 'builtin-tool-return'}, input_type=dict]

Expected vs Actual

Expected: ModelMessagesTypeAdapter.validate_json(ModelMessagesTypeAdapter.dump_json(messages)) == messages for any input the runtime can produce via AgentRun.all_messages().

Actual: A ValidationError aborts the round-trip whenever the request side carries a NativeToolReturnPart. The same JSON deserializes fine if the part is rebuilt as part of a ModelResponse (already registered at messages.py:2756).

Evidence

  • pydantic_ai_slim/pydantic_ai/messages.py:2698-2708ModelRequestPart declaration (8 tags, no builtin-tool-return).
  • pydantic_ai_slim/pydantic_ai/messages.py:2748-2762ModelResponsePart declaration (includes the tag at line 2756).
  • pydantic_ai_slim/pydantic_ai/messages.py:1718part_kind: Literal['builtin-tool-return'] = 'builtin-tool-return' on the base class.
  • pydantic_ai_slim/pydantic_ai/ui/ag_ui/_adapter.py:533 — produces NativeToolReturnPart inside a ModelRequest.
  • pydantic_ai_slim/pydantic_ai/messages.py:1725-1729 — docstring on narrow_type already states the round-trip intent.
  • pydantic_ai/run.py:171-192 / pydantic_ai/result.py:545-562AgentRun/StreamedRunResult JSON dump methods, which silently emit UserWarnings today and break on any consumer that tries to re-parse.

Adversarial review

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    p:4-lowSignal 0-3: low, probably unplanned.pydanty:botManaged by pydanty dogfooding automationroundtrip-sweep

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions