Skip to content

feat(protocols)!: support cached multimodal UUID content parts#119

Merged
rmccorm4 merged 1 commit into
ai-dynamo:mainfrom
furionw:qiwa/cached-mm-uuid-schema
Jul 16, 2026
Merged

feat(protocols)!: support cached multimodal UUID content parts#119
rmccorm4 merged 1 commit into
ai-dynamo:mainfrom
furionw:qiwa/cached-mm-uuid-schema

Conversation

@furionw

@furionw furionw commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Why

vLLM extends OpenAI-compatible chat content parts with opaque multimodal cache UUIDs, including UUID-only requests where the media field is null. Dynamo needs to represent that wire contract without removing its existing nested UUID API. The canonical top-level field is explicitly documented as vLLM-only; legacy nested UUID fields remain available but deprecated.

ref: https://docs.vllm.ai/en/stable/features/multimodal_inputs/#image-inputs_1

Effect

{"type":"image_url","image_url":null,"uuid":"catalog-image-42"}

What Change

  • Add top-level opaque UUIDs and nullable media content.
  • Preserve required nested URLs and deprecated UUID fields.
  • Cover URL-backed, UUID-only, and legacy nested payloads.

Test Plan

  • cargo test -p dynamo-protocols
  • cargo check --workspace --all-targets

@furionw furionw marked this pull request as ready for review July 16, 2026 03:16
devin-ai-integration[bot]

This comment was marked as resolved.

@furionw furionw marked this pull request as draft July 16, 2026 03:52
@furionw furionw force-pushed the qiwa/cached-mm-uuid-schema branch from 31cd250 to 522a09c Compare July 16, 2026 06:37
@furionw furionw changed the title feat(protocols)!: support cached multimodal UUID content parts feat(protocols): support cached multimodal UUID content parts Jul 16, 2026
@furionw furionw marked this pull request as ready for review July 16, 2026 16:35
devin-ai-integration[bot]

This comment was marked as resolved.

Comment thread protocols/src/types/chat.rs
Add vLLM-compatible top-level cache UUIDs and nullable media content while retaining the legacy nested UUID fields as deprecated API.

Signed-off-by: Indrajit Bhosale <iamindrajitb@gmail.com>

Signed-off-by: Qi Wang <qiwa@nvidia.com>
@furionw furionw changed the title feat(protocols): support cached multimodal UUID content parts feat(protocols)!: support cached multimodal UUID content parts Jul 16, 2026

@keivenchang keivenchang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pre-approving, but please address the change-of-type comments

@furionw furionw force-pushed the qiwa/cached-mm-uuid-schema branch from 522a09c to 026c4b7 Compare July 16, 2026 18:00

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 0 new potential issues.

Open in Devin Review

@rmccorm4 rmccorm4 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't find any particular reason to keep uuid nested within the current fields.

I don't see it defined in https://github.com/64bit/async-openai nor https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create#(resource)%20chat.completions%20%3E%20(model)%20chat_completion_content_part_image%20%3E%20(schema)

I see SMG defined it in a similar level as the existing location, nested in image_url dict: https://github.com/lightseekorg/smg/blob/d43aa0420fe7e30504801e3af77d26d0f59826f7/crates/multimodal/src/types.rs#L52, but I don't see it used in their documentation or examples either, it is always set to None: https://github.com/lightseekorg/smg/blob/d43aa0420fe7e30504801e3af77d26d0f59826f7/model_gateway/src/routers/grpc/multimodal/detect.rs#L38

Similarly I don't see dynamo using the uuid field in the current location with any value other than None: https://github.com/search?q=repo%3Aai-dynamo%2Fdynamo+%22uuid%22+image_url+language%3ARust&type=code

Let's go with this to support the vllm uuid feature. Thanks for adding the deprecation notice.

@rmccorm4 rmccorm4 merged commit 5cf0f58 into ai-dynamo:main Jul 16, 2026
8 checks passed
@rmccorm4

Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants