feat(protocols)!: support cached multimodal UUID content parts#119
Conversation
31cd250 to
522a09c
Compare
Add vLLM-compatible top-level cache UUIDs and nullable media content while retaining the legacy nested UUID fields as deprecated API. Signed-off-by: Indrajit Bhosale <iamindrajitb@gmail.com> Signed-off-by: Qi Wang <qiwa@nvidia.com>
keivenchang
left a comment
There was a problem hiding this comment.
pre-approving, but please address the change-of-type comments
522a09c to
026c4b7
Compare
There was a problem hiding this comment.
Can't find any particular reason to keep uuid nested within the current fields.
I don't see it defined in https://github.com/64bit/async-openai nor https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create#(resource)%20chat.completions%20%3E%20(model)%20chat_completion_content_part_image%20%3E%20(schema)
I see SMG defined it in a similar level as the existing location, nested in image_url dict: https://github.com/lightseekorg/smg/blob/d43aa0420fe7e30504801e3af77d26d0f59826f7/crates/multimodal/src/types.rs#L52, but I don't see it used in their documentation or examples either, it is always set to None: https://github.com/lightseekorg/smg/blob/d43aa0420fe7e30504801e3af77d26d0f59826f7/model_gateway/src/routers/grpc/multimodal/detect.rs#L38
Similarly I don't see dynamo using the uuid field in the current location with any value other than None: https://github.com/search?q=repo%3Aai-dynamo%2Fdynamo+%22uuid%22+image_url+language%3ARust&type=code
Let's go with this to support the vllm uuid feature. Thanks for adding the deprecation notice.
|
Further context here: https://docs.vllm.ai/en/stable/features/multimodal_inputs/#image-inputs_1 |
Why
vLLM extends OpenAI-compatible chat content parts with opaque multimodal cache UUIDs, including UUID-only requests where the media field is null. Dynamo needs to represent that wire contract without removing its existing nested UUID API. The canonical top-level field is explicitly documented as vLLM-only; legacy nested UUID fields remain available but deprecated.
ref: https://docs.vllm.ai/en/stable/features/multimodal_inputs/#image-inputs_1
Effect
{"type":"image_url","image_url":null,"uuid":"catalog-image-42"}What Change
Test Plan
cargo test -p dynamo-protocolscargo check --workspace --all-targets