[Feature]: AG-UI Protocol Multi-modal Messages

[asheng]

AgentScope-Java is an open-source project. To involve a broader community, we recommend asking your questions in English.

Is your feature request related to a problem? Please describe.
AG-UI Protocol multi-modal messages have already been implemented. I’d like to have the same support in AgentScope-Java (Spring Boot integration), along with a working example.

Describe the solution you'd like
Add support for AG-UI multi-modal messages in AgentScope-Java , and provide example code / a sample project demonstrating how to use it.

Additional context
Multi-modal Messages doc: https://docs.ag-ui.com/drafts/multimodal-messages
AgentScope-Java doc: https://java.agentscope.io/zh/task/agui.html

[junnan2014]

Cross-referencing: PR #1380 already implements exactly this (AG-UI InputContent[] → ContentBlock conversion in AguiMessage / AguiMessageConverter, following the multimodal-messages draft spec linked above).

I verified it today (2026-07-28) against the latest main (ed81704):

• The PR commit 5d6cb4b cherry-picks onto current main with zero conflicts (rename detection handles the agui module move to agentscope-extensions-protocol/ automatically).
• With the patch applied, the agui module passes 258 tests, 0 failures (81 of them are new multimodal test cases from the PR).

Since the core content blocks (ImageBlock/AudioBlock/VideoBlock) and the model-side converters (e.g. DashScope) already exist, that PR closes the only remaining gap for end-to-end multimodal input via AG-UI. It would be great if maintainers could prioritize reviewing #1380 rather than re-implementing from scratch — details of my verification are in the comment I left there.

[oss-maintainer]

Thanks for the feature request, @junnan2014.

Status: PR #1380 by @NoiAI already implements AG-UI multimodal input support (image/video/audio/document) in the agentscope-extensions-agui module. The implementation:

• Adds InputContent[] → ContentBlock conversion in AguiMessageConverter
• Supports both URL and Base64 data sources
• Changes AguiMessage.content from String to Object (backward-compatible at the API level)
• Includes 22 new test cases

Current blockers for merge:

1. CLA not yet signed by the PR author
2. PR may need rebase onto current main (open since May 2026)
3. Minor review feedback: document content handling is lossy (only preserves MIME type, not the actual document reference)

Once the CLA is signed and the PR is rebased, this should be merge-ready. @chickenbj Please prioritize reviewing this — there's clear community demand.

[oss-maintainer]

This feature is addressed by PR #1380 from @NoiAI, which adds multimodal input support (image, video, audio, document) to the AG-UI extension.

PR #1380 implements:

• AguiMessage.content: String → Object (backward compatible)
• AguiMessageConverter: InputContent[] → ContentBlock conversion
• Both URL and base64 data source types

CI is all green ✅. The PR has been open since May 2026 and is awaiting maintainer review.

Related: #1380 (PR), #1321 (expandable sessions + request context)