fix(chat): render inline base64 data-URI images from omni model responses#5495
Open
RonitSachdev wants to merge 1 commit into
Open
fix(chat): render inline base64 data-URI images from omni model responses#5495RonitSachdev wants to merge 1 commit into
RonitSachdev wants to merge 1 commit into
Conversation
Omni backends (e.g. Lemonade Server omni collections) embed generated images directly in the assistant text as . The markdown image pass rejected every data: URL, so users saw only the literal alt text "generated image" and never the image; the raw-HTML sanitizer pass would also have stripped the src attribute. Allow exactly the strict raster form data:image/(png|jpe?g|gif|webp);base64 with base64 payload chars only — the same shape chatRenderer.js already accepts for tool screenshots and display images. SVG (script-capable) and every other data: URL remain blocked, both in imageHtml and in the sanitizer, whose new exception is scoped to the src attribute of IMG elements matching the anchored pattern. Fixes odysseus-dev#5436
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Omni backends — e.g. Lemonade Server omni collections — run image generation server-side and embed the result directly in the assistant text as
(Lemonade docs). Odysseus's markdown image pass (imageHtmlinstatic/js/markdown.js) rejected everydata:URL viasafeLinkUrl, so the user saw only the literal alt text "generated image" and never the image (the raw-HTML sanitizer pass would also have stripped thesrc). This PR allows exactly the strict raster data-URI formdata:image/(png|jpe?g|gif|webp);base64,with base64 payload characters only — the same shapechatRenderer.jsalready accepts for tool screenshots and display images. SVG (script-capable) and every otherdata:URL remain blocked in both layers; the sanitizer exception is scoped to thesrcattribute ofIMGelements matching the anchored pattern, so nothing weakens forhref,srcset, or any other tag. Because messages re-render from stored markdown, previously "blank" chats retroactively show their images after this fix.Target branch
dev, notmain. All PRs land indev;mainis curated by the maintainer at each release. If your PR is onmainby accident, click "Edit" on this PR and change the base.Linked Issue
Fixes #5436
Type of Change
Checklist
devuvicorn app:app) and verified the change works end-to-end. Type-checks and unit tests are not enough.How to Test
python -m pytest tests/test_markdown_rendering_js.py tests/test_markdown_dom_xss_helpers.py -q(5 new rendering cases + a sanitizer scope guard; requiresnodeon PATH) andnode --check static/js/markdown.js., add it in Settings as an LLM endpoint, and send a chat message. Before this patch the reply renders as the bare text "generated image"; after it, the image itself renders (screenshots below, captured exactly this way against the running app).,, or)must still render as plain alt text with no<img>— covered by the new unit tests.safeDisplayImageSrc/safeToolScreenshotSrcvalidators instatic/js/chatRenderer.js;test_markdown_dom_xss_helpers.pyasserts the generaljavascript:/vbscript:/data:strip for all other URL-bearing attributes is unchanged.Visual / UI changes — REQUIRED if you touched anything that renders
<img>path (already styled by the app) work for inline data URIs. No emoji introduced.Screenshots / clips
Before (dev, reproduces #5436): the omni response renders as the literal text "generated image":
After (this PR): the generated image renders inline:
After, mobile viewport (390x844):
Checks run:
python -m pytest tests/test_markdown_rendering_js.py tests/test_markdown_dom_xss_helpers.py tests/test_markdown_table_row_js.py -q(26 passed),node --check static/js/markdown.js, and the manual end-to-end run above. Note for reviewers: the second part of #5436's "Additional Information" (Lemonade image models not appearing in the image-model picker) is a separate model-classification concern and is intentionally not addressed here, per the one-fix-per-PR guideline.