Skip to content

fix(chat): render inline base64 data-URI images from omni model responses#5495

Open
RonitSachdev wants to merge 1 commit into
odysseus-dev:devfrom
RonitSachdev:fix/render-omni-inline-data-images
Open

fix(chat): render inline base64 data-URI images from omni model responses#5495
RonitSachdev wants to merge 1 commit into
odysseus-dev:devfrom
RonitSachdev:fix/render-omni-inline-data-images

Conversation

@RonitSachdev

@RonitSachdev RonitSachdev commented Jul 13, 2026

Copy link
Copy Markdown

Summary

Omni backends — e.g. Lemonade Server omni collections — run image generation server-side and embed the result directly in the assistant text as ![generated image](data:image/png;base64,...) (Lemonade docs). Odysseus's markdown image pass (imageHtml in static/js/markdown.js) rejected every data: URL via safeLinkUrl, so the user saw only the literal alt text "generated image" and never the image (the raw-HTML sanitizer pass would also have stripped the src). This PR allows exactly the strict raster data-URI form data:image/(png|jpe?g|gif|webp);base64, with base64 payload characters only — the same shape chatRenderer.js already accepts for tool screenshots and display images. SVG (script-capable) and every other data: URL remain blocked in both layers; the sanitizer exception is scoped to the src attribute of IMG elements matching the anchored pattern, so nothing weakens for href, srcset, or any other tag. Because messages re-render from stored markdown, previously "blank" chats retroactively show their images after this fix.

Target branch

  • This PR targets dev, not main. All PRs land in dev; main is curated by the maintainer at each release. If your PR is on main by accident, click "Edit" on this PR and change the base.

Linked Issue

Fixes #5436

Type of Change

  • Bug fix (non-breaking — fixes a confirmed issue)

Checklist

  • I searched open issues and open PRs — this is not a duplicate.
  • This PR targets dev
  • My changes are limited to the scope described above — no unrelated refactors or whitespace changes mixed in.
  • I actually ran the app (uvicorn app:app) and verified the change works end-to-end. Type-checks and unit tests are not enough.

How to Test

  1. python -m pytest tests/test_markdown_rendering_js.py tests/test_markdown_dom_xss_helpers.py -q (5 new rendering cases + a sanitizer scope guard; requires node on PATH) and node --check static/js/markdown.js.
  2. End-to-end without Lemonade hardware: run any OpenAI-compatible mock that streams a chat delta containing ![generated image](data:image/png;base64,<tiny png>), add it in Settings as an LLM endpoint, and send a chat message. Before this patch the reply renders as the bare text "generated image"; after it, the image itself renders (screenshots below, captured exactly this way against the running app).
  3. Negative cases: a message containing ![x](data:image/svg+xml;base64,...), ![x](data:text/html;base64,...), or ![x](javascript:alert(1)) must still render as plain alt text with no <img> — covered by the new unit tests.
  4. Security posture: the allowed pattern is anchored, raster-only, base64-payload-only (no quotes/angle brackets/nested URLs), mirroring the existing safeDisplayImageSrc/safeToolScreenshotSrc validators in static/js/chatRenderer.js; test_markdown_dom_xss_helpers.py asserts the general javascript:/vbscript:/data: strip for all other URL-bearing attributes is unchanged.

Visual / UI changes — REQUIRED if you touched anything that renders

  • Screenshot or short clip of the change in the running app, attached below. Mobile screenshot too if the change affects mobile.
  • Style match: no CSS, HTML, classes, colors, fonts, or components were added or changed — the fix makes the existing markdown <img> path (already styled by the app) work for inline data URIs. No emoji introduced.
  • No new component patterns.
  • I am not an LLM agent submitting a bulk PR. — Honest disclosure instead of a checked box: this fix was developed in a Claude Code session driven interactively by me. It is not bulk-generated: it addresses the existing triaged issue Image generation failing when using a omni model with Lemonade Server #5436 only, was verified end-to-end against the running app (screenshots below are from that run), and touches no visual styling.

Screenshots / clips

Before (dev, reproduces #5436): the omni response renders as the literal text "generated image":

before

After (this PR): the generated image renders inline:

after

After, mobile viewport (390x844):

after mobile


Checks run: python -m pytest tests/test_markdown_rendering_js.py tests/test_markdown_dom_xss_helpers.py tests/test_markdown_table_row_js.py -q (26 passed), node --check static/js/markdown.js, and the manual end-to-end run above. Note for reviewers: the second part of #5436's "Additional Information" (Lemonade image models not appearing in the image-model picker) is a separate model-classification concern and is intentionally not addressed here, per the one-fix-per-PR guideline.

Omni backends (e.g. Lemonade Server omni collections) embed generated
images directly in the assistant text as
![generated image](data:image/png;base64,...). The markdown image pass
rejected every data: URL, so users saw only the literal alt text
"generated image" and never the image; the raw-HTML sanitizer pass
would also have stripped the src attribute.

Allow exactly the strict raster form
data:image/(png|jpe?g|gif|webp);base64 with base64 payload chars only —
the same shape chatRenderer.js already accepts for tool screenshots and
display images. SVG (script-capable) and every other data: URL remain
blocked, both in imageHtml and in the sanitizer, whose new exception is
scoped to the src attribute of IMG elements matching the anchored
pattern.

Fixes odysseus-dev#5436
@github-actions github-actions Bot added ready for review Description complete — ready for maintainer review needs work PR description incomplete — please update before review and removed ready for review Description complete — ready for maintainer review needs work PR description incomplete — please update before review labels Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for review Description complete — ready for maintainer review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Image generation failing when using a omni model with Lemonade Server

1 participant