Skip to content

GET /v1/files/:id/content serves text files without charset → non-ASCII content renders as mojibake in Chrome #305

Description

@leixf-lab

Summary

Text files containing non-ASCII characters (Chinese, Japanese, emoji, etc.) render as mojibake when opened in the browser via the web UI Files page. The stored bytes are valid UTF-8; the problem is the response headers.

Root cause

getFileContent in src/api/routes/surface.ts sends the stored mimetype verbatim:

res.writeHead(200, {
  "content-type": opened.mimetype || "application/octet-stream",
  ...
});

For a markdown file this yields content-type: text/markdown with no charset parameter. The web-ui proxy (plugins/web-ui/server/index.ts, /api/files/:id/content) forwards this header unchanged and adds x-content-type-options: nosniff, so the browser cannot fall back to sniffing the encoding. Chrome (which no longer has a manual encoding override) then decodes the UTF-8 bytes as windows-1252 → mojibake. Any non-ASCII text file is affected on every surface that proxies this endpoint.

Repro

  1. In any conversation, have the agent create a markdown file containing Chinese text and share it.
  2. Open the file from the web UI Files page in Chrome.
  3. Observe mojibake. curl -sD - on /api/files/<id>/content shows content-type: text/markdown without charset; the body bytes are valid UTF-8.

Suggested fix

Append ; charset=utf-8 for text-like mimetypes that don't already declare one, at the core layer so all proxying surfaces inherit it:

const mimetype = opened.mimetype || "application/octet-stream";
res.writeHead(200, {
  "content-type":
    /^(text\/|application\/(json|xml))/.test(mimetype) && !/charset=/i.test(mimetype)
      ? `${mimetype}; charset=utf-8`
      : mimetype,
  ...
});

Verified locally (repo @ 0f0e0adccce2): with this change the endpoint returns content-type: text/markdown; charset=utf-8 and Chrome renders CJK content correctly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions