Skip to content

fix: decode UTF-16 markdown with BOM - #991

Open
tianrking wants to merge 1 commit into
charmbracelet:masterfrom
tianrking:agent/decode-utf16-markdown
Open

fix: decode UTF-16 markdown with BOM#991
tianrking wants to merge 1 commit into
charmbracelet:masterfrom
tianrking:agent/decode-utf16-markdown

Conversation

@tianrking

Copy link
Copy Markdown

Summary

  • decode UTF-16 LE and BE Markdown inputs that include a byte-order mark
  • use the same decoding path for the CLI, direct TUI documents, and locally loaded stash documents
  • retain existing bytes for UTF-8 input and malformed odd-length UTF-16 payloads

Root cause

Glow passed file bytes directly to the Markdown renderer. UTF-16 encoded Markdown therefore contained NUL bytes rather than usable UTF-8 text, resulting in mostly blank output.

Validation

  • go test ./...
  • go build ./...
  • git diff --check

Fixes #513

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Problems parsing UTF16 files

1 participant