Repository navigation
Conversation
Composite fonts are read through their encoding CMap: Identity and embedded CMaps give CIDs, read through Adobe-Japan1 when the font is in that collection (the Mac's Hiragino without ToUnicode); the Unicode CMaps give Unicode; the Shift_JIS, EUC-JP, and JIS CMaps are decoded. Radicals read as ideographs, /ActualText replaces its glyphs' text, vertical writing reads by column, and Japanese lines join without spaces in documents.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A user's project found Japanese PDFs reading badly: a PDF made on a Mac gave 1 Japanese character in 360, and one in a standard Japanese encoding gave nothing.
Identity-H/Vand embedded CMaps give CIDs, read through Adobe-Japan1 when the font is in that collection. That's how the Mac writes Hiragino, with no ToUnicode map. The Unicode CMaps (UniJIS-UCS2-Hand the like, Chinese and Korean too) give Unicode. The Shift_JIS (90ms-RKSJ-H), EUC-JP and JIS CMaps are decoded. The tables are generated byscripts/gen-cjk-tables.pyfrom Adobe's mapping resources and WHATWG's JIS X 0208 index, both BSD 3-Clause; NOTICE credits them./ActualTextreplaces its glyphs' text. LibreOffice draws one placeholder glyph per character when it has no Japanese font.-Vfont and for the Mac's glyph-by-glyph columns.Tested on twelve PDFs from seven producers (Quartz with four fonts, Chrome, cupsfilter, PyMuPDF, reportlab in three encodings, and vertical text): every character reads as PyMuPDF reads it. On 70 to 100-page documents, a whole run takes 0.01 to 0.05 s at 8 to 13 MB, against PyMuPDF's extraction alone at 0.06 to 0.14 s and 186 MB.
tests/pdf_japanese.rscovers eight fixtures fromtests/fixtures/pdf-japanese/make.py, none of which embeds a font.The tables add about 75 KB to the binary; CI will show whether the size budget needs raising.