feat(engines): WeasyPrint as an optional engine for HTML, Markdown and e-mail to PDF

.github/workflows/ci.yml

@@ -154,11 +154,11 @@ jobs:
           path: build/.cache/zig-x86_64-windows-0.16.0
           key: zig-0.16.0-x86_64-windows
 
-      # rhwp, Pandoc, 7-Zip and ExifTool are small; with them the HWP/HWPX, archive and metadata engine tests run too
-      # (LibreOffice is skipped to save time).
-      - name: Fetch rhwp, Pandoc, 7-Zip and ExifTool for engine tests
+      # rhwp, Pandoc, 7-Zip, ExifTool and WeasyPrint are small; with them the HWP/HWPX, archive, metadata and WeasyPrint
+      # engine tests run too, WeasyPrint's against the pinned Windows build (LibreOffice is skipped to save time).
+      - name: Fetch rhwp, Pandoc, 7-Zip, ExifTool and WeasyPrint for engine tests
         shell: pwsh
-        run: ./build/fetch-engines.ps1 -Only rhwp,pandoc,7zip,exiftool
+        run: ./build/fetch-engines.ps1 -Only rhwp,pandoc,7zip,exiftool,weasyprint
 
       # The UI tests on their own first, as the release runs them (there they crashed with 0xC0000005): a crash
       # names the test that was running and leaves a dump in TestResults.

THIRD-PARTY-NOTICES.md

@@ -87,6 +87,7 @@ Downloaded from their official release pages when the user chooses to install th
 | Microsoft Visual C++ Redistributable (for Ghostscript and Upscayl, conda-forge `vc14_runtime` and `vcomp14`) | Microsoft Visual C++ Redistributable license | https://github.com/conda-forge/vc-feedstock |
 | FFmpeg 9.0.2 (gyan.dev "full_build-shared" Windows build; ffmpeg.exe, ffprobe.exe and their libraries, incl. x264, x265, libvpx, LAME, Opus, Vorbis, Theora, OpenCORE AMR) | GPL-3.0-or-later (the build is configured with `--enable-gpl --enable-version3`); FFmpeg itself LGPL-2.1-or-later | https://ffmpeg.org, build: https://www.gyan.dev/ffmpeg/builds/ (source: https://github.com/GyanD/codexffmpeg) |
 | calibre (ebook-convert) | GPL-3.0 | https://calibre-ebook.com |
+| WeasyPrint 70.0 (Windows "onedir" build, `weasyprint-windows-onedir.zip`: `weasyprint.exe` with its `_internal` folder, built with PyInstaller on MSYS2 UCRT64; it embeds Python 3.14, Pango, HarfBuzz, FreeType, Fontconfig, FriBidi, GLib, libthai, Pillow, fontTools, Pyphen, pydyf, tinycss2, cssselect2, Brotli, Zopfli, OpenSSL, zlib, libpng, Expat and the MinGW runtime) | BSD-3-Clause (WeasyPrint); PSF-2.0 (Python); LGPL-2.1-or-later (Pango, GLib, FriBidi, libthai); MIT (HarfBuzz, Fontconfig, fontTools, Brotli, Expat); FreeType License; HPND (Pillow); GPL/LGPL/MPL tri-licence (Pyphen); BSD-3-Clause (pydyf, tinycss2, cssselect2); Apache-2.0 (Zopfli, OpenSSL); GCC Runtime Library Exception (MinGW runtime); zlib and libpng licences | https://github.com/Kozea/WeasyPrint |
 | Tesseract OCR 5.4 (UB Mannheim Windows build, with Leptonica and the MinGW libraries it ships; their notices are in its `doc` folder) | Apache-2.0 (Tesseract); Leptonica BSD-2-Clause; others as listed in the build | https://github.com/tesseract-ocr/tesseract, build: https://github.com/UB-Mannheim/tesseract |
 | Tesseract language data (tessdata_fast 4.1.0), downloaded per language when used | Apache-2.0 | https://github.com/tesseract-ocr/tessdata_fast |
 | whisper.cpp (whisper-cli.exe Windows build) | MIT | https://github.com/ggml-org/whisper.cpp |

build/fetch-engines.ps1

@@ -22,6 +22,8 @@
                      installer with 7-Zip (never run); the language data is downloaded by the app when needed
       exiftool/      ExifTool (Perl's licence, separate program), the Windows build: exiftool(-k).exe renamed
                      exiftool.exe (the "-k" makes it wait for a key), with its exiftool_files/ beside it
+      weasyprint/    WeasyPrint (BSD-3-Clause, separate program), its Windows "onedir" build: weasyprint.exe with
+                     its _internal/ folder (Python, Pango, HarfBuzz, ...) and LICENSE, HTML -> PDF
 
   Every download is pinned to a version and verified with SHA-256 (src/Filee.Engines/Infrastructure/engines.json,
   shared with the app). Downloads are cached in build/.cache.
@@ -381,6 +383,23 @@ if ('exiftool' -in $selected) {
     if (-not (Test-Path (Join-Path $target 'exiftool.exe'))) { throw 'exiftool.exe not found in the archive.' }
 }
 
+if ('weasyprint' -in $selected) {
+    # As EngineInstaller.WeasyPrintFolder does: the folder with weasyprint.exe (onedir/weasyprint), with the licence
+    # from the top of the zip moved into it.
+    $tmp = Join-Path $cache ('w-' + [Guid]::NewGuid().ToString('N').Substring(0, 8))
+    Expand-Archive -Path (Get-Engine 'weasyprint') -DestinationPath $tmp
+    $exe = Get-ChildItem $tmp -Recurse -Filter 'weasyprint.exe' | Select-Object -First 1
+    if (-not $exe) { throw 'weasyprint.exe not found in the archive.' }
+    $target = Join-Path $Destination 'weasyprint'
+    Reset-Folder $target
+    Get-ChildItem $exe.DirectoryName -Force | Move-Item -Destination $target
+    foreach ($name in 'LICENSE', 'README.rst') {
+        $file = Join-Path $tmp $name
+        if (Test-Path $file) { Move-Item -LiteralPath $file -Destination $target -Force }
+    }
+    Remove-Item $tmp -Recurse -Force
+}
+
 if ('calibre' -in $selected) {
     $msi = Get-Engine 'calibre'
     $tmp = Join-Path $cache 'calibre-admin'

docs/ARCHITECTURE.md

@@ -25,6 +25,7 @@
             │   · E-mail · Signed files · Data · E-books · CAD · Fonts · Subtitles · 3D models   │
             │ Bundled: rhwp (HWP→PDF, HWP↔HWPX) · 7-Zip (archives) · Unhwp · Markdig             │
             │ Optional: FFmpeg · LibreOffice + H2Orestart · calibre · Ghostscript · Pandoc       │
+            │   · WeasyPrint                                                                     │
             └────────────────────────────────────────────────────────────────────────────────────┘
 ```
 

docs/ENGINES.md

@@ -40,6 +40,7 @@ Filee chooses engines automatically. Settings → *Engines* shows their status,
 | Apple Vision | part of macOS (Mac only) | **text recognition (OCR) on a Mac**, the same step as Tesseract without a download; Tesseract reads the languages Vision doesn't (see "Apple Vision on macOS") |
 | FFmpeg | **optional download** (~100 MB, 272 MB on disk) | **all video and audio**: video ↔ video (H.264, HEVC, AV1), video → animated GIF / APNG, a still frame or every frame, audio and subtitle extraction, audio ↔ audio (also APE, WavPack, TTA, DSD, DTS, ALAC, …), GIF, APNG and photos → MP4/WEBM/MOV, Live / Motion Photos → their video, a clip of a video → an iPhone Live Photo or Android Motion Photo (experimental), images → HEIC, clips or audio files joined into one, pictures → a slideshow MP4, a video split into parts every so often or into equal parts (copied, not encoded), copies without metadata (see "Media tools") |
 | Calibre | **optional download** (~216 MB, 660 MB on disk) | rare e-book formats: LIT, LRF, CHM, PDB, PML, RB, SNB, TCR, OEB → EPUB; EPUB → MOBI, AZW3, LIT, LRF, PDB, PML, RB, SNB, TCR; e-books → PDF (ranked above the HWPX writer, see below) |
+| WeasyPrint | **optional download** (Windows only for now; ~32 MB, 65 MB on disk) | **HTML, Markdown, EML, MSG, MHTML → PDF** laid out as a browser prints them (headings, tables, bold, Arabic and Indic shaping), instead of the built-in route; nothing changes without it (see below) |
 | Whisper | **optional download** (Windows only; ~157 MB with model, ~160 MB on disk) | **Speech-to-text transcription**: video and audio → SRT subtitles or plain TXT in ~100 languages using OpenAI Whisper (whisper.cpp); runs locally and offline; needs FFmpeg |
 | ExifTool | **optional download** (~11 MB on Windows, ~8 MB elsewhere; 30 MB on disk) | **the Metadata tool**: reads every metadata field of pictures, PDFs, video and sound, and writes changes into a copy in the file's own format without re-encoding it (JPG, PNG, TIFF, WEBP, HEIC, AVIF, JPEG XL, GIF, PSD, camera RAW, PDF, MP4, MOV, M4A, 3GP, …; not MP3, FLAC, WAV, MKV, AVI or BMP, which it shows read-only). Without it the Metadata tool shows nothing and offers it |
 
@@ -56,7 +57,8 @@ XLSX shared strings and sheets, HWPX sections, OFD pages and templates, EPUB cha
 RTF character references) for the Hebrew, Arabic, Syriac, Thaana, N'Ko, Indic, Sinhala, Thai, Lao, Tibetan, Myanmar,
 Khmer, Mongolian and other South / Southeast Asian blocks. For such a file `JobQueue` plans again without unshaped
 edges: DOCX, TXT, Markdown, HTML, XLSX, PPTX and HWPX → PDF (and → images through PDF) go through LibreOffice even
-when Settings → *Engines* doesn't choose it for documents. Without LibreOffice the file fails with "This script needs
+when Settings → *Engines* doesn't choose it for documents; HTML, Markdown and e-mails go through WeasyPrint instead
+when it is installed (it shapes text, see *WeasyPrint*). Without LibreOffice the file fails with "This script needs
 LibreOffice to convert correctly" and the script's name instead of becoming a broken PDF. The same scan also finds
 text rhwp has no widths for: its width tables cover only ASCII and Latin-1, general punctuation, math, the CJK forms
 and Hangul (rhwp#8143), so a letter of any other script — Polish, Turkish, Vietnamese, Greek, Cyrillic, ... — or a
@@ -139,6 +141,7 @@ the runtime next to every program that needs it and does not ship it (`Infrastru
 | Ghostscript (download) | `vcruntime` component copied next to `gswin64c.exe` (`EngineInstaller`, `fetch-engines.ps1`) |
 | LibreOffice (download) | its own copy, which the administrative install leaves in `System64`, copied into `program/` (`EngineInstaller`, `fetch-engines.ps1`; `LibreOfficeConverter` also repairs a copy installed by an earlier Filee) |
 | calibre, Java (downloads) | ship their own next to their programs (`app/bin`, `bin`) |
+| WeasyPrint (download) | ships its own (the MinGW / UCRT runtime and `VCRUNTIME140.dll` of its Python) in `_internal` |
 | 7-Zip, FFmpeg (GyanD), Tesseract (UB Mannheim, MinGW), Pandoc, PDFium, SkiaSharp / HarfBuzz, Magick.Native, the Explorer menu DLL, .NET | import neither (static runtime or MinGW / GHC builds) |
 
 The installer and the portable zip are made from the same publish folder, so both have the DLLs. Filee installed in
@@ -1519,6 +1522,43 @@ never switches engines by itself.
   (`CALIBRE_CONFIG_DIRECTORY`, ...), so a calibre the user installed is never read or changed; messages are
   English (`CALIBRE_OVERRIDE_LANG`) and progress comes from its "34% ..." lines.
 
+## WeasyPrint
+
+WeasyPrint (BSD-3-Clause, `Text/WeasyPrintConverter.cs`) is an optional download that makes PDFs of HTML the way a
+browser prints it, with Pango and HarfBuzz: headings, tables, bold, links, and Arabic, Hebrew or Indic text shaped and
+in the right order. The built-in route (HWPX writer → rhwp) keeps structure only and draws unshaped text (#1403).
+
+- **Download**: Windows only for now, WeasyPrint's own "onedir" build (`weasyprint-windows-onedir.zip`, pinned in
+  `engines.json`): `weasyprint.exe` with an `_internal` folder holding Python, Pango, HarfBuzz, Fontconfig, ... The
+  folder with `weasyprint.exe` becomes `engines/weasyprint`, with WeasyPrint's `LICENSE` moved into it. It is found
+  only as Filee's download or a copy the user picked in Settings → *Engines*, never on the PATH. Linux and macOS
+  follow in a separate change once WeasyPrint publishes a self-contained build for them; until then they keep the
+  routes below.
+- **Routes**: one edge, HTML → PDF, cost 1, used only in routes that start from HTML, Markdown, EML, MSG or MHTML
+  (`ConversionEdge.Sources`). So a page goes straight to WeasyPrint, Markdown through Markdig's page (Markdown → HTML
+  → PDF, 9 + 10 at the end of the engine order) and an e-mail or a saved web page through its page (19 + 10), all
+  cheaper than the built-in route (31) and LibreOffice (37). A Word document, an e-book or a PDF never goes through a
+  page to reach it. Without WeasyPrint installed the edge isn't planned and every route stays as it was. → PNG, JPG,
+  ... of such a file go through WeasyPrint's PDF and PDFium. A source with text that needs shaping takes it too: the
+  edge isn't `Unshaped`.
+- **Locked down**: WeasyPrint gets a copy of the page (`WeasyPrintConverter.PreparePage`, the HTML rule of
+  `LinkedContent`): pictures inside the document's own folder (or the Markdown file's or e-mail's, for a page made
+  from it; `DocumentFiles.cs`) become `data:` URIs, SVG included; every other picture, frame, object, embed, `<base>`,
+  `<link>` style sheet, `@import` and CSS `url()` address is taken out, except references to the page's own elements
+  (`#id`: an inline SVG's `<use href="#icon">`, a gradient's `xlink:href`, `fill: url(#g)`), which WeasyPrint resolves
+  within the page. It then runs with `--allowed-protocols data`,
+  so whatever the copy still names (a relative or absolute path, a `file:`, `http:` or `https:` address, an
+  attachment link, a CSS `image-set()` string the copy didn't recognise) is refused by WeasyPrint itself: no local
+  file outside the document is read and no address is contacted. Removing first and refusing second means a
+  reference the HTML rewrite misses still loads nothing, and one WeasyPrint would refuse anyway doesn't leave an error
+  per picture in its output. Tests: `WeasyPrintTests` (the page prepared without WeasyPrint, and the PDF with it).
+- **Options**: `--presentational-hints` (an e-mail's `width`, `bgcolor`, ...) and `--verbose`, whose "Step 5 -
+  Creating layout - Page 12" lines drive the progress bar and the stall watch. Page size and margins come from the
+  page's CSS, else A4 (WeasyPrint's default). Pictures embedded in one page are limited to 256 MB in all, like for
+  LibreOffice.
+- **Time**: WeasyPrint may run as long as it keeps printing lines, but is stopped after 5 minutes without one
+  (`error.engine_stalled`) and after 30 minutes in all. Cancelling stops it at once. Two run at the same time.
+
 ## HWP ↔ HWPX
 
 rhwp converts between the two 한글 formats without Hancom Office (`export-hwpx` and `convert`). Anything → HWP goes
@@ -1542,7 +1582,8 @@ per MB of unpacked `Contents/section*.xml` (an HWP counts three times its file s
 60 minutes, after which a rhwp that hangs is stopped. The progress estimate assumes 15 minutes per MB. Cancelling
 stops rhwp at once. A time-out fails with "rhwp couldn't finish this long document in time" (with calibre installed,
 it suggests ranking calibre above the HWPX writer for e-books). The other engines keep fixed limits: LibreOffice 3
-minutes, Ghostscript 5, Pandoc 2 and calibre 10 (it made the 640-page novel's PDF in a minute).
+minutes, Ghostscript 5, Pandoc 2 and calibre 10 (it made the 640-page novel's PDF in a minute); WeasyPrint 5 minutes
+without output, 30 in all.
 
 ## Archives
 

src/Filee.App/Assets/i18n/ar.json

@@ -270,6 +270,7 @@
   "engine.tesseract.description": "يتعرّف على نص الصور والمستندات الممسوحة (OCR): PDF قابل للبحث وDOCX وTXT.",
   "engine.vision.description": "يتعرّف على نص الصور والمستندات الممسوحة بأداة التعرّف على النص المدمجة في macOS (OCR): PDF قابل للبحث وDOCX وTXT. لا حاجة إلى أي تنزيل.",
   "engine.calibre.description": "صيغ الكتب الإلكترونية النادرة (LIT وLRF وCHM وPDB وRB…) وإخراج Kindle ‏(MOBI وAZW3).\nعند وضعه فوق محرك كتابة HWPX، يُنشئ أيضًا PDF من الكتب الإلكترونية (بسرعة أكبر بكثير للكتب الطويلة).",
+  "engine.weasyprint.description": "ينشئ ملفات PDF من صفحات HTML وMarkdown والبريد الإلكتروني كما يطبعها المتصفح: تظهر العناوين والجداول والخط العريض والنص العربي أو الهندي بشكل صحيح.\nلا يحمّل سوى الصور الموجودة بجانب المستند، ولا شيء من الإنترنت.",
   "engine.libreoffice.description": "قراءة الصيغ القديمة والأقل شيوعًا: DOC وPPT وPages وKeynote وNumbers وWordPerfect وClarisWorks وPublisher وVisio…\nوالحفظ بصيغ DOC وXLS وPPT وODT وODP وقوالب OpenDocument وEMF.",
   "engine.translator.description": "يقوم بترجمة المستندات والكتب الإلكترونية محلياً باستخدام نماذج Ollama دون اتصال بالإنترنت.\nيترجم أيضاً ملفات الترجمة بصيغتي SRT وWebVTT: يُرسَل نص المقاطع فقط إلى النموذج، وتبقى أرقام المقاطع وتوقيتاتها كما هي.",
   "engine.exiftool.description": "يقرأ كل حقول البيانات الوصفية للصور وملفات PDF والفيديو والصوت، ويعدّلها في معظم أنواع الملفات دون المساس بمحتوى الملف.\nتستخدمه أداة البيانات الوصفية؛ ومن دونه تُعرض الحقول للقراءة فقط.",
@@ -1317,6 +1318,8 @@
   "engines.package.whisper.description": "تحويل الكلام في الفيديو والصوت إلى ملفات ترجمة SRT. يعمل بدون اتصال بالإنترنت.",
   "engines.package.calibre.name": "Calibre (Kindle وصيغ الكتب النادرة)",
   "engines.package.calibre.description": "يحفظ MOBI وAZW3 لأجهزة Kindle، ويحوّل صيغ الكتب الإلكترونية النادرة (LIT وLRF وPDB وCHM…).\nتُقرأ EPUB وMOBI وAZW3 وFB2 والقصص المصورة بدونه.",
+  "engines.package.weasyprint.name": "WeasyPrint (من HTML وMarkdown والبريد إلى PDF)",
+  "engines.package.weasyprint.description": "ينشئ ملفات PDF أفضل من صفحات HTML وMarkdown والبريد الإلكتروني (EML وMSG وMHTML): العناوين والجداول والخط العريض والكتابات من اليمين إلى اليسار أو الهندية كما يطبعها المتصفح.\nوبدونه تصبح أيضًا ملفات PDF عبر المسار المدمج.",
   "engines.package.size": "التنزيل {0} · على القرص {1}",
   "engines.package.system_required": "استخدم نسخة مثبّتة على هذا الحاسوب.",
   "engines.guidance.macos": "ثبّت {0} لهذا الـ Mac، ثم اختر «استخدام نسختي…» وحدّد البرنامج أو مجلد ‎.app الخاص به.",