Skip to content

Refuse paths zli cannot read on Windows (#1116) - #1116

Open
aidangrimshaw751-netizen wants to merge 8 commits into
meta:devfrom
aidangrimshaw751-netizen:export-D123741307
Open

aidangrimshaw751-netizen wants to merge 8 commits into
meta:devfrom
aidangrimshaw751-netizen:export-D123741307

Conversation

@aidangrimshaw751-netizen

@aidangrimshaw751-netizen aidangrimshaw751-netizen commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary:

X-link: #1116

On Windows, zli reads its arguments in the ANSI code page. A path with characters outside that code page reaches zli garbled, or best-fit mapped to a different file. The desktop app now checks the input path and the temp folder before a run, and fails with an error that names the characters.

  • ansi-path.cjs reads the code page (ACP) from the registry, running reg.exe by its full path under %SystemRoot%\System32. A failed read is retried on the next run.
  • iconv-lite decides which characters the code page can encode. It ships Windows' own tables; Node's TextDecoder CJK tables vary with the ICU version. UTF-8 and unknown code pages are never refused.
  • checkInputPath runs once per run, before any job starts, so a path zli cannot read fails the run once rather than failing every job. measureOpenZL also checks before it creates or spawns anything.
  • Other platforms are not checked.
  • iconv-lite 0.6.3, already in the lockfile, becomes a dependency. after-pack.cjs ships it and safer-buffer with the packaged app, and ansi-path.cjs goes beside the worker in extraResources.

Reviewed By: jlee303

Differential Revision: D123741307

@meta-cla meta-cla Bot added the cla signed label Oct 14, 2026
@meta-codesync

meta-codesync Bot commented Oct 14, 2026

Copy link
Copy Markdown

@aidangrimshaw751-netizen has exported this pull request. If you are a Meta employee, you can view the originating Diff in D123741307.

aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 14, 2026
Summary:
Pull Request resolved: meta#1116

On Windows `zli` reads its arguments in the ANSI code page, so a path with a character outside it reaches `zli` mangled, or best-fit mapped to a different file name. The desktop app now checks first, instead of passing `zli` a bad path.

- `electron/ansi-path.cjs` reads this PC's ANSI code page from the registry once, and asks that code page which characters it can encode (by decoding its byte sequences; Windows-1252's 0x80-0x9F block is a table, because Node's `TextDecoder` treats that label as Latin-1). A code page it does not know, or UTF-8, is never refused. It runs `reg.exe` by its full path under `%SystemRoot%\System32`, since Windows resolves a bare name through the current directory before `PATH`.
- An input path the code page cannot represent is copied into the run's scratch directory, and `zli` reads the copy.
- A temp folder the code page cannot represent is refused before anything is created or spawned, with a message naming the characters and the code page. It matters because the scratch directory, the CSV and the trained candidates all live under it, so a user profile name outside the code page would otherwise fail even with an ASCII input path.
- Windows only: other platforms do no check.

Differential Revision: D123741307
@meta-codesync meta-codesync Bot changed the title Handle paths zli cannot read on Windows Handle paths zli cannot read on Windows (#1116) Oct 14, 2026
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 14, 2026
Summary:
Pull Request resolved: meta#1116

On Windows `zli` reads its arguments in the ANSI code page, so a path with a character outside it reaches `zli` mangled, or best-fit mapped to a different file name. The desktop app now checks first, instead of passing `zli` a bad path.

- `electron/ansi-path.cjs` reads this PC's ANSI code page from the registry once, and asks that code page which characters it can encode (by decoding its byte sequences; Windows-1252's 0x80-0x9F block is a table, because Node's `TextDecoder` treats that label as Latin-1). A code page it does not know, or UTF-8, is never refused. It runs `reg.exe` by its full path under `%SystemRoot%\System32`, since Windows resolves a bare name through the current directory before `PATH`.
- An input path the code page cannot represent is copied into the run's scratch directory, and `zli` reads the copy.
- A temp folder the code page cannot represent is refused before anything is created or spawned, with a message naming the characters and the code page. It matters because the scratch directory, the CSV and the trained candidates all live under it, so a user profile name outside the code page would otherwise fail even with an ASCII input path.
- Windows only: other platforms do no check.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 14, 2026
Summary:
Pull Request resolved: meta#1116

On Windows `zli` reads its arguments in the ANSI code page, so a path with a character outside it reaches `zli` mangled, or best-fit mapped to a different file name. The desktop app now checks first, instead of passing `zli` a bad path.

- `electron/ansi-path.cjs` reads this PC's ANSI code page from the registry (a failed read is not kept, so the next run tries again), and asks that code page which characters it can encode (by decoding its byte sequences; Windows-1252's 0x80-0x9F block is a table, because Node's `TextDecoder` treats that label as Latin-1). A code page it does not know or this runtime cannot decode, or UTF-8, is never refused. It runs `reg.exe` by its full path under `%SystemRoot%\System32`, since Windows resolves a bare name through the current directory before `PATH`.
- An input path the code page cannot represent is copied into the run's scratch directory, and `zli` reads the copy. When the code page could not be read, is unknown, or cannot be decoded, any input path past ASCII is copied: copying is safe either way, unlike refusing.
- A temp folder the code page cannot represent is refused before anything is created or spawned, with a message naming the characters and the code page. It matters because the scratch directory, the CSV and the trained candidates all live under it, so a user profile name outside the code page would otherwise fail even with an ASCII input path.
- Windows only: other platforms do no check.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 14, 2026
Summary:
Pull Request resolved: meta#1116

On Windows `zli` reads its arguments in the ANSI code page, so a path with a character outside it reaches `zli` mangled, or best-fit mapped to a different file name. The desktop app now checks first, instead of passing `zli` a bad path.

- `electron/ansi-path.cjs` reads this PC's ANSI code page from the registry (a failed read is not kept, so the next run tries again), and asks that code page which characters it can encode (by decoding its byte sequences; Windows-1252's 0x80-0x9F block is a table, because Node's `TextDecoder` treats that label as Latin-1). A code page it does not know or this runtime cannot decode, or UTF-8, is never refused. It runs `reg.exe` by its full path under `%SystemRoot%\System32`, since Windows resolves a bare name through the current directory before `PATH`.
- An input path the code page cannot represent is copied into the run's scratch directory, and `zli` reads the copy. When the code page could not be read, is unknown, or cannot be decoded, any input path past ASCII is copied: copying is safe either way, unlike refusing.
- A temp folder the code page cannot represent is refused before anything is created or spawned, with a message naming the characters and the code page. It matters because the scratch directory, the CSV and the trained candidates all live under it, so a user profile name outside the code page would otherwise fail even with an ASCII input path.
- Windows only: other platforms do no check.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 14, 2026
Summary:
Pull Request resolved: meta#1116

On Windows `zli` reads its arguments in the ANSI code page, so a path with a character outside it reaches `zli` mangled, or best-fit mapped to a different file name. The desktop app now checks first, instead of passing `zli` a bad path.

- `electron/ansi-path.cjs` reads this PC's ANSI code page from the registry (a failed read is not kept, so the next run tries again), and asks that code page which characters it can encode (by decoding its byte sequences; Windows-1252's 0x80-0x9F block is a table, because Node's `TextDecoder` treats that label as Latin-1). A code page it does not know or this runtime cannot decode, or UTF-8, is never refused. It runs `reg.exe` by its full path under `%SystemRoot%\System32`, since Windows resolves a bare name through the current directory before `PATH`.
- An input path the code page cannot represent is copied into the run's scratch directory, and `zli` reads the copy. When the code page could not be read, is unknown, or cannot be decoded, any input path past ASCII is copied: copying is safe either way, unlike refusing.
- A temp folder the code page cannot represent is refused before anything is created or spawned, with a message naming the characters and the code page. It matters because the scratch directory, the CSV and the trained candidates all live under it, so a user profile name outside the code page would otherwise fail even with an ASCII input path.
- Windows only: other platforms do no check.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 14, 2026
Summary:
Pull Request resolved: meta#1116

On Windows `zli` reads its arguments in the ANSI code page, so a path with a character outside it reaches `zli` mangled, or best-fit mapped to a different file name. The desktop app now checks first, instead of passing `zli` a bad path.

- `electron/ansi-path.cjs` reads this PC's ANSI code page from the registry (a failed read is not kept, so the next run tries again), and asks that code page which characters it can encode (by decoding its byte sequences; Windows-1252's 0x80-0x9F block is a table, because Node's `TextDecoder` treats that label as Latin-1). A code page it does not know or this runtime cannot decode, or UTF-8, is never refused. It runs `reg.exe` by its full path under `%SystemRoot%\System32`, since Windows resolves a bare name through the current directory before `PATH`.
- An input path the code page cannot represent is copied into the run's scratch directory, and `zli` reads the copy. When the code page could not be read, is unknown, or cannot be decoded, any input path past ASCII is copied: copying is safe either way, unlike refusing.
- A temp folder the code page cannot represent is refused before anything is created or spawned, with a message naming the characters and the code page. It matters because the scratch directory, the CSV and the trained candidates all live under it, so a user profile name outside the code page would otherwise fail even with an ASCII input path.
- Windows only: other platforms do no check.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 14, 2026
Summary:
Pull Request resolved: meta#1116

On Windows `zli` reads its arguments in the ANSI code page, so a path with a character outside it reaches `zli` mangled, or best-fit mapped to a different file name. The desktop app now checks first, instead of passing `zli` a bad path.

- `electron/ansi-path.cjs` reads this PC's ANSI code page from the registry (a failed read is not kept, so the next run tries again), and asks that code page which characters it can encode (by decoding its byte sequences; Windows-1252's 0x80-0x9F block is a table, because Node's `TextDecoder` treats that label as Latin-1). A code page it does not know or this runtime cannot decode, or UTF-8, is never refused. It runs `reg.exe` by its full path under `%SystemRoot%\System32`, since Windows resolves a bare name through the current directory before `PATH`.
- An input path the code page cannot represent is copied into the run's scratch directory, and `zli` reads the copy. When the code page could not be read, is unknown, or cannot be decoded, any input path past ASCII is copied: copying is safe either way, unlike refusing.
- A temp folder the code page cannot represent is refused before anything is created or spawned, with a message naming the characters and the code page. It matters because the scratch directory, the CSV and the trained candidates all live under it, so a user profile name outside the code page would otherwise fail even with an ASCII input path.
- Windows only: other platforms do no check.

Differential Revision: D123741307
aidangrimshaw751-netizen pushed a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 15, 2026
Summary:
Pull Request resolved: meta#1116

On Windows `zli` reads its arguments in the ANSI code page, so a path with a character outside it reaches `zli` mangled, or best-fit mapped to a different file name. The desktop app now checks first, instead of passing `zli` a bad path.

- `electron/ansi-path.cjs` reads this PC's ANSI code page from the registry (a failed read is not kept, so the next run tries again), and asks that code page which characters it can encode (by decoding its byte sequences; Windows-1252's 0x80-0x9F block is a table, because Node's `TextDecoder` treats that label as Latin-1). A code page it does not know or this runtime cannot decode, or UTF-8, is never refused. It runs `reg.exe` by its full path under `%SystemRoot%\System32`, since Windows resolves a bare name through the current directory before `PATH`.
- An input path the code page cannot represent is copied into the run's scratch directory, and `zli` reads the copy. When the code page could not be read, is unknown, or cannot be decoded, any input path past ASCII is copied: copying is safe either way, unlike refusing.
- A temp folder the code page cannot represent is refused before anything is created or spawned, with a message naming the characters and the code page. It matters because the scratch directory, the CSV and the trained candidates all live under it, so a user profile name outside the code page would otherwise fail even with an ASCII input path.
- Windows only: other platforms do no check.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 15, 2026
Summary:
Pull Request resolved: meta#1116

On Windows `zli` reads its arguments in the ANSI code page, so a path with a character outside it reaches `zli` mangled, or best-fit mapped to a different file name. The desktop app now checks first, instead of passing `zli` a bad path.

- `electron/ansi-path.cjs` reads this PC's ANSI code page from the registry (a failed read is not kept, so the next run tries again), and asks that code page which characters it can encode (by decoding its byte sequences; Windows-1252's 0x80-0x9F block is a table, because Node's `TextDecoder` treats that label as Latin-1). A code page it does not know or this runtime cannot decode, or UTF-8, is never refused. It runs `reg.exe` by its full path under `%SystemRoot%\System32`, since Windows resolves a bare name through the current directory before `PATH`.
- An input path the code page cannot represent is copied into the run's scratch directory, and `zli` reads the copy. When the code page could not be read, is unknown, or cannot be decoded, any input path past ASCII is copied: copying is safe either way, unlike refusing.
- A temp folder the code page cannot represent is refused before anything is created or spawned, with a message naming the characters and the code page. It matters because the scratch directory, the CSV and the trained candidates all live under it, so a user profile name outside the code page would otherwise fail even with an ASCII input path.
- Windows only: other platforms do no check.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 15, 2026
Summary:
Pull Request resolved: meta#1116

On Windows `zli` reads its arguments in the ANSI code page, so a path with a character outside it reaches `zli` mangled, or best-fit mapped to a different file name. The desktop app now checks first, instead of passing `zli` a bad path.

- `electron/ansi-path.cjs` reads this PC's ANSI code page from the registry (a failed read is not kept, so the next run tries again), and asks that code page which characters it can encode (by decoding its byte sequences; Windows-1252's 0x80-0x9F block is a table, because Node's `TextDecoder` treats that label as Latin-1). A code page it does not know or this runtime cannot decode, or UTF-8, is never refused. It runs `reg.exe` by its full path under `%SystemRoot%\System32`, since Windows resolves a bare name through the current directory before `PATH`.
- An input path the code page cannot represent is copied into the run's scratch directory, and `zli` reads the copy. When the code page could not be read, is unknown, or cannot be decoded, any input path past ASCII is copied: copying is safe either way, unlike refusing.
- A temp folder the code page cannot represent is refused before anything is created or spawned, with a message naming the characters and the code page. It matters because the scratch directory, the CSV and the trained candidates all live under it, so a user profile name outside the code page would otherwise fail even with an ASCII input path.
- Windows only: other platforms do no check.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 15, 2026
Summary:
Pull Request resolved: meta#1116

On Windows `zli` reads its arguments in the ANSI code page, so a path with a character outside it reaches `zli` mangled, or best-fit mapped to a different file name. The desktop app now checks first, instead of passing `zli` a bad path.

- `electron/ansi-path.cjs` reads this PC's ANSI code page from the registry (a failed read is not kept, so the next run tries again), and asks that code page which characters it can encode (by decoding its byte sequences; Windows-1252's 0x80-0x9F block is a table, because Node's `TextDecoder` treats that label as Latin-1). A code page it does not know or this runtime cannot decode, or UTF-8, is never refused. It runs `reg.exe` by its full path under `%SystemRoot%\System32`, since Windows resolves a bare name through the current directory before `PATH`.
- An input path the code page cannot represent is copied into the run's scratch directory, and `zli` reads the copy. When the code page could not be read, is unknown, or cannot be decoded, any input path past ASCII is copied: copying is safe either way, unlike refusing.
- A temp folder the code page cannot represent is refused before anything is created or spawned, with a message naming the characters and the code page. It matters because the scratch directory, the CSV and the trained candidates all live under it, so a user profile name outside the code page would otherwise fail even with an ASCII input path.
- Windows only: other platforms do no check.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 19, 2026
Summary:
Pull Request resolved: meta#1116

On Windows `zli` reads its arguments in the ANSI code page, so a path with a character outside it reaches `zli` mangled, or best-fit mapped to a different file name. The desktop app now checks first, instead of passing `zli` a bad path.

- `electron/ansi-path.cjs` reads this PC's ANSI code page from the registry (a failed read is not kept, so the next run tries again), parsing the `ACP` value by its name, not the `OEMCP` or `MACCP` beside it. It asks that code page which characters it can encode, by decoding its byte sequences; Windows-1252's 0x80-0x9F block is a table, because Node's `TextDecoder` treats that label as Latin-1. It runs `reg.exe` by its full path under `%SystemRoot%\System32`, since Windows resolves a bare name through the current directory before `PATH`.
- A code page it does not know, cannot decode, or cannot decode exactly is never refused. 936, 949 and 950 count as inexact: their `TextDecoder` tables depend on the ICU version (newer ICU maps GBK 0xA8BC to ḿ, which CP936 lacks; some builds' `euc-kr` lacks CP949's extended Hangul; `big5` has HKSCS characters CP950 does not).
- The input is checked once per run: `prepareInput` copies a path the code page may not represent into the run's scratch folder, and every row reads that one file. When the code page could not be read or checked exactly, any input path past ASCII is copied: copying is safe either way, unlike refusing. Byte inputs are written once per run the same way.
- A temp folder the code page cannot represent is refused before anything is created or spawned, with a message naming the characters and the code page. It matters because the scratch directory, the CSV and the trained candidates all live under it.
- Windows only: other platforms do no check.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 19, 2026
Summary:
Pull Request resolved: meta#1116

On Windows `zli` reads its arguments in the ANSI code page, so a path with a character outside it reaches `zli` mangled, or best-fit mapped to a different file name. The desktop app now checks first, instead of passing `zli` a bad path.

- `electron/ansi-path.cjs` reads this PC's ANSI code page from the registry (a failed read is not kept, so the next run tries again), parsing the `ACP` value by its name, not the `OEMCP` or `MACCP` beside it. It asks that code page which characters it can encode, by decoding its byte sequences; Windows-1252's 0x80-0x9F block is a table, because Node's `TextDecoder` treats that label as Latin-1. It runs `reg.exe` by its full path under `%SystemRoot%\System32`, since Windows resolves a bare name through the current directory before `PATH`.
- A code page it does not know, cannot decode, or cannot decode exactly is never refused. 936, 949 and 950 count as inexact: their `TextDecoder` tables depend on the ICU version (newer ICU maps GBK 0xA8BC to ḿ, which CP936 lacks; some builds' `euc-kr` lacks CP949's extended Hangul; `big5` has HKSCS characters CP950 does not).
- The input is checked once per run: `prepareInput` copies a path the code page may not represent into the run's scratch folder, and every row reads that one file. When the code page could not be read or checked exactly, any input path past ASCII is copied: copying is safe either way, unlike refusing. Byte inputs are written once per run the same way.
- A temp folder the code page cannot represent is refused before anything is created or spawned, with a message naming the characters and the code page. It matters because the scratch directory, the CSV and the trained candidates all live under it.
- Windows only: other platforms do no check.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 20, 2026
Summary:
Pull Request resolved: meta#1116

On Windows `zli` reads its arguments in the ANSI code page, so a path with a character outside it reaches `zli` mangled, or best-fit mapped to a different file name. The desktop app now checks first, instead of passing `zli` a bad path.

- `electron/ansi-path.cjs` reads this PC's ANSI code page from the registry (a failed read is not kept, so the next run tries again), parsing the `ACP` value by its name, not the `OEMCP` or `MACCP` beside it. It asks that code page which characters it can encode, by decoding its byte sequences; Windows-1252's 0x80-0x9F block is a table, because Node's `TextDecoder` treats that label as Latin-1. It runs `reg.exe` by its full path under `%SystemRoot%\System32`, since Windows resolves a bare name through the current directory before `PATH`.
- A code page it does not know, cannot decode, or cannot decode exactly is never refused. 936, 949 and 950 count as inexact: their `TextDecoder` tables depend on the ICU version (newer ICU maps GBK 0xA8BC to ḿ, which CP936 lacks; some builds' `euc-kr` lacks CP949's extended Hangul; `big5` has HKSCS characters CP950 does not).
- The input is checked once per run: `prepareInput` copies a path the code page may not represent into the run's scratch folder, and every row reads that one file. When the code page could not be read or checked exactly, any input path past ASCII is copied: copying is safe either way, unlike refusing. Byte inputs are written once per run the same way.
- A temp folder the code page cannot represent is refused before anything is created or spawned, with a message naming the characters and the code page. It matters because the scratch directory, the CSV and the trained candidates all live under it.
- Windows only: other platforms do no check.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 20, 2026
Summary:
Pull Request resolved: meta#1116

On Windows `zli` reads its arguments in the ANSI code page, so a path with a character outside it reaches `zli` mangled, or best-fit mapped to a different file name. The desktop app now checks first, instead of passing `zli` a bad path.

- `electron/ansi-path.cjs` reads this PC's ANSI code page from the registry (a failed read is not kept, so the next run tries again), parsing the `ACP` value by its name, not the `OEMCP` or `MACCP` beside it. It asks that code page which characters it can encode, by decoding its byte sequences; Windows-1252's 0x80-0x9F block is a table, because Node's `TextDecoder` treats that label as Latin-1. It runs `reg.exe` by its full path under `%SystemRoot%\System32`, since Windows resolves a bare name through the current directory before `PATH`.
- A code page it does not know, cannot decode, or cannot decode exactly is never refused. 936, 949 and 950 count as inexact: their `TextDecoder` tables depend on the ICU version (newer ICU maps GBK 0xA8BC to ḿ, which CP936 lacks; some builds' `euc-kr` lacks CP949's extended Hangul; `big5` has HKSCS characters CP950 does not).
- The input is checked once per run: `prepareInput` copies a path the code page may not represent into the run's scratch folder, and every row reads that one file. When the code page could not be read or checked exactly, any input path past ASCII is copied: copying is safe either way, unlike refusing. Byte inputs are written once per run the same way.
- A temp folder the code page cannot represent is refused before anything is created or spawned, with a message naming the characters and the code page. It matters because the scratch directory, the CSV and the trained candidates all live under it.
- Windows only: other platforms do no check.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 22, 2026
Summary:
Pull Request resolved: meta#1116

On Windows `zli` reads its arguments in the ANSI code page, so a path with a character outside it reaches `zli` mangled, or best-fit mapped to a different file name. The desktop app now checks first, instead of passing `zli` a bad path.

- `electron/ansi-path.cjs` reads this PC's ANSI code page from the registry (a failed read is not kept, so the next run tries again), parsing the `ACP` value by its name, not the `OEMCP` or `MACCP` beside it. It asks that code page which characters it can encode, by decoding its byte sequences; Windows-1252's 0x80-0x9F block is a table, because Node's `TextDecoder` treats that label as Latin-1. It runs `reg.exe` by its full path under `%SystemRoot%\System32`, since Windows resolves a bare name through the current directory before `PATH`.
- A code page it does not know, cannot decode, or cannot decode exactly is never refused. 936, 949 and 950 count as inexact: their `TextDecoder` tables depend on the ICU version (newer ICU maps GBK 0xA8BC to ḿ, which CP936 lacks; some builds' `euc-kr` lacks CP949's extended Hangul; `big5` has HKSCS characters CP950 does not).
- The input is checked once per run: `prepareInput` copies a path the code page may not represent into the run's scratch folder, and every row reads that one file. When the code page could not be read or checked exactly, any input path past ASCII is copied: copying is safe either way, unlike refusing. Byte inputs are written once per run the same way.
- A temp folder the code page cannot represent is refused before anything is created or spawned, with a message naming the characters and the code page. It matters because the scratch directory, the CSV and the trained candidates all live under it.
- Windows only: other platforms do no check.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 22, 2026
Summary:
Pull Request resolved: meta#1116

On Windows `zli` reads its arguments in the ANSI code page, so a path with a character outside it reaches `zli` mangled, or best-fit mapped to a different file name. The desktop app now checks first and refuses such a path with a message naming the characters, instead of passing `zli` a bad path.

- `electron/ansi-path.cjs` reads this PC's ANSI code page from the registry (a failed read is not kept, so the next run tries again), parsing the `ACP` value by its name, not the `OEMCP` or `MACCP` beside it. It runs `reg.exe` by its full path under `%SystemRoot%\System32`, since Windows resolves a bare name through the current directory before `PATH`.
- Which characters a code page has comes from `iconv-lite`, which ships Windows' own tables for 874, 932, 936, 949, 950 and 1250-1258: a character is representable if it survives encoding and decoding. This replaces decoding byte sequences with Node's `TextDecoder`, whose `gbk`, `euc-kr` and `big5` tables depend on the ICU version and differ from CP936, CP949 and CP950, so those code pages no longer need special handling. UTF-8, and a code page `iconv-lite` does not know, are never refused.
- `prepareInput` checks the input path and the temp folder once per run, before anything is created or spawned, and writes a byte input to a file once, so every row reads that one file. The temp folder matters because the scratch directory, the CSV and the trained candidates all live under it.
- Windows only: other platforms do no check.
- `iconv-lite` 0.6.3, already in the lockfile through the build tools, is now a dependency. `after-pack.cjs` copies it and `safer-buffer` into `resources/node_modules` beside zod, since the worker loads it.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 22, 2026
Summary:
Pull Request resolved: meta#1116

On Windows, `zli` reads its arguments in the ANSI code page. A path with characters outside that code page reaches `zli` garbled, or best-fit mapped to a different file. The desktop app now checks the input path and the temp folder before a run, and fails with an error that names the characters.

- `ansi-path.cjs` reads the code page (`ACP`) from the registry, running `reg.exe` by its full path under `%SystemRoot%\System32`. A failed read is retried on the next run.
- `iconv-lite` decides which characters the code page can encode. It ships Windows' own tables; Node's `TextDecoder` CJK tables vary with the ICU version. UTF-8 and unknown code pages are never refused.
- `prepareInput` runs once per run: it checks the paths, and writes a byte input to one file that every row reads.
- Other platforms are not checked.
- `iconv-lite` 0.6.3, already in the lockfile, becomes a dependency. `after-pack.cjs` ships it and `safer-buffer` with the packaged app.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 22, 2026
Summary:
Pull Request resolved: meta#1116

On Windows, `zli` reads its arguments in the ANSI code page. A path with characters outside that code page reaches `zli` garbled, or best-fit mapped to a different file. The desktop app now checks the input path and the temp folder before a run, and fails with an error that names the characters.

- `ansi-path.cjs` reads the code page (`ACP`) from the registry, running `reg.exe` by its full path under `%SystemRoot%\System32`. A failed read is retried on the next run.
- `iconv-lite` decides which characters the code page can encode. It ships Windows' own tables; Node's `TextDecoder` CJK tables vary with the ICU version. UTF-8 and unknown code pages are never refused.
- `prepareInput` runs once per run: it checks the paths, and writes a byte input to one file that every row reads.
- Other platforms are not checked.
- `iconv-lite` 0.6.3, already in the lockfile, becomes a dependency. `after-pack.cjs` ships it and `safer-buffer` with the packaged app.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Oct 22, 2026
Summary:
Pull Request resolved: meta#1116

On Windows, `zli` reads its arguments in the ANSI code page. A path with characters outside that code page reaches `zli` garbled, or best-fit mapped to a different file. The desktop app now checks the input path and the temp folder before a run, and fails with an error that names the characters.

- `ansi-path.cjs` reads the code page (`ACP`) from the registry, running `reg.exe` by its full path under `%SystemRoot%\System32`. A failed read is retried on the next run.
- `iconv-lite` decides which characters the code page can encode. It ships Windows' own tables; Node's `TextDecoder` CJK tables vary with the ICU version. UTF-8 and unknown code pages are never refused.
- `prepareInput` runs once per run: it checks the paths, and writes a byte input to one file that every row reads.
- Other platforms are not checked.
- `iconv-lite` 0.6.3, already in the lockfile, becomes a dependency. `after-pack.cjs` ships it and `safer-buffer` with the packaged app.

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Nov 5, 2026
Summary:
X-link: meta#1116

On Windows, `zli` reads its arguments in the ANSI code page. A path with characters outside that code page reaches `zli` garbled, or best-fit mapped to a different file. The desktop app now checks the input path and the temp folder before a run, and fails with an error that names the characters.

- `ansi-path.cjs` reads the code page (`ACP`) from the registry, running `reg.exe` by its full path under `%SystemRoot%\System32`. A failed read is retried on the next run.
- `iconv-lite` decides which characters the code page can encode. It ships Windows' own tables; Node's `TextDecoder` CJK tables vary with the ICU version. UTF-8 and unknown code pages are never refused.
- `checkInputPath` runs once per run, before any job starts, so a path `zli` cannot read fails the run once rather than failing every job. `measureOpenZL` also checks before it creates or spawns anything.
- Other platforms are not checked.
- `iconv-lite` 0.6.3, already in the lockfile, becomes a dependency. `after-pack.cjs` ships it and `safer-buffer` with the packaged app, and `ansi-path.cjs` goes beside the worker in `extraResources`.

Reviewed By: jlee303

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Nov 5, 2026
Summary:
X-link: meta#1116

On Windows, `zli` reads its arguments in the ANSI code page. A path with characters outside that code page reaches `zli` garbled, or best-fit mapped to a different file. The desktop app now checks the input path and the temp folder before a run, and fails with an error that names the characters.

- `ansi-path.cjs` reads the code page (`ACP`) from the registry, running `reg.exe` by its full path under `%SystemRoot%\System32`. A failed read is retried on the next run.
- `iconv-lite` decides which characters the code page can encode. It ships Windows' own tables; Node's `TextDecoder` CJK tables vary with the ICU version. UTF-8 and unknown code pages are never refused.
- `checkInputPath` runs once per run, before any job starts, so a path `zli` cannot read fails the run once rather than failing every job. `measureOpenZL` also checks before it creates or spawns anything.
- Other platforms are not checked.
- `iconv-lite` 0.6.3, already in the lockfile, becomes a dependency. `after-pack.cjs` ships it and `safer-buffer` with the packaged app, and `ansi-path.cjs` goes beside the worker in `extraResources`.

Reviewed By: jlee303

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Nov 5, 2026
Summary:
X-link: meta#1116

On Windows, `zli` reads its arguments in the ANSI code page. A path with characters outside that code page reaches `zli` garbled, or best-fit mapped to a different file. The desktop app now checks the input path and the temp folder before a run, and fails with an error that names the characters.

- `ansi-path.cjs` reads the code page (`ACP`) from the registry, running `reg.exe` by its full path under `%SystemRoot%\System32`. A failed read is retried on the next run.
- `iconv-lite` decides which characters the code page can encode. It ships Windows' own tables; Node's `TextDecoder` CJK tables vary with the ICU version. UTF-8 and unknown code pages are never refused.
- `checkInputPath` runs once per run, before any job starts, so a path `zli` cannot read fails the run once rather than failing every job. `measureOpenZL` also checks before it creates or spawns anything.
- Other platforms are not checked.
- `iconv-lite` 0.6.3, already in the lockfile, becomes a dependency. `after-pack.cjs` ships it and `safer-buffer` with the packaged app, and `ansi-path.cjs` goes beside the worker in `extraResources`.

Reviewed By: jlee303

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Nov 5, 2026
Summary:
X-link: meta#1116

On Windows, `zli` reads its arguments in the ANSI code page. A path with characters outside that code page reaches `zli` garbled, or best-fit mapped to a different file. The desktop app now checks the input path and the temp folder before a run, and fails with an error that names the characters.

- `ansi-path.cjs` reads the code page (`ACP`) from the registry, running `reg.exe` by its full path under `%SystemRoot%\System32`. A failed read is retried on the next run.
- `iconv-lite` decides which characters the code page can encode. It ships Windows' own tables; Node's `TextDecoder` CJK tables vary with the ICU version. UTF-8 and unknown code pages are never refused.
- `checkInputPath` runs once per run, before any job starts, so a path `zli` cannot read fails the run once rather than failing every job. `measureOpenZL` also checks before it creates or spawns anything.
- Other platforms are not checked.
- `iconv-lite` 0.6.3, already in the lockfile, becomes a dependency. `after-pack.cjs` ships it and `safer-buffer` with the packaged app, and `ansi-path.cjs` goes beside the worker in `extraResources`.

Reviewed By: jlee303

Differential Revision: D123741307
@meta-codesync meta-codesync Bot changed the title Handle paths zli cannot read on Windows (#1116) Refuse paths zli cannot read on Windows Nov 5, 2026
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Nov 5, 2026
Summary:
Pull Request resolved: meta#1116

X-link: meta#1116

On Windows, `zli` reads its arguments in the ANSI code page. A path with characters outside that code page reaches `zli` garbled, or best-fit mapped to a different file. The desktop app now checks the input path and the temp folder before a run, and fails with an error that names the characters.

- `ansi-path.cjs` reads the code page (`ACP`) from the registry, running `reg.exe` by its full path under `%SystemRoot%\System32`. A failed read is retried on the next run.
- `iconv-lite` decides which characters the code page can encode. It ships Windows' own tables; Node's `TextDecoder` CJK tables vary with the ICU version. UTF-8 and unknown code pages are never refused.
- `checkInputPath` runs once per run, before any job starts, so a path `zli` cannot read fails the run once rather than failing every job. `measureOpenZL` also checks before it creates or spawns anything.
- Other platforms are not checked.
- `iconv-lite` 0.6.3, already in the lockfile, becomes a dependency. `after-pack.cjs` ships it and `safer-buffer` with the packaged app, and `ansi-path.cjs` goes beside the worker in `extraResources`.

Reviewed By: jlee303

Differential Revision: D123741307
@meta-codesync meta-codesync Bot changed the title Refuse paths zli cannot read on Windows Refuse paths zli cannot read on Windows (#1116) Nov 5, 2026
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Nov 5, 2026
Summary:
Pull Request resolved: meta#1116

X-link: meta#1116

On Windows, `zli` reads its arguments in the ANSI code page. A path with characters outside that code page reaches `zli` garbled, or best-fit mapped to a different file. The desktop app now checks the input path and the temp folder before a run, and fails with an error that names the characters.

- `ansi-path.cjs` reads the code page (`ACP`) from the registry, running `reg.exe` by its full path under `%SystemRoot%\System32`. A failed read is retried on the next run.
- `iconv-lite` decides which characters the code page can encode. It ships Windows' own tables; Node's `TextDecoder` CJK tables vary with the ICU version. UTF-8 and unknown code pages are never refused.
- `checkInputPath` runs once per run, before any job starts, so a path `zli` cannot read fails the run once rather than failing every job. `measureOpenZL` also checks before it creates or spawns anything.
- Other platforms are not checked.
- `iconv-lite` 0.6.3, already in the lockfile, becomes a dependency. `after-pack.cjs` ships it and `safer-buffer` with the packaged app, and `ansi-path.cjs` goes beside the worker in `extraResources`.

Reviewed By: jlee303

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Nov 5, 2026
Summary:
Pull Request resolved: meta#1116

X-link: meta#1116

On Windows, `zli` reads its arguments in the ANSI code page. A path with characters outside that code page reaches `zli` garbled, or best-fit mapped to a different file. The desktop app now checks the input path and the temp folder before a run, and fails with an error that names the characters.

- `ansi-path.cjs` reads the code page (`ACP`) from the registry, running `reg.exe` by its full path under `%SystemRoot%\System32`. A failed read is retried on the next run.
- `iconv-lite` decides which characters the code page can encode. It ships Windows' own tables; Node's `TextDecoder` CJK tables vary with the ICU version. UTF-8 and unknown code pages are never refused.
- `checkInputPath` runs once per run, before any job starts, so a path `zli` cannot read fails the run once rather than failing every job. `measureOpenZL` also checks before it creates or spawns anything.
- Other platforms are not checked.
- `iconv-lite` 0.6.3, already in the lockfile, becomes a dependency. `after-pack.cjs` ships it and `safer-buffer` with the packaged app, and `ansi-path.cjs` goes beside the worker in `extraResources`.

Reviewed By: jlee303

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Nov 5, 2026
Summary:
Pull Request resolved: meta#1116

X-link: meta#1116

On Windows, `zli` reads its arguments in the ANSI code page. A path with characters outside that code page reaches `zli` garbled, or best-fit mapped to a different file. The desktop app now checks the input path and the temp folder before a run, and fails with an error that names the characters.

- `ansi-path.cjs` reads the code page (`ACP`) from the registry, running `reg.exe` by its full path under `%SystemRoot%\System32`. A failed read is retried on the next run.
- `iconv-lite` decides which characters the code page can encode. It ships Windows' own tables; Node's `TextDecoder` CJK tables vary with the ICU version. UTF-8 and unknown code pages are never refused.
- `checkInputPath` runs once per run, before any job starts, so a path `zli` cannot read fails the run once rather than failing every job. `measureOpenZL` also checks before it creates or spawns anything.
- Other platforms are not checked.
- `iconv-lite` 0.6.3, already in the lockfile, becomes a dependency. `after-pack.cjs` ships it and `safer-buffer` with the packaged app, and `ansi-path.cjs` goes beside the worker in `extraResources`.

Reviewed By: jlee303

Differential Revision: D123741307
Summary:
Pull Request resolved: meta#1215

The web playground's sample button has never done anything: `App` passes it `onTrySample={null}`. This wires it up without making the page any heavier to load.

Pressing "Try a sample" downloads `examples/getting_started/sample_inputs/sra0.bin` (2,071,976 bytes), makes it the chosen file, and runs the benchmark with the current rows, so Run repeats it afterwards. The file is imported with Vite's `?url` suffix, which emits it as an asset of its own (`assets/sra0-<hash>.bin`) rather than into a bundle: the page holds only its URL, nothing downloads it until the button is pressed, and the hashed name lets the browser cache it.

- `src/fetchSample.ts` fetches the file and wraps it in a `File` named `sra0.bin`. Every way the download can fail (an error status, no network, a body cut off partway) rejects with "Could not download the sample: <reason>", where the reason is the status (without an empty status text, which HTTP/2 never sends) or the browser's error.
- `App` sets the run to `loading` from the press, so both buttons and the file picker stay disabled while the download is in flight, and nothing can be chosen over the sample. A failed download ends as an error ("Run failed: Could not download the sample: …") and nothing runs. The rows and iterations stay editable during the download, so the run takes them as they are when it starts, not as they were at the press; a row emptied in the meantime leaves the sample chosen but not run, and Run then says why. As with Run, the sample is held back while a zstd or gzip row has no levels: that check moves out of `runBlockersFor` into `configBlockersFor`, which both use.
- `UploadCard` takes a `disabled` prop, which `SetupColumn` sets while a run or the sample download is in flight: the file input, the remove button and drops are refused, and the card is dimmed, like the disabled buttons beside it.
- A tooltip on "Try a sample", and the same words as its description for a screen reader, say what the sample is: "The SAO star catalog, part of the Silesia compression corpus", with the corpus name linking to its page.
- The button and the empty results text read "Try a sample", since the file is 2.1 MB rather than 5; the comments that promised 5 MB no longer do. The progress label while loading reads "Loading…" rather than "Loading the OpenZL module…", since the sample download is part of that state.
- `vite.config.ts` adds `examples/getting_started/sample_inputs` to the dev server's `fs.allow`, because `examples/` is outside `tools/`. Without it the dev server answers the sample's URL with a 403. Only that directory is added, not the repository root. Builds are unaffected.

The deployed site gains one 2.07 MB file, downloaded only on demand. D123708843, in the desktop playground stack, uses the same file and wording for the desktop app's installed sample, so whichever lands second chooses between the two in `onTrySample`.

Differential Revision: D127029921
Summary:
Pull Request resolved: meta#1118

X-link: meta#1118

Add an Electron desktop host for the compression playground whose OpenZL jobs run the `zli` CLI, so OpenZL runs in child processes outside Electron. The backend speaks the web playground's protocol, with its types. The page starts calling it in D123708843; until then the window renders the web playground as it is, benchmarking with WASM.

- Request: `DesktopRunConfig`, added to `benchmarkTypes.ts`, which is the browser's `RunConfig` with the input named by its path. The worker splits it into `BenchmarkJob`s as `buildJobs` does: one job per compressor and level, with ids `${rowId}-${compressor}-${level}`, and a row with no levels is `rejected` with "No levels selected" rather than dropped.
- Messages: the browser worker's `RunMessage`s (also added to `benchmarkTypes.ts`): `loading`, `started` with `totalJobs` and `rejected`, a `result` (`JobResult`) per measurement or a `failure` (`JobFailure`), then `finished` or `failed`. Metrics are wasm_api's `BenchmarkResult`. Candidates are `{index, total}` counted from 1, best ratio first, or `null` for a job that does not train. As in `measureRun.ts`, a candidate that cannot be benchmarked is a `failure` naming it and the other candidates are still measured; a job that fails is reported and the run goes on. The preload's `benchmark(config)` resolves with the run's messages once it ends; D122436832 streams them.
- Untrained jobs run `zli benchmark --output-csv`. Trained jobs run `zli train --pareto-frontier --use-all-samples`, then `zli benchmark --compressor` on each `<i>.zc` candidate. `zli train` has no `--level` and benchmarks at a fixed iteration count, so the job's level and iteration count apply when each candidate is benchmarked. Only the leading numeric CSV columns are read; the trailing `path` column is ignored.
- The input is always a file on disk, which `zli` reads where it is. The scratch directory only holds `zli`'s CSV and candidate files.
- The Electron code is CommonJS type-checked by TypeScript (`checkJs`), importing the web types with JSDoc `import`, so `yarn typecheck` fails if the two builds drift apart.
- `benchmark-request.cjs` checks the request's shape only: integers, at least one iteration, at least one candidate, a non-empty profile, and a path with no NUL. Which levels, candidate counts, iteration counts and how many rows are valid is for `zli` and zlib to say; a job they refuse becomes a `failure`. A training time budget (`TrainingConfig.maxTimeSecs`, which only the browser's smoke test sets) is not part of the desktop request and is dropped from the copy: `zli train` runs until it finishes.
- A utility process runs the jobs (`native-worker.cjs`, `zli-client.cjs`); zstd and gzip are measured with Electron's zlib. `electron/native/CMakeLists.txt` builds `zli` for electron-builder to package.
- `main.cjs` opens a sandboxed window, denies browser permissions, and only opens `https://openzl.org` links externally. The preload exposes `getPathForFile` and `benchmark`.
- `setup.cjs` (`yarn setup`, which `yarn dev`, `yarn build` and `yarn package` run first) uses `getPlatform()` to do what the platform needs: install the `tools/web` workspace's dependencies, install ALSA's library with apt on Linux if it is missing, configure CMake with `-T ClangCL` on Windows and remove a build directory configured for another toolset or source directory (compared by real path), then build `zli`.
- The README's getting started is `yarn install` and `yarn dev`, followed by a testing and packaging section and a note to use a short checkout path on Windows.
- The app is its own Yarn project in `tools/desktop/compression_playground`, apart from the `tools/web` workspace, so the documentation build never installs Electron. Its page is the web playground, which `yarn build:renderer` builds into the app's `dist/`.
- `window-security-smoke.cjs` also checks the playground's CSP from D122405702.

Reviewed By: jlee303

Differential Revision: D122387996
Summary:
Pull Request resolved: meta#1110

X-link: meta#1110

Serve the packaged playground from a privileged `app://bundle` scheme, and flip Electron fuses on the packaged binary.

- Custom protocol: `protocol.handle` serves the renderer instead of `file://`, so CSP `'self'` and fetch have a precise origin and file pages' legacy privileges no longer apply. `electron/app-protocol.cjs` maps requests to bundle files and refuses other hosts and anything that resolves outside the bundle (including an encoded `..%2F`). Only a missing file is a 404; any other read error is logged and returns a 500, so a page that will not load is diagnosable. IPC is accepted only from a frame whose origin matches the renderer (`app://bundle` packaged, the Vite origin in `yarn dev`); an origin that cannot be parsed is untrusted.
- Fuses: an electron-builder `afterPack` hook (`electron/after-pack.cjs`, `electron/fuses` 1.8.0) disables `RunAsNode`, `NODE_OPTIONS`, `--inspect` arguments and `GrantFileProtocolExtraPrivileges`, and enables `OnlyLoadAppFromAsar` and cookie encryption. `EnableEmbeddedAsarIntegrityValidation` stays off because electron-builder 24 does not embed the integrity hash on macOS and Windows. `electron/fuses` is pinned to 1.8.0, the version this was written and tested against; 2.x needs Node 22.12+, which the workspace's Node 24 requirement (D123528576) already meets, so moving to it can be a follow-up.
- `packaged-app-smoke.cjs` (`yarn test:packaged`) checks the packaged binary: the fuse wire, `ELECTRON_RUN_AS_NODE` ignored, no inspector, and a real benchmark from `app://bundle`. It fails with a message, rather than hanging, if no `app://` page target appears or the DevTools socket closes or does not reply within 5 minutes.

Reviewed By: jlee303

Differential Revision: D122425385
Summary:
X-link: meta#1113

Make benchmarks cancellable and make sure `zli` never outlives the app. A child does not exit with its parent on Linux and macOS, so quitting mid-training used to leave `zli train` running.

- Each run has its own message port. The preload's `startBenchmark(config)` returns `{messages, cancel}`, so the page still never touches IPC. D122436832 turns `messages` into a callback for each message as it arrives.
- Cancel: the utility process aborts. A running `zli` gets SIGTERM, then SIGKILL after 2 s; the worker stops between jobs, and zstd and gzip yield between timed iterations. A closed port (reload, crash, window closed) cancels the run.
- Backstop: the worker reports every `zli` PID, and the main process SIGKILLs any still recorded when a run ends, when the worker dies or misses a 5 s cancel deadline, and on quit. A `spawned` report can be lost if the worker is killed first, so the main process also looks the worker's children up itself before killing it: on Linux and macOS it stops the worker (SIGSTOP), so it cannot start another, finds its children with `ps`, SIGKILLs them, then SIGKILLs the worker; on Windows `taskkill /T` kills the tree. The lookup is skipped once the worker has reported `finished`, `failed` or `cancelled`, which it only sends after its `zli` has exited.
- A worker that exits while a cancel is in flight counts as cancelled only with exit code 0; a non-zero exit is reported as a failure with its code, not hidden as a cancel.
- The main process owns each run's scratch directory, created before the worker is forked, and removes it after killing the worker, in every case.
- The main process forwards the run's messages on its port, and the run ends with `finished`, `failed`, or the desktop-only `cancelled`. A failed job does not end the run; a cancel does.

Backend only: there is no Cancel button.

Reviewed By: jlee303

Differential Revision: D122435268
Summary:
X-link: meta#1117

Stream progress and each measurement from the desktop backend instead of returning everything at the end, so a trained job no longer shows a bare spinner for minutes.

- `zli train` runs at `--verbose 3`, which prints a progress-bar line per redraw on a pipe (`[=====-----...] Training ACE graph 1 / 2: ACE progress`). Each line becomes a `step` with the current job's own progress through training, as the browser forwards its trainer's progress callback, placed within its ACE graph's share so the fraction does not restart for the next graph. The page already places a job's step within the run, so the backend reports nothing else: there are no stage events and no fixed training share. Other `zli` calls stay at `--verbose 1`.
- On failure only `zli`'s fatal error is shown, which it always prints last as a heading and tab-indented detail, not the progress and INFO lines before it. A `zli` killed by a signal is reported by its signal.
- Each message is sent as soon as it is made: a trained job's candidates one by one, each a `result` or a `failure` with its `{index, total}`, while the other candidates are still measured, as `measureRun.ts` does.
- The preload's `startBenchmark(config, onMessage)` passes each message to `onMessage` as it arrives and returns `{done, cancel}`; `done` settles when the run ends, rejecting if it failed or was cancelled.

Backend only. The page folds these messages into `RunState` in D123708843.

Nothing later in the stack changes the progress protocol.

Reviewed By: jlee303

Differential Revision: D122436832
Summary:
X-link: meta#1114

Concurrency limits for the desktop backend.

- One run at a time: the main process refuses a second concurrent run with "A benchmark is already running", since concurrent runs would distort each other's timings.
- Errors refused at the IPC boundary reach the page as plain messages, without an "Error:" or "TypeError:" prefix.

Progress is not throttled: a trained job sends about a hundred steps, few enough not to matter. A throttle can come back if it turns out to be needed.

Reviewed By: jlee303

Differential Revision: D122437936
Summary:
Pull Request resolved: meta#1115

X-link: meta#1115

Replace the hand-written request checks with zod schemas, and validate the two boundaries that were trusted before. Each schema is typed to the web playground's types through JSDoc, under `checkJs`, so a schema that drifts from `benchmarkTypes.ts` fails `yarn typecheck`.

- Renderer to main (`electron/benchmark-request.cjs`): `validateRequest` parses a `DesktopRunConfig` (the web row shape: `rowId`, `levels`, `profile`, `training` or `null`) and returns a validated copy with unknown keys stripped, so only its fields reach the worker. It checks shape only, with no limits on levels, candidates, iterations or rows, and training is `paretoFrontier` and `candidates` with no time budget; a bad field is a TypeError naming it ("Iterations must be a positive integer", "Unknown compressor", "OpenZL training options are invalid", ...).
- Worker to main (`electron/worker-messages.cjs`): every utility-process message is parsed as the worker's `WorkerOutput`, the web `RunMessage`s plus the desktop's `cancelled` and `child`, before the main process acts on it or forwards it. Jobs are full `BenchmarkJob`s, metrics are `BenchmarkResult`, candidates are counted from 1 and not past their total, a `step` fraction is from 0 to 1, and sizes and timings are non-negative (throughput may be Infinity for 0 ms). An invalid message ends the run with an error naming the field, and the usual cleanup still runs.
- `zli` to worker (`electron/zli-client.cjs`): a schema for the `zli benchmark --output-csv` row, read by column name (whole-number sizes and iteration count, non-negative finite timings), replaces the column check, so format drift fails as "zli returned an unexpected result (srcSize: expected a whole number)" instead of showing NaN. A repeated column is refused.
- zod 4.6.5, pinned. The packaged app excludes `node_modules`, so `after-pack.cjs` copies zod to `resources/node_modules`.

The renderer does not validate what it receives: everything it gets has passed the worker-message schema, and the sandboxed preload cannot load zod.

Reviewed By: jlee303

Differential Revision: D122441258
Summary:
Pull Request resolved: meta#1116

X-link: meta#1116

On Windows, `zli` reads its arguments in the ANSI code page. A path with characters outside that code page reaches `zli` garbled, or best-fit mapped to a different file. The desktop app now checks the input path and the temp folder before a run, and fails with an error that names the characters.

- `ansi-path.cjs` reads the code page (`ACP`) from the registry, running `reg.exe` by its full path under `%SystemRoot%\System32`. A failed read is retried on the next run.
- `iconv-lite` decides which characters the code page can encode. It ships Windows' own tables; Node's `TextDecoder` CJK tables vary with the ICU version. UTF-8 and unknown code pages are never refused.
- `checkInputPath` runs once per run, before any job starts, so a path `zli` cannot read fails the run once rather than failing every job. `measureOpenZL` also checks before it creates or spawns anything.
- Other platforms are not checked.
- `iconv-lite` 0.6.3, already in the lockfile, becomes a dependency. `after-pack.cjs` ships it and `safer-buffer` with the packaged app, and `ansi-path.cjs` goes beside the worker in `extraResources`.

Reviewed By: jlee303

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Nov 6, 2026
Summary:
Pull Request resolved: meta#1116

X-link: meta#1116

On Windows, `zli` reads its arguments in the ANSI code page. A path with characters outside that code page reaches `zli` garbled, or best-fit mapped to a different file. The desktop app now checks the input path and the temp folder before a run, and fails with an error that names the characters.

- `ansi-path.cjs` reads the code page (`ACP`) from the registry, running `reg.exe` by its full path under `%SystemRoot%\System32`. A failed read is retried on the next run.
- `iconv-lite` decides which characters the code page can encode. It ships Windows' own tables; Node's `TextDecoder` CJK tables vary with the ICU version. UTF-8 and unknown code pages are never refused.
- `checkInputPath` runs once per run, before any job starts, so a path `zli` cannot read fails the run once rather than failing every job. `measureOpenZL` also checks before it creates or spawns anything.
- Other platforms are not checked.
- `iconv-lite` 0.6.3, already in the lockfile, becomes a dependency. `after-pack.cjs` ships it and `safer-buffer` with the packaged app, and `ansi-path.cjs` goes beside the worker in `extraResources`.

Reviewed By: jlee303

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Nov 6, 2026
Summary:
Pull Request resolved: meta#1116

X-link: meta#1116

On Windows, `zli` reads its arguments in the ANSI code page. A path with characters outside that code page reaches `zli` garbled, or best-fit mapped to a different file. The desktop app now checks the input path and the temp folder before a run, and fails with an error that names the characters.

- `ansi-path.cjs` reads the code page (`ACP`) from the registry, running `reg.exe` by its full path under `%SystemRoot%\System32`. A failed read is retried on the next run.
- `iconv-lite` decides which characters the code page can encode. It ships Windows' own tables; Node's `TextDecoder` CJK tables vary with the ICU version. UTF-8 and unknown code pages are never refused.
- `checkInputPath` runs once per run, before any job starts, so a path `zli` cannot read fails the run once rather than failing every job. `measureOpenZL` also checks before it creates or spawns anything.
- Other platforms are not checked.
- `iconv-lite` 0.6.3, already in the lockfile, becomes a dependency. `after-pack.cjs` ships it and `safer-buffer` with the packaged app, and `ansi-path.cjs` goes beside the worker in `extraResources`.

Reviewed By: jlee303

Differential Revision: D123741307
aidangrimshaw751-netizen added a commit to aidangrimshaw751-netizen/openzl that referenced this pull request Nov 6, 2026
Summary:
Pull Request resolved: meta#1116

X-link: meta#1116

On Windows, `zli` reads its arguments in the ANSI code page. A path with characters outside that code page reaches `zli` garbled, or best-fit mapped to a different file. The desktop app now checks the input path and the temp folder before a run, and fails with an error that names the characters.

- `ansi-path.cjs` reads the code page (`ACP`) from the registry, running `reg.exe` by its full path under `%SystemRoot%\System32`. A failed read is retried on the next run.
- `iconv-lite` decides which characters the code page can encode. It ships Windows' own tables; Node's `TextDecoder` CJK tables vary with the ICU version. UTF-8 and unknown code pages are never refused.
- `checkInputPath` runs once per run, before any job starts, so a path `zli` cannot read fails the run once rather than failing every job. `measureOpenZL` also checks before it creates or spawns anything.
- Other platforms are not checked.
- `iconv-lite` 0.6.3, already in the lockfile, becomes a dependency. `after-pack.cjs` ships it and `safer-buffer` with the packaged app, and `ansi-path.cjs` goes beside the worker in `extraResources`.

Reviewed By: jlee303

Differential Revision: D123741307

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant