feat: the write side of #475 — text-format options, and a CLI --bom - #563
Merged
Conversation
#475 answered the read side: `parseCsv` takes bytes, honours the byte-order mark, and `--encoding` names what no mark can. The write side was left with a documented remedy nobody could reach. ## Two ways to not get a BOM `hucre convert veriler.xlsx out.csv` emits UTF-8 with no mark, and had no flag to ask for one. Excel on a Turkish, Polish or Greek Windows reads a UTF-8 CSV as the system code page unless the file opens with `EF BB BF`, so the CLI's own output came back as mojibake — the #475 complaint, mirrored on the way out. `write({ sheets, format: "csv" })` had the same problem for a different reason: **it called every text writer with no options at all.** `writeCsv`, `writeTsv`, `writeJson`, `writeNdjson`, `writeXml`, `toHtml` and `toMarkdown` each take an options bag, and `write` passed none to any of them. So the entry #469 added precisely so one call could reach all nine formats was the only way to reach seven of them that could not configure a single thing about them — no delimiter, no `pretty`, no `rootTag`, no `caption`, and no `bom`. ## What changed `write` takes one bag per text format, passed to the writer it dispatches to: await write({ sheets, format: "csv", csv: { delimiter: ";", bom: true } }) await write({ sheets, format: "json", json: { pretty: true } }) and the CLI takes `--bom`: hucre convert veriler.xlsx out.csv --bom `renderWorkbook`'s separate CSV branch is gone with it: it existed to pass a delimiter that `write` could not take, and the comment above it already said everything else goes through `write`, "which is the function that is supposed to know how to do this". Now CSV does too. ## Not changed, and worth saying Output is UTF-8 and cannot be anything else — `TextEncoder` encodes only UTF-8 by specification, and `src/` is Web-APIs-only. A caller who needs windows-1254 bytes has to encode them, and the BOM is what makes UTF-8 work everywhere instead. ## Checked 10 new tests. The two CLI ones assert `EF BB BF` is present with the flag and absent without it, on both `.csv` and `.tsv`, and that the rows survive the prefix. The eight library ones each pass an option through `write` and assert it reached the writer. `pnpm test` green — 10,605 tests, 235 files. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
productdevbook
added a commit
that referenced
this pull request
Aug 13, 2026
`writeXml` turns a column heading into an element name and validated it
with
/^[A-Za-z_][\w.-]*(?::[A-Za-z_][\w.-]*)?$/
which is ASCII. XML 1.0 §2.3's `NameStartChar` runs from #xC0, so every
accented or non-Latin heading was refused:
writeXml([{ Şehir: "İzmir" }])
ParseError: Invalid XML name for element "Şehir": "Şehir"
`Şehir`, `Ünvan`, `Größe`, `café`, `naïve`, `名前`, `Ελλάδα`,
`Кириллица` — all valid XML, all rejected. A spreadsheet whose columns
are not named in English could not be written to XML **at all**. Not
mangled, not escaped: the format was unavailable, and the error said the
name was invalid when it was the check that was.
Found while testing #563, where a Turkish sheet could go to CSV, TSV,
JSON, NDJSON, HTML and Markdown but not XML.
## The productions, minus the colon
`NameStartChar` and `NameChar` verbatim, with `:` pulled out so a name
may carry one prefix rather than a colon wherever it likes — that is
`QName` from Namespaces in XML §4, and it is what the old regex was
reaching for too.
The `u` flag is not optional here: #x10000–#xEFFFF is above the BMP, and
without it the surrogate halves match separately.
Nothing new is accepted that XML forbids. A leading digit, a leading
hyphen or dot, a space, `<`, two colons, the empty string, and a leading
combining mark — `NameChar` but not `NameStartChar` — are still refused,
each with a test.
## Checked
27 tests, **14 of which fail against the old regex** and none of which is
a rejection case: the tests that say what stays refused pass either way,
which is the point of having them.
The reader was never the problem — `parseXml` reads any well-formed name
— and a non-ASCII name now round-trips through both.
`pnpm test` green — 10,622 tests, 236 files.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
productdevbook
added a commit
that referenced
this pull request
Aug 13, 2026
`writeXml` turns a column heading into an element name and validated it
with
/^[A-Za-z_][\w.-]*(?::[A-Za-z_][\w.-]*)?$/
which is ASCII. XML 1.0 §2.3's `NameStartChar` runs from #xC0, so every
accented or non-Latin heading was refused:
writeXml([{ Şehir: "İzmir" }])
ParseError: Invalid XML name for element "Şehir": "Şehir"
`Şehir`, `Ünvan`, `Größe`, `café`, `naïve`, `名前`, `Ελλάδα`,
`Кириллица` — all valid XML, all rejected. A spreadsheet whose columns
are not named in English could not be written to XML **at all**. Not
mangled, not escaped: the format was unavailable, and the error said the
name was invalid when it was the check that was.
Found while testing #563, where a Turkish sheet could go to CSV, TSV,
JSON, NDJSON, HTML and Markdown but not XML.
## The productions, minus the colon
`NameStartChar` and `NameChar` verbatim, with `:` pulled out so a name
may carry one prefix rather than a colon wherever it likes — that is
`QName` from Namespaces in XML §4, and it is what the old regex was
reaching for too.
The `u` flag is not optional here: #x10000–#xEFFFF is above the BMP, and
without it the surrogate halves match separately.
Nothing new is accepted that XML forbids. A leading digit, a leading
hyphen or dot, a space, `<`, two colons, the empty string, and a leading
combining mark — `NameChar` but not `NameStartChar` — are still refused,
each with a test.
## Checked
27 tests, **14 of which fail against the old regex** and none of which is
a rejection case: the tests that say what stays refused pass either way,
which is the point of having them.
The reader was never the problem — `parseXml` reads any well-formed name
— and a non-ASCII name now round-trips through both.
`pnpm test` green — 10,622 tests, 236 files.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: the write side of #475 — text-format options, and a CLI --bom
#475 answered the read side:
parseCsvtakes bytes, honours thebyte-order mark, and
--encodingnames what no mark can. The write sidewas left with a documented remedy nobody could reach.
Two ways to not get a BOM
hucre convert veriler.xlsx out.csvemits UTF-8 with no mark, and had noflag to ask for one. Excel on a Turkish, Polish or Greek Windows reads a
UTF-8 CSV as the system code page unless the file opens with
EF BB BF,so the CLI's own output came back as mojibake — the #475 complaint,
mirrored on the way out.
write({ sheets, format: "csv" })had the same problem for a differentreason: it called every text writer with no options at all.
writeCsv,writeTsv,writeJson,writeNdjson,writeXml,toHtmlandtoMarkdowneach take an options bag, andwritepassed none to any ofthem. So the entry #469 added precisely so one call could reach all nine
formats was the only way to reach seven of them that could not configure
a single thing about them — no delimiter, no
pretty, norootTag, nocaption, and nobom.What changed
writetakes one bag per text format, passed to the writer it dispatchesto:
and the CLI takes
--bom:renderWorkbook's separate CSV branch is gone with it: it existed to passa delimiter that
writecould not take, and the comment above it alreadysaid everything else goes through
write, "which is the function that issupposed to know how to do this". Now CSV does too.
Not changed, and worth saying
Output is UTF-8 and cannot be anything else —
TextEncoderencodes onlyUTF-8 by specification, and
src/is Web-APIs-only. A caller who needswindows-1254 bytes has to encode them, and the BOM is what makes UTF-8
work everywhere instead.
Checked
10 new tests. The two CLI ones assert
EF BB BFis present with the flagand absent without it, on both
.csvand.tsv, and that the rowssurvive the prefix. The eight library ones each pass an option through
writeand assert it reached the writer.pnpm testgreen — 10,605 tests, 235 files.Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
🤖 Generated with Claude Code