fix(adif): declare export field lengths as UTF-8 byte count - #228
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Pull request overview
Fixes ADIF export correctness for non-ASCII data by declaring field lengths as UTF-8 byte counts (per ADIF <name:length>value requirements) and refactors the exporter into a shared, testable library module.
Changes:
- Extracts ADIF export generation into
src/lib/adif.tswithadifField()(UTF-8 byte length) andgenerateAdif(). - Updates the ADIF export API route to delegate to the shared generator without changing the API contract.
- Adds unit-style Playwright tests validating UTF-8 byte-length headers and a parser round-trip for multi-byte operator/QTH data.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| tests/adif-generate.spec.ts | Adds coverage for UTF-8 byte-length field headers and generateAdif() output/round-trip behavior. |
| src/lib/adif.ts | Introduces AdifExportContact, adifField() using UTF-8 byte length, and generateAdif() to build ADIF documents. |
| src/app/api/adif/export/route.ts | Replaces inline exporter with generateAdif() from the shared ADIF library. |
|
Review of PR #228 (approval blocked by GitHub: can't approve one's own PR under this account — posting findings as a comment instead) Verified against the PR head commit directly (not just the diff). Correctness
Tests: New Style/conventions: snake_case fields preserved per repo convention, no No issues found. This PR looks good to merge. |
…ength ADIF field headers must declare data length as the number of UTF-8 octets, not UTF-16 code units. The exporter used String.length, so a value like "José" (5 bytes) was written as <name:5>... only when ASCII — multi-byte characters (accents, ø, CJK, emoji) under-counted, e.g. <name:4>José. Strict ADIF readers (LoTW/TQSL, Cloudlog, N1MM) then truncate or mis-align the field. Extract the generator into src/lib/adif.ts as pure, tested generateAdif() / adifField() (mirrors the existing parseAdifRecords test pattern) and count bytes via Buffer.byteLength. The export route now delegates to it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bc3ae7d to
06d8bad
Compare
…rator (#230) The /api/contacts/search?export=true path hand-rolled its own ADIF serializer instead of reusing generateAdif(). That copy carried three defects the main export had already shed: - Declared field lengths with JS String.length (UTF-16 code units) rather than the UTF-8 byte count ADIF requires — the interop bug fixed for the main export in #228. Accented names, ø-callsigns and CJK QTHs came out with wrong lengths and got truncated/mis-aligned by strict readers (LoTW/TQSL, Cloudlog, N1MM). - Called .toString()/.length on frequency/mode/band unconditionally, so a single contact imported without one of those fields 500'd the whole export. - Emitted only ~11 fields, silently dropping DXCC, QSL status, country, zones and station info. Delegating to generateAdif() fixes all three and keeps the two export paths in lockstep. Co-authored-by: Optio Agent <optio-agent@noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Problem
ADIF field headers (
<name:length>value) must declarelengthas the number of UTF-8 octets in the data, per the ADIF spec. The exporter (generateADIFinsrc/app/api/adif/export/route.ts) computed length with JavaScript'sString.length, which counts UTF-16 code units.For plain ASCII the two agree, so this went unnoticed. But any operator whose data contains multi-byte characters got the wrong length declared:
.length(emitted)José<name:4>München<qth:7>東京<qth:2>RN3Ø(slashed-zero callsign)<call:4>Strict ADIF readers — LoTW/TQSL, Cloudlog/Wavelog, N1MM — read exactly the declared number of bytes, so the under-count truncates the field or mis-aligns the parse of everything after it. This silently corrupts exports for non-English names, QTHs, and notes.
Solution
src/lib/adif.tsas pure, testablegenerateAdif()andadifField()functions — mirroring the existingparseAdifRecords()there and itstests/adif-parse.spec.ts.adifField()declares length viaBuffer.byteLength(value, 'utf8')so it is byte-correct. It also centralizes the "omit empty/null/zero optional fields" gating the route did inline, so field selection and value transforms (callsign/mode/band/grid uppercasing, freq in MHz, etc.) are unchanged.generateAdif/AdifExportContactand delegates. No API request/response shape changes; ASCII exports are byte-for-byte identical apart from the header'screated_timestamplength, which was previously off-by-one (:15for a 16-byte value) and is now correct.Testing
tests/adif-generate.spec.ts(12 assertions acrossadifFieldandgenerateAdif), including a round-trip throughparseAdifRecordsforJürgen/Münchento prove no truncation.npm run typecheck— cleannpm run lint— cleannpm run build— compiled successfullynpx playwright test adif-generate adif-parse— 12 passedBackwards compatibility
Fully preserved. Pure ASCII logs (the common case) export identically; only multi-byte fields change, and they change from broken to correct.
Future follow-up
The ADIF parser (
parseSingleRecord) reads field data with[^<]*and clips by declared length using JS string length rather than reading exactly N UTF-8 bytes. It round-trips our own output fine, but a length-driven byte reader would be more robust against third-party files whose data contains a literal<. Out of scope here.🤖 Generated with Claude Code