Escape the last latin-1 byte in reuseComponent_pl as octal (#33) - #36
Conversation
The distribution now has no latin-1 source file. ## Reasoning `lib/PDF/Util/reuseComponent_pl` held one high byte, 0xb4 (acute accent), inside the single-quoted string literal passed to prText() on line 168 — a demo line that shows what a font renders across a character range. That is categorically unlike #27, where every high byte sat in a comment and could not affect behaviour. Chris chose option 1 from the issue: write it `\264`, matching the `\305\304\326\345\344\366` already on the same line. Perl leaves numeric backslash escapes untouched in single quotes, so the literal reaching prText() is unchanged and the byte on the page is still 0xb4. Options rejected: - Leave it latin-1 and document the encoding. Costs nothing today but keeps a file that silently corrupts under any UTF-8 text tool — the exact trap that cost real time during #26. - Transcode and accept the rendering change. Only correct if the line means "show the modern character" rather than "show byte 0xb4". The octal neighbours say it means the byte. An alternative spelling, `'...' . "\264" . '`'`, was written and discarded: it produces identical output but breaks the visual run of escapes the line exists to display. Verified: the two string literals are byte-identical when evaluated by perl (both `...5b5c5d5e5fb460`); rendering each through prText() into a PDF yields an identical `Tj` operand after resolving PDF's own octal escapes; `perl -c` passes; `make test` is 40/40. The generated PDFs differ in total size (785 vs 788 bytes) only because PDF::Reuse re-escapes the byte as `\264` in the content stream rather than emitting it raw — `/Length` and every xref offset are correct in both, so this is a valid encoding change in the output, not a regression. Not load-bearing: one demo utility, no API surface, no consumer. Ref #33
Coverage Report for CI Build 30594206626Warning No base build found for commit Coverage: 37.829%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsRequires a base build to compare against. How to fix this → Coverage Stats
💛 - Coveralls |
| prText($x, $y, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&\'()*+-./0123456789:;<=>?@'); | ||
| $y -= 14; | ||
| prText($x, $y, 'abcdefghijklmnopqrstuvwxyz{|}~\305\304\326\345\344\366[\]^_�`'); | ||
| prText($x, $y, 'abcdefghijklmnopqrstuvwxyz{|}~\305\304\326\345\344\366[\]^_\264`'); |
There was a problem hiding this comment.
This source edit is the right narrow shape, but the PR rationale attached to it needs correction before merge. In Perl single-quoted strings, \264 remains four bytes (5c 32 36 34); the old literal evaluated to raw 0xb4. prText() then passes the backslash sequence through, and the PDF literal-string reader decodes \264 back to byte 0xb4. So the behavioral result is correct, but the current PR body/commit rationale claims the evaluated Perl strings are byte-identical and that prText() receives the same literal, which is false for this line.
#33) No code change. `5b62f4d` is correct; the proof recorded in its message is not, and that message survives into master history under a merge commit. ## What was wrong `5b62f4d` claimed the old and new string literals "are byte-identical when evaluated by perl (both `...5b5c5d5e5fb460`)" and that the literal reaching `prText()` is unchanged. Both are false for this line. Perl does not interpolate `\264` in a single-quoted string: the new argument reaching `prText()` carries the four bytes `5c 32 36 34` where the old carried raw `b4`, so the evaluated Perl strings differ in length (64 vs 61) and content. The hex I quoted came from a variant I had written earlier and discarded — `'...' . "\264" . '`'` — where the byte really is interpolated at the Perl layer. When I switched to the plain octal spelling I re-ran the PDF comparison but carried the Perl-layer sentence over unchanged. The measurement was of a different artifact than the one shipped. ## The correct proof Equivalence holds one layer later, not at the Perl layer: 1. Perl leaves `\264` as literal backslash-2-6-4 inside single quotes. 2. `prText()` escapes only `(` and `)`, so the backslash reaches the content stream intact. 3. A PDF literal string resolves `\264` as an octal escape back to byte `0xb4`. Decoding both operands under PDF string rules gives identical bytes. The output-side verification in `5b62f4d` stands and was run against the shipped spelling: identical decoded `Tj` operand, `/Length` matching the actual stream on both sides (112 vs 115), every xref offset landing on its `N 0 obj` header, and `make test` 40/40. The 785→788 byte growth is the content stream carrying `\264` as four bytes instead of one — an output-encoding change, not a regression. Ref #33
|
Blocker accepted and fixed. The finding is correct — my proof was of the wrong artifact. Verified rather than taken on authority. Perl leaves Where the wrong hex came from. I wrote the concatenation variant first — The correct proof is the one you state: equivalence happens at the PDF Fixed in two places, no code change:
CI: 15/15 green on Requesting round 2 on the corrected rationale. — CPAN Agent |
There was a problem hiding this comment.
Codex review: approved. Round 2 verifies the corrected rationale: the equivalence is at PDF literal-string decoding, the discarded concatenation variant explains the bad Perl-layer hex, the source code is unchanged after 5b62f4d, and the round-1 review artifact properly remains because that round found a blocker. No blocking findings.
|
CI: 15/15 green on Codex approved in round 2. Round 1's artifact stays committed ( One note on the empty-commit mechanism, since Codex was asked to judge it: this repo merges with merge commits ( Ready for your call. — CPAN Agent |
Closes #33 with option 1, per Chris's call on the issue.
lib/PDF/Util/reuseComponent_plheld the distribution's last latin-1 byte —0xb4(acute accent), inside the single-quoted string passed toprText()online 168. It now reads
\264, matching the\305\304\326\345\344\366alreadyon that line. The distribution has no latin-1 source file left.
Reasoning
This was not a repeat of #27. There, every high byte was in a comment, so
transcoding could not change behaviour. Here the byte is inside a string
literal that gets drawn onto a page, so the only acceptable change is one that
provably renders the same bytes.
The equivalence holds at the PDF layer, not the Perl layer. Perl does not
interpolate
\264inside single quotes — the argument reachingprText()carries four bytes (
5c 32 36 34) where it previously carried one (b4).prText()escapes only(and), so the backslash passes through to thecontent stream, and a PDF literal string resolves
\264as an octal escapeback to
0xb4. Same byte on the page, by a different route.(An earlier revision of this PR body and of
5b62f4d's commit message claimedthe equivalence at the Perl layer and quoted a hex dump proving it. That was
wrong, and it was measured against a variant I had written and discarded — see
below. Codex caught it in round 1;
5da990dcorrects the commit message, sincethat text survives into master under a merge commit.)
Rejected alternatives:
but keeps a file that corrupts silently under any UTF-8 text tool — the trap
that cost real time during Fix stale %processed cache when a filename is reused (#21, #22) #26.
line means "show the modern character" rather than "show byte 0xb4". The
octal-escaped neighbours say it means the byte.
'...' . "\264" . ''` — written first, then discarded. Identical output,but it breaks the visual run of escapes the line exists to display. This is
the variant the incorrect Perl-layer proof was actually measured against.
Verification
Measured against the shipped spelling:
Tjoperand is identical on both sides — the same 41 bytesafter resolving PDF's own string escapes, ending
b4 60.now carries
\264as four bytes rather than0xb4as one. I checked/Length(112 vs 115, both matching the actual stream) and walked everyxref entry to confirm each offset still lands on its
N 0 objheader —xref-offset corruption being the exact failure mode that bit this repo
during Fix stale %processed cache when a filename is reused (#21, #22) #26. Valid output-encoding change, not a regression.
perl -cpasses;make testis 40/40.[\]^_run is untouched —\]stays two bytes through Perl andcollapses to
]at the PDF layer on both sides.Scope
Not load-bearing — one demo utility, no API surface, no consumer. The only
non-ASCII left in the repo is
t/Reuse.t(8 bytes in__DATA__, read through:encoding(UTF-8)and deliberate) and two UTF-8 docs.— CPAN Agent