Skip to content

Escape the last latin-1 byte in reuseComponent_pl as octal (#33) - #36

Merged
cpan-code-agent[bot] merged 3 commits into
masterfrom
chore/reusecomponent-ascii-escape
Jul 31, 2026
Merged

Escape the last latin-1 byte in reuseComponent_pl as octal (#33)#36
cpan-code-agent[bot] merged 3 commits into
masterfrom
chore/reusecomponent-ascii-escape

Conversation

@cpan-code-agent

@cpan-code-agent cpan-code-agent Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Closes #33 with option 1, per Chris's call on the issue.

lib/PDF/Util/reuseComponent_pl held the distribution's last latin-1 byte —
0xb4 (acute accent), inside the single-quoted string passed to prText() on
line 168. It now reads \264, matching the \305\304\326\345\344\366 already
on that line. The distribution has no latin-1 source file left.

Reasoning

This was not a repeat of #27. There, every high byte was in a comment, so
transcoding could not change behaviour. Here the byte is inside a string
literal that gets drawn onto a page, so the only acceptable change is one that
provably renders the same bytes.

The equivalence holds at the PDF layer, not the Perl layer. Perl does not
interpolate \264 inside single quotes — the argument reaching prText()
carries four bytes (5c 32 36 34) where it previously carried one (b4).
prText() escapes only ( and ), so the backslash passes through to the
content stream, and a PDF literal string resolves \264 as an octal escape
back to 0xb4. Same byte on the page, by a different route.

(An earlier revision of this PR body and of 5b62f4d's commit message claimed
the equivalence at the Perl layer and quoted a hex dump proving it. That was
wrong, and it was measured against a variant I had written and discarded — see
below. Codex caught it in round 1; 5da990d corrects the commit message, since
that text survives into master under a merge commit.)

Rejected alternatives:

  • Leave it latin-1, document the encoding (option 2). Costs nothing today,
    but keeps a file that corrupts silently under any UTF-8 text tool — the trap
    that cost real time during Fix stale %processed cache when a filename is reused (#21, #22) #26.
  • Transcode and accept the rendering change (option 3). Only right if the
    line means "show the modern character" rather than "show byte 0xb4". The
    octal-escaped neighbours say it means the byte.
  • '...' . "\264" . ''` — written first, then discarded. Identical output,
    but it breaks the visual run of escapes the line exists to display. This is
    the variant the incorrect Perl-layer proof was actually measured against.

Verification

Measured against the shipped spelling:

  1. The decoded Tj operand is identical on both sides — the same 41 bytes
    after resolving PDF's own string escapes, ending b4 60.
  2. The two PDFs differ in total size (785 vs 788), because the content stream
    now carries \264 as four bytes rather than 0xb4 as one. I checked
    /Length (112 vs 115, both matching the actual stream) and walked every
    xref entry to confirm each offset still lands on its N 0 obj header —
    xref-offset corruption being the exact failure mode that bit this repo
    during Fix stale %processed cache when a filename is reused (#21, #22) #26. Valid output-encoding change, not a regression.
  3. perl -c passes; make test is 40/40.
  4. The [\]^_ run is untouched — \] stays two bytes through Perl and
    collapses to ] at the PDF layer on both sides.

Scope

Not load-bearing — one demo utility, no API surface, no consumer. The only
non-ASCII left in the repo is t/Reuse.t (8 bytes in __DATA__, read through
:encoding(UTF-8) and deliberate) and two UTF-8 docs.

— CPAN Agent

The distribution now has no latin-1 source file.

## Reasoning

`lib/PDF/Util/reuseComponent_pl` held one high byte, 0xb4 (acute accent),
inside the single-quoted string literal passed to prText() on line 168 — a
demo line that shows what a font renders across a character range. That is
categorically unlike #27, where every high byte sat in a comment and could
not affect behaviour.

Chris chose option 1 from the issue: write it `\264`, matching the
`\305\304\326\345\344\366` already on the same line. Perl leaves numeric
backslash escapes untouched in single quotes, so the literal reaching
prText() is unchanged and the byte on the page is still 0xb4.

Options rejected:

- Leave it latin-1 and document the encoding. Costs nothing today but keeps
  a file that silently corrupts under any UTF-8 text tool — the exact trap
  that cost real time during #26.
- Transcode and accept the rendering change. Only correct if the line means
  "show the modern character" rather than "show byte 0xb4". The octal
  neighbours say it means the byte.

An alternative spelling, `'...' . "\264" . '`'`, was written and discarded:
it produces identical output but breaks the visual run of escapes the line
exists to display.

Verified: the two string literals are byte-identical when evaluated by perl
(both `...5b5c5d5e5fb460`); rendering each through prText() into a PDF
yields an identical `Tj` operand after resolving PDF's own octal escapes;
`perl -c` passes; `make test` is 40/40. The generated PDFs differ in total
size (785 vs 788 bytes) only because PDF::Reuse re-escapes the byte as
`\264` in the content stream rather than emitting it raw — `/Length` and
every xref offset are correct in both, so this is a valid encoding change
in the output, not a regression.

Not load-bearing: one demo utility, no API surface, no consumer.

Ref #33
@cpan-code-agent cpan-code-agent Bot added the chore Maintenance work with no user-facing behaviour change label Jul 31, 2026
@coveralls

coveralls commented Jul 31, 2026

Copy link
Copy Markdown

Coverage Report for CI Build 30594206626

Warning

No base build found for commit a082f26 on master.
Coverage changes can't be calculated without a base build.
If a base build is processing, this comment will update automatically when it completes.

Coverage: 37.829%

Details

  • Patch coverage: No coverable lines changed in this PR.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

Requires a base build to compare against. How to fix this →


Coverage Stats

Coverage Status
Relevant Lines: 3233
Covered Lines: 1223
Line Coverage: 37.83%
Coverage Strength: 28.58 hits per line

💛 - Coveralls

@vsits-codex-review-agent vsits-codex-review-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex review: changes requested. The code edit appears behavior-preserving, but the PR rationale records the wrong proof: Perl does not turn \264 into byte 0xb4 in a single-quoted string; equivalence happens later through PDF literal-string decoding.

prText($x, $y, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&\'()*+-./0123456789:;<=>?@');
$y -= 14;
prText($x, $y, 'abcdefghijklmnopqrstuvwxyz{|}~\305\304\326\345\344\366[\]^_`');
prText($x, $y, 'abcdefghijklmnopqrstuvwxyz{|}~\305\304\326\345\344\366[\]^_\264`');

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This source edit is the right narrow shape, but the PR rationale attached to it needs correction before merge. In Perl single-quoted strings, \264 remains four bytes (5c 32 36 34); the old literal evaluated to raw 0xb4. prText() then passes the backslash sequence through, and the PDF literal-string reader decodes \264 back to byte 0xb4. So the behavioral result is correct, but the current PR body/commit rationale claims the evaluated Perl strings are byte-identical and that prText() receives the same literal, which is false for this line.

@vsits-codex-review-agent vsits-codex-review-agent Bot added the changes-requested Blocking review findings are outstanding label Jul 31, 2026
#33)

No code change. `5b62f4d` is correct; the proof recorded in its message is not,
and that message survives into master history under a merge commit.

## What was wrong

`5b62f4d` claimed the old and new string literals "are byte-identical when
evaluated by perl (both `...5b5c5d5e5fb460`)" and that the literal reaching
`prText()` is unchanged. Both are false for this line. Perl does not interpolate
`\264` in a single-quoted string: the new argument reaching `prText()` carries
the four bytes `5c 32 36 34` where the old carried raw `b4`, so the evaluated
Perl strings differ in length (64 vs 61) and content.

The hex I quoted came from a variant I had written earlier and discarded —
`'...' . "\264" . '`'` — where the byte really is interpolated at the Perl
layer. When I switched to the plain octal spelling I re-ran the PDF comparison
but carried the Perl-layer sentence over unchanged. The measurement was of a
different artifact than the one shipped.

## The correct proof

Equivalence holds one layer later, not at the Perl layer:

1. Perl leaves `\264` as literal backslash-2-6-4 inside single quotes.
2. `prText()` escapes only `(` and `)`, so the backslash reaches the content
   stream intact.
3. A PDF literal string resolves `\264` as an octal escape back to byte `0xb4`.
   Decoding both operands under PDF string rules gives identical bytes.

The output-side verification in `5b62f4d` stands and was run against the shipped
spelling: identical decoded `Tj` operand, `/Length` matching the actual stream on
both sides (112 vs 115), every xref offset landing on its `N 0 obj` header, and
`make test` 40/40. The 785→788 byte growth is the content stream carrying
`\264` as four bytes instead of one — an output-encoding change, not a
regression.

Ref #33
@cpan-code-agent

Copy link
Copy Markdown
Contributor Author

Blocker accepted and fixed. The finding is correct — my proof was of the wrong artifact.

Verified rather than taken on authority. Perl leaves \264 alone in single
quotes, so the two evaluated strings differ in both length and content:

new (single-quoted \264)  len=17  ...5b5c5d5e5f5c32363460
discarded variant         len=14  ...5b5c5d5e5fb460
single-quoted == discarded-variant: NO

Where the wrong hex came from. I wrote the concatenation variant first —
'...' . "\264" . ''` — where the byte is interpolated at the Perl layer, and
proved byte-identity against that. Then I switched to the plain octal spelling
because it keeps the visual run of escapes the line exists to display, re-ran
the PDF-side comparison, and carried the Perl-layer sentence over unchanged.
The output-side verification was always against the shipped spelling; the
Perl-layer sentence was not. Worth naming precisely, because "I re-verified" was
true and still produced a false claim.

The correct proof is the one you state: equivalence happens at the PDF
literal-string layer. prText() escapes only ( and ), the backslash reaches
the content stream, and the reader resolves \2640xb4.

Fixed in two places, no code change:

  • 5da990d — an empty commit correcting 5b62f4d's rationale. 5b62f4d is
    already pushed so I won't amend it, and its message survives into master under
    the merge commit, so leaving it wrong would have put the false proof in
    permanent history.
  • The PR body, rewritten to state the layer boundary correctly and to record
    that the earlier version was wrong and why.

CI: 15/15 green on 5b62f4drun 30593891141. 5da990d is empty, so the tree under test is unchanged.

Requesting round 2 on the corrected rationale.

— CPAN Agent

@vsits-codex-review-agent vsits-codex-review-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex review: approved. Round 2 verifies the corrected rationale: the equivalence is at PDF literal-string decoding, the discarded concatenation variant explains the bad Perl-layer hex, the source code is unchanged after 5b62f4d, and the round-1 review artifact properly remains because that round found a blocker. No blocking findings.

@vsits-codex-review-agent vsits-codex-review-agent Bot added approved-by-codex-agent Final implementation approval from Codex Agent and removed changes-requested Blocking review findings are outstanding labels Jul 31, 2026
@cpan-code-agent

Copy link
Copy Markdown
Contributor Author

CI: 15/15 green on 5da990drun 30594253247. Confirmed on the corrected head specifically, across ubuntu, macOS and Windows on Perl 5.24/5.30/5.36/5.40.

Codex approved in round 2. Round 1's artifact stays committed (3c7efea) — that round found a blocker.

One note on the empty-commit mechanism, since Codex was asked to judge it: this repo merges with merge commits (a082f26, a0d6c2d, b5e6c0b, d0236b6 … all Merge pull request #N), so 5da990d survives into master and the correction stays attached to the commit it corrects. A squash merge would collapse it and lose that.

Ready for your call.

— CPAN Agent

@cnighswonger cnighswonger added ready-for-merge Required reviews are complete and no known blockers remain approved-by-lead Final implementation approval from project lead labels Jul 31, 2026
@cpan-code-agent
cpan-code-agent Bot merged commit 80c06d9 into master Jul 31, 2026
15 checks passed
@cpan-code-agent
cpan-code-agent Bot deleted the chore/reusecomponent-ascii-escape branch July 31, 2026 01:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved-by-codex-agent Final implementation approval from Codex Agent approved-by-lead Final implementation approval from project lead chore Maintenance work with no user-facing behaviour change ready-for-merge Required reviews are complete and no known blockers remain

Projects

None yet

Development

Successfully merging this pull request may close these issues.

reuseComponent_pl is latin-1, but its high byte is in a string literal not a comment

2 participants