Repository navigation
Convert non-device image samples without whole-image intermediates - #562
Merged
Merged
Conversation
Reading a 12608x16806 JPEG 2000 image in an ICCBased space with a soft mask peaked at 5.5 GB: every sample was widened to f32 before color conversion, and the RGBA buffer was copied again to apply the mask. Samples are now converted to RGBA one pixel at a time, the soft mask is multiplied into the decoded buffer in place, and an ICCBased space whose alternate is a device space is decoded as that device space, which is how it is drawn until ICC color management exists. The read now peaks at 2.2 GB and takes 8 s instead of 57 s in the regression build. Fixes #520 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conformance readsDocuments that Safe-PDF reads differently from the base commit, compared with PDFium. A read pdf.js corpus1008 documents compared with Regressions
Improvements
840 documents of the PDFium corpus read as on Details are in the job summaries of the workflow run. Rerun a case locally with |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #520
Cause
issue19517(pdf.js corpus) is one page with a 12608×16806 JPEG 2000 image in[/ICCBased](alternate DeviceRGB) and a same-size soft mask. Reading it peaked at 5.5 GB, above the 3 GB worker limit:Vec<f32>(~2.5 GB) before color conversion, in both the JPX and the plain-sample pathsimage_from_rgbacopied the full RGBA buffer (~850 MB) to multiply in the soft maskFix
pdf-image: aComponentDomainconverts samples to RGBA one pixel at a time, which replaces both whole-imagef32buffers.image_from_rgbareuses the uniquely owned buffer (Vec::from(Bytes)) instead of copying it.pdf-color-space:ColorSpace::device_equivalent. An ICCBased space whose alternate is a device space (or that has no alternate) is decoded as that device space. That is exactly whatICCBasedColorSpace::applyalready draws, so the per-pixel conversion is skipped for such images.pdf-decode:DecodeMap::apply_to_f32now writes into a caller-provided slice, so it works per pixel.Peak RSS 5.5 GB → 2.2 GB. Read time in the regression build 57 s → 8 s. The memory fix alone timed out at 65 s, which is why the ICC shortcut is included.
Verification
cargo conformance verify conf2-bb565df26b2e: ✓ passcargo conformance render: the page shows the full-page red image the upstream report expectscargo test,cargo clippy --all --workspace -- -D warnings,cargo fmtNo new unit test was added, following this workspace's practice of not adding new tests. The existing
pdf-imageandpdf-decodetests cover the Lab, Indexed, DeviceN-alpha,/Decodeand soft-mask paths.🤖 Generated with Claude Code