Repository navigation
perf(io): bail out on oversized images before decoding - #2750
Merged
wkentaro merged 9 commits intoOct 16, 2026
Merged
Conversation
tongwentao
added a commit
to tongwentao/labelme
that referenced
this pull request
Oct 10, 2026
wkentaro
force-pushed
the
perf/read-image-file-header-check-2418
branch
from
October 15, 2026 14:36
89963f3 to
2a83a7d
Compare
wkentaro
pushed a commit
to tongwentao/labelme
that referenced
this pull request
Oct 15, 2026
Owner
Known caveat: oversized planar TIFFs that previously produced garbled strips can now be rejected; the existing planar decoder bug remains. |
read_image_file() eagerly decoded and re-encoded oversized images (such as large GeoTIFFs exceeding 32767 pixels per side) into PNG/JPEG in memory, costing tens of seconds and gigabytes of memory only to have Qt reject the decoded image anyway. Check image dimensions lazily from the image header (PIL image_pil.size or tifffile IFD page shape) before decoding pixels. If either dimension exceeds the Qt raster engine's 32767 pixel limit, immediately raise ImageTooLargeError without decoding pixels or re-encoding. Closes wkentaro#2418
For a planar-separate TIFF, page.shape is (samples, height, width), so shape[:2] compared the band count and height and missed an oversized width, falling back to the full decode. Read imagelength and imagewidth from the first page instead.
Move the translated too-large message into one helper and use the RASTER_MAX_SIDE constant directly instead of an alias in _app.py.
read_label_file wraps ImageTooLargeError in LabelFileReadError, so the dialog missed the isinstance check and showed the exception's English text. Unwrap the cause, and keep the exception text to the plain fact now that the dialog always supplies the translated advice.
Fragments are named by PR number, and the entry now describes what a user sees instead of the internal read_image_file function.
Qt decodes and paints images wider than 32767 pixels when they fit the allocation limit, so rejecting every such image refused a 40000x100 PNG that opens on main. main refuses an image only when a side exceeds 32767 and the decode exceeds the allocation limit, because the app does not offer to raise the limit in that case. Check that same condition before decoding, using a lower bound on Qt's bytes per pixel so the check never rejects an image Qt would accept.
The early error now covers only images over both the per-side and the decode limits, not every image wider or taller than 32767 pixels.
_imread_tiff now decodes through TiffPageSeries.asarray instead of tifffile.imread, so patching imread no longer proved that the early check runs before any pixels are decoded.
wkentaro
force-pushed
the
perf/read-image-file-header-check-2418
branch
from
October 16, 2026 00:38
828131b to
3b0885f
Compare
Owner
The original GeoTIFF benchmark and full-suite/lint claims were not independently reproduced. The conservative estimate can still let some oversized images reach decoding before Qt rejects them. |
alfiethology
pushed a commit
to alfiethology/labelme
that referenced
this pull request
Oct 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #2418.
read_image_filenow reads the image size from the header and raisesImageTooLargeErrorbefore decoding any pixels whenmainwould refuse the image anyway: a side over Qt's 32767 px raster limit and a decode overQImageReader.allocationLimit(). In that case the app does not offer to raise the limit, so the decode was wasted.bytes_per_pixelis a lower bound on Qt's decoded format (1/8 for 1-bit, otherwise 1), so the check never rejects an image Qt would accept. A 40000 x 100 PNG still opens, as it does onmain.The dialog shows the existing translated "too large" message whether the user opens the image directly or through a label JSON.
read_label_filewraps the error inLabelFileReadError, so_show_file_open_errorunwraps__cause__.Evidence
main, the 37296 x 49319 GeoTIFF from Misleading "Allowed formats" error when the decoded image exceeds Qt's allocation limit #2388 took about 36 s and peaked at 7.4 GB RSS before Qt rejected it (numbers from read_image_file eagerly decodes and re-encodes the whole image, costing ~36s and ~7.4 GB on a large GeoTIFF #2418).After: the PNG and TIFF tests use files whose headers claim 37296 x 49319 but contain no pixel data. They get
ImageTooLargeErrorwithout decoding. The planar TIFF test fails the test iftifffile.imreadis called.mainor on an earlier revision of this branch:just lintpasses.Merge Danger
Door: two-way
Blast Radius: small
The early error only fires when the size, read from the header, means Qt would refuse to decode.