Context
While building the JRC European Flood Hazard Map backend in earthlens (serapeum-org/earthlens#958), Dataset.crop(bbox=)
could not be used against the EFHM rasters (110162 x 51992) served over /vsicurl: it triggers a full read of the
source, which hung over the network. The backend had to hand-roll read_array(window=) + GeoTIFF write instead.
Problem / Current Behaviour
Dataset.crop(bbox=) reads the entire source array before cropping. For a very large or network-backed (/vsicurl
https) raster this is prohibitively slow or hangs, defeating the point of a spatial crop. Every consumer reading a
small AOI out of a large remote raster must re-implement windowed reads.
Steps to Reproduce
from pyramids.dataset import Dataset
ds = Dataset.read_file("/vsicurl/https://<host>/efhm_rp100.tif") # 110162 x 51992
ds.crop(bbox=[west, south, east, north]) # full read of the source -> hangs over the network
Proposed Solution
# Offer a windowed crop that reads only the AOI window straight from the path/URL:
Dataset.crop_from_path(src, bbox=[...], out_path=...) # computes the pixel window, reads only it
# or an option on the existing API:
ds.crop(bbox=[...], windowed=True) # read_array(window=) under the hood
So consumers need not hand-roll read_array(window=) + origin math + GeoTIFF write.
Out of Scope
- The earthlens-side consolidation of the current hand-rolled helpers (tracked in earthlens).
Effort Estimate
Size: M
Rationale: window/geotransform math plus a new API path and tests; self-contained in pyramids.
Definition of Done
Context
While building the JRC European Flood Hazard Map backend in earthlens (serapeum-org/earthlens#958),
Dataset.crop(bbox=)could not be used against the EFHM rasters (110162 x 51992) served over
/vsicurl: it triggers a full read of thesource, which hung over the network. The backend had to hand-roll
read_array(window=)+ GeoTIFF write instead.Problem / Current Behaviour
Dataset.crop(bbox=)reads the entire source array before cropping. For a very large or network-backed (/vsicurlhttps) raster this is prohibitively slow or hangs, defeating the point of a spatial crop. Every consumer reading a
small AOI out of a large remote raster must re-implement windowed reads.
Steps to Reproduce
Proposed Solution
So consumers need not hand-roll
read_array(window=)+ origin math + GeoTIFF write.Out of Scope
Effort Estimate
Size:
MRationale: window/geotransform math plus a new API path and tests; self-contained in pyramids.
Definition of Done
/vsicurlURL (no full read)cropbehaviour unchanged