Cut the downscaled pano copy on demand, at the width the viewer asks for

[jonfroehlich]

Fixes #5256.

Why

We've precomputed a downscaled copy of every wide panorama twice now — nightly in the app (OOM-killed prod JVMs, #5239) and at stitch time in the scraper, which needs a long backfill before it helps at all. Both spend that work up front, across the board, on frames that are individually almost never looked at — so precomputing cuts overwhelmingly more copies than anyone ever asks for.

And the threshold was wrong by 2x. Pannellum uploads an equirect as two halves, so its refusal test is max(width/2, height) > MAX_TEXTURE_SIZE — a device advertising 8192 renders a 16384-wide pano, exactly the widest GSV produces. Most hardware never needed a copy, and no fixed server-side cap can know which does.

What this does

The viewer asks. PannellumViewer reads 2 * MAX_TEXTURE_SIZE from a throwaway context (cached, released immediately) and appends ?maxWidth= only when the stored pano exceeds it. Every device that can render the pano as stored gets it untouched, and there's no failed first load to recover from — the device's limit is knowable before the request, so this never depends on parsing Pannellum's localized error string.

The server cuts it on demand. PanoDisplayCopyService produces that width and caches it under the crop store (derived, disposable, app-writable) at pano-display/<xx>/<panoId>.w<width>.jpg — a new directory, so the crop job's retired pano-downscaled/ stays unambiguously deletable on prod.

The scraper sidecar path is removed, not kept alongside. /backupImage could serve a <panoId>.w8192.jpg written beside the native file, and the nightly crop job counted how many wide panos were missing one, warning when any were. Both rested on the same 2x-wrong threshold, and on a scraper-side writer that does not exist — ProjectSidewalk/sidewalk-panorama-tools#115, the issue that would have produced these files, was closed unbuilt. So the count was 100% missing forever, warning on every nightly background_job_run row about devices that either render the native frame fine or now get a copy cut at the width they asked for.

That drops pano.downscaled.max-width, PanoDataService's three sidecar members, PanoDataTable.getWideBackupPanos (its only caller was the count), and CropService's countSidecars with its four run-result fields and JSON keys. /backupImage without ?maxWidth= now serves the native file. Nothing here touches the scraper or EC2.

The measurement this rests on

Benchmarked on JDK 17 against a real 16384x8192 Chicago pano:

approach	peak heap	min -Xmx	time
the code that OOM'd (#5239)	~390 MB	—	~10 s
setSourceSubsampling(2,2,0,0)	~105 MB	112 MB	2.0 s
strip read + hand box filter	523 MB	—	12.5 s

Letting the JPEG decoder subsample does the reduction inside the decode, so no full-size raster and no rescale pass ever exist. 3.5x less memory and 5x faster than the strip-and-getScaledInstance code that took prod down, and ~7% of a 1.5 GB heap. 112m is near the floor, since the 8192x4096 3BYTE_BGR output is 100 MB by itself.

Also settled by measurement so it stops being re-litigated: ImageIO cannot stream a JPEG encode. Given a genuinely multi-tile RenderedImage, JPEGImageWriter requests zero tiles and calls getData() for the whole raster. (A single-tile probe looks like streaming — getTile()=1 — which is what makes this easy to get wrong.) It doesn't matter here: 112 MB doesn't need streaming.

The cost is quality. Subsampling drops pixels where an area average would blend them — PSNR ~30.6 dB against a box-filtered reference. Accepted deliberately given how rarely this runs; @jonfroehlich signed off on the trade.

Guardrails

• Concurrency capped at 2, on a dedicated pool. Cuts run on their own two-thread executor rather than cpu-intensive, and the bound is the pool itself rather than a permit a thread blocks on — without a cap a burst puts several 105 MB allocations in a 1.5 GB heap at once, Nightly crop job OOM-kills prod JVMs: move the downscaled-pano derivative to the scraper, delete it from the app #5239's shape from the other direction. Four may queue behind them; past that a request is refused outright and served the native file. A device asking for a copy is one that already failed to texture the native frame, and it has its own ladder of smaller widths to walk, so a fast refusal sends it down that ladder while a long wait would only delay the same outcome.
• Single-flight per (pano, width), so a burst on one pano cuts once.
• Widths snapped to an allowlist. The HMAC covers the path, not the query, so without this anything holding a signed URL could mint unbounded widths and fill the store. Snapped down rather than rejected, so an odd value still yields something the device can render.
• Every failure path answers None and serves the native file — which is what the route did before this existed.

Tests

7 new cases: the period arithmetic (including a non-2:1 pano where the height binds, and the 16384-on-an-8192-device case that needs no copy), the allowlist invariant, and a real subsampled read and JPEG write. 51 existing crop/backupImage specs still pass; ESLint clean.

Docs

Corrects PanoDataService, CropService and docs/architecture.md, which all asserted a whole-pano derivative needs more heap than a city stage has. It doesn't — precomputing one for the whole store, nightly, was the problem.

🤖 Generated with Claude Code — Opus 5 (1M context), claude-opus-5[1m]