feat(resizer): scope the cache key by the source's upload path - #153
feat(resizer): scope the cache key by the source's upload path#153parisek wants to merge 16 commits into
Conversation
Agent review — Claude (opus) and Codex (gpt-5-codex), 2026-08-27Two independent reviewers, different models, neither able to see the other's output. Recorded here because the rejected and deferred findings are the ones the diff cannot explain. They converged on nothing. That is worth stating plainly rather than hiding: it describes the lenses, not the findings. Claude was asked to verify the fix works; Codex was asked to find where it still does not. Both did their job and returned disjoint sets. What Codex found that seven Claude review rounds did notThe branch originally added only the source's directory to the cache key. Codex found that closes half the problem: the derivative is named for the source without its extension, so Measured on production before acting: So the first version fixed 344 of 496 while the ADR, CHANGELOG and README all claimed the job was done — and the ADR's own opening example ( The key now carries the source's whole sanitized identity, directory and full filename: Also found and fixed
RejectedCodex rated "sanitization can collapse two distinct source names" as High — Rejected, and Codex was asked to challenge the rejection and agreed with it. Measured zero occurrences of Deferred
A reviewer error worth recordingCodex's first pass called the Tests 1750 + 18 green, PHPStan clean, ADR index in sync, CI 6/6. |
…nd misreporting root uploads as conflicts buildNameToDirs() silently dropped an attachment whose upload directory failed guardSourceDir(), instead of folding it into the flat-key '' candidate the way a genuine root upload does. A same-named clean attachment then looked unambiguous and plan() moved the wrong attachment's derivative into it. plan() also built a target path from dirs[0] even when it was '' (a root upload), which collapses via the double slash to the source file itself: is_file() was always true, so every root-upload derivative was reported as a conflict with itself. plan() now recognises dirs[0] === '' as already in its final location and reports it in a new already_in_place bucket instead. MigrateImageCacheCommand's summary output and ImageCacheMigrator::apply()'s docblock type were updated to match the new bucket.
…a leftover plan note guardSourceDir() (MigrateImageCacheCommand) and guarded_cached_derivative_source_dir() (StarterBase) both split on '/' without normalizing backslashes first, unlike Resizer::sourcePathSegment(), which reads a URL. Both guards read dirname() of _wp_attached_file, a database value WordPress always stores forward-slash, so a backslash cannot reach either guard; documented that with a matching comment in both methods rather than adding unreachable normalization code. Also removes an authoring-plan leftover sentence from SourcePathCacheKeyTest's cacheKeysFor() docblock.
…sion Retains the source's own extension in the derivative filename whenever StarterBase::$resizer_source_path_in_cache_key is on, so hero.jpg and hero.png sharing one upload directory no longer collide on one derivative name -- the directory-only fix left 152 of 496 measured production collisions (same directory, different extension) unaddressed. Flag off stays byte-identical to today. Updates the four components that must agree on the new shape: the writer (Resizer::resizer()), the deleter (StarterBase::cached_derivative_paths_by_source_path()), and the migrator's map (MigrateImageCacheCommand::buildNameToSourcePaths(), renamed from buildNameToDirs()) and planner (ImageCacheMigrator::plan()). The migrator's map now carries full source paths, not bare directories, because the target filename needs the source's own extension; two sources sharing a directory and stem but not an extension are recorded as distinct candidates, so a legacy flat derivative they both map to is reported ambiguous and left unmigrated rather than guessed. Also, from the same review round: - ImageCacheMigrator::apply() moves via link()+unlink() instead of an is_file() check followed by rename() -- POSIX rename() silently overwrites an existing destination, and a target could appear in the gap between the check and the syscall. link() fails atomically when the target exists, leaving both files untouched; only a successful link() is followed by unlink() of the source. - MigrateImageCacheCommand::__invoke() now warns about every failed move, then calls WP_CLI::error() (non-zero exit) when any failed, and WP_CLI::success() only when none did -- it previously called success() unconditionally before the failure warnings, so a script harness reading only the exit code saw 0 on a run that left files unmoved. CHANGELOG.md and README.md corrected to state precisely what the flag resolves (directory and same-directory-extension collisions) versus what it cannot (ambiguous flat-legacy names, left unmigrated). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Su5MBeKhMxER2XEcba6Qpt (petr@pari.cz)
Resizer sanitized the still-encoded URL basename when building the source-path cache key, while StarterBase::cached_derivative_paths_by_source_path() and MigrateImageCacheCommand's map builder both name the source from the decoded _wp_attached_file value. A space or percent-encoded character made the writer and the deleter/migrator disagree on the derivative's name -- 230 _wp_attached_file values on the measured production site contain a space. Strip the query string and rawurldecode() the basename once before sanitizing, mirroring what Resizer::sourcePathSegment() already does per path component for the directory half. The flag-off branch is untouched.
…gacy lookup A root upload's migrated derivative (hero.png.avif) sits at the same depth in the size directory as an unmigrated legacy one, so a second migration run re-scanned it, failed the legacy-name lookup (the map is keyed by un-migrated names), and reported it orphaned. Index every root-upload source by its own full name (extension included) and merge that into the legacy lookup before counting matches. A name resolving to the same single source under both interpretations stays unambiguous; a name resolving to two distinct identities -- a root upload and an unrelated legacy source whose stripped name happens to coincide, ADR 0007's aliasing case -- is reported ambiguous rather than guessed. Also asserts the full plan (not just `move`) is empty on the existing second-run idempotence test, since an orphaned/ambiguous bucket populated would have passed the old, narrower assertion.
The old-name description in Context contradicted the correct one already given lower in the same file (sanitize_file_name(basename()) vs the actual pathinfo(..., PATHINFO_FILENAME)). The Decision section claimed the new key needs "no filename parsing", which the encoded- filename fix in this branch makes explicitly false. The byte-identical- path claim for folder-less sites didn't hold for a root upload with an extension of its own -- only an extensionless root source produces one. Consequences now also states the key's sanitization-derived (not whole-identity) collision limit, and the idempotence claim reflects the already-migrated-root-upload fix.
f11e122 to
c3775b8
Compare
Rollout notes — measured scope, and what to do when you come back to thisWritten for whoever picks this up later, including me. Everything below is measured on Merging this changes nothing
So this can sit merged and dormant. Nothing reaches a site until someone opts in and bumps the package. Contrast with #152, already merged: that one is not behind a flag, deliberately, because gating a data-loss fix would leave the loss live by default. It applies on the next version bump. How much is actually broken todayThe bug this PR fixes is not hypothetical — it is live right now. But the blast radius is smaller than the raw collision count suggests, and the difference matters for scheduling.
90 is the ceiling on visible damage. A collision in the database harms nothing by itself; a wrong image can only be served where the shared derivative physically exists. Ninety cache files are currently standing in for two or more different source images each. What those 90 areAlmost all of them are auto-generated names — pasted screenshots and import artefacts: No A looser check over ACF placements (attachment IDs in Conclusion: real, not acute. The risk concentrates in inline article images, not in page furniture. That is also why nobody has reported it in three years — one screenshot looks like another, and nothing here gets compared side by side. It is the dangerous category: a failure that renders as a plausible success. When you turn it onPrerequisite: the Cloudways AVIF encoder must be fixed first. Its ImageMagick 6.9.11 (2021) drops the alpha channel, so anything regenerated there comes out with a black background — see Then, in order:
Unproven, and worth knowing before you rely on itThe migration command has never run against real data. It has unit tests over synthetic directory trees and two independent reviews, and three Critical defects were caught and fixed in it — but zero production mileage. The dry-run in step 2 is its first real exercise; treat the first run as an experiment, not a formality. Still open on this branchThree Minor findings from the Codex review of #152 were recorded there and not fixed: the SQL contract test does not pin the table's position in the query, |
From project
sloneek. A homepage hero rendered with a black background. The cause turned out to be two independent defects, and this PR is the second:Resizernames a derivative after its source's basename alone, so two uploads that share a name share one cached file.The measurement
On that site's staging database:
The cache namespace is flatter than the namespace it caches, twice over — the derivative is named for the source without its extension, so
11.pngand11.jpgcollide as surely as two11.pngin different months do:Whichever renders first writes the file; the rest read it and get a picture of something else. Nothing errors, and the result is a plausible image, so it stays invisible until someone recognises the wrong photograph on a page.
What changed
The source's own upload directory now sits between the size directory and the filename:
Behind
StarterBase::$resizer_source_path_in_cache_key, defaultfalse, mirroring the$resizer_quality_in_cache_keyflag it sits beside. It is derived from the source URL — not the resolved file path — becauseresize()needs the value before it knows whether the file exists locally: a missing source is handed to DevMediaProxy, which addresses the same cache path on another host.The segment is written into
$variant['cache_key'], not into the target path.cache_keyis already the one value both the local writer andDevMediaProxy::build_remote_resizer_variant()derive from, and the comment atDevMediaProxy.php:456-460names that as deliberate. Appending inprocessVariant()would have left the remote path in the old shape.wp timber-kit migrate-image-cachemoves an existing cache into the new shape rather than re-encoding it. Dry-run by default;--applywrites. It moves rather than copies (no transient doubling), re-checks the target immediately before everyrename(), never deletes, never overwrites, and is idempotent so an interrupted run is resumed by running it again.Derivatives whose name maps to more than one source cannot be placed — recovering that association is exactly what the flat layout destroyed. They are reported and left alone, and re-encoded on first view.
With the flag on,
cleanup_cached_images()addresses derivatives by computed path instead of matching basenames across the tree. That closes the delete half of the same bug. The sibling-attachment guard from #152 stays in force — it answers a different question and is independent of the layout.A site with year/month folders switched off gets byte-identical paths and needs no migration at all. The suite pins that.
Review
Every task was reviewed independently before it landed. Three Critical defects were caught this way and are fixed in the branch, all of the same shape — a check performed at one moment and relied on at another, where the gap is silent:
apply()re-usedplan()'s conflict check, and POSIXrename()silently replaces its destination. A target written between plan and apply was destroyed with no error and no entry infailed._wp_attached_fileand used it as aglob()pattern, while the writer sanitizes it. Deletinga[1].pngremoved unrelateda1.png's derivative, because[1]is a glob character class. Names carrying stripped characters silently no-opped and leaked instead.buildNameToDirs()dropped a guard-rejected attachment from the map entirely, which could make a genuine collision read as unambiguous — and move one attachment's derivative into another's directory. The reopened swapped-image bug, inside the tool operators run to make the flag-flip safe.Each has a regression test that fails against the code as it was.
Three components now implement "is this path component safe to use as a directory" — the writer, the migrator and the deleter. They are deliberately not coupled; the final review compared all three against each other, since any input they disagree on is a bug.
Deliberately not done
Helpers::resizeImage()— the legacy path predatingResizer— keeps the flat layout. Not migrated, not flagged.composer update. This differs from fix(media): keep cached derivatives a sibling attachment still uses #152, where I argued against a flag because nothing was at stake.Decision, rejected alternatives and consequences:
docs/adr/0007-resizer-source-path-cache-key.md, which lands in this PR.