Cache key moved on an rlang upgrade, silently orphaning every cached cube and fetch #48

Both frozen cache-key goldens fail on main

test-dft_stac_cube.R:62   expected "638a2be11fdf"   actual "45685ccbda33"
test-dft_stac_fetch.R:88  (same class, golden "79f67b7b9dae")

Verified against a clean main worktree, not a feature branch: main is
FAIL 2 | PASS 407. stac_cube_cache_key() and stac_cache_key() are
byte-identical to the commits that pinned those hashes, so nothing in drift
changed — this is environment drift underneath rlang::hash()
(sf / PROJ / R serialization version).

Why it matters more than a red test

The hash is the cache filename. <source>/cube_<key>.tif and
<year>_<key>.nc. So when the key moved, every cached cube and every cached
fetch in every user's dft_cache_path() was silently orphaned: nothing hits
cache any more, and every call re-streams from the Planetary Computer.

Each cube costs 10–30 minutes to rebuild. There is no error, no warning, and no
log line — the symptom is "the pipeline got slower for no reason", which is
exactly the kind of thing nobody files.

The tests were doing their job. They are the only reason this is known.

What NOT to do

Do not re-pin the goldens to the current values. That is loosening an assertion
until it stops failing, and it would discard the one signal that says the key
moved. The tests' own comments say the hash must be stable precisely because
caches are named by it.

What to establish first

• When did it move, and how far back? Bisect rlang::hash() of the key
  parts list against installed sf / PROJ / rlang versions. git log on the
  two key functions will show they did not change, which localises it to the
  environment.
• Which member drifted? Hash each element of parts separately. The prime
  suspect is sf::st_as_binary() output or the serialization of the sfc
  attributes — code-check.md already records that an sfc carries a
  PROJ-generated CRS WKT that drifts across PROJ versions, which is why the key
  hashes WKB and passes the CRS separately. Worth confirming that reasoning
  still holds.
• Is the key stable going forward, or will it move again on the next PROJ
  bump? That is the question that decides the fix. A key that silently
  re-shuffles on a dependency upgrade is a recurring outage, not a one-off.

Candidate fixes, in rough order of preference

1. Make the key robust to the drifting member — hash a canonical
   representation rather than a serialized R object, so a PROJ or sf upgrade
   cannot move it.
2. Version the cache directory (cache/v2/...), so a deliberate key change
   is a migration rather than a silent orphaning, and old caches are findable.
3. Detect and report, at minimum: on a cache miss where a same-AOI file
   exists under a different key, say so, rather than silently re-streaming.

Related

• dft_stac_fetch cache key omits AOI -> second area silently gets first area's raster #25 — the cache-key-covers-every-input work that established these goldens
• filter_geom evaluated and rejected on measurement; ship gdalcubes parallel instead (was: adopt the fixed filter_geom) #47 — where this surfaced (unrelated branch; the two failures are identical on
  main, so they did not block it)

Reconciliation — what the work established

Recorded at merge time (PR #57). The analysis above is kept as written; these are the points where
measurement gave a different answer.

1. The suspected cause is wrong. It is not sf/PROJ — it is rlang::hash() itself.

The body nominates sf::st_as_binary() output or sfc attribute serialization drifting across
PROJ versions. Measured, that is not what happened. rlang::hash() was rewritten in rlang
1.3.0, and rlang's own NEWS says so:

hash() now uses its own walking strategy… This does mean that with this version all hash
values will now be different. …you should assume it's always possible for a new version to
invalidate existing hashes.

This matters because it changes the fix. Candidate fix 1 ("make the key robust to the drifting
member") presumes there is a drifting member; there is not, and a fix aimed at the geometry
member would have left the real cause in place to recur on the next rlang bump.

2. "Which member drifted?" has no answer — the whole hash function moved. Evidence:

The key function extracted from its own pinning commit reproduces today's value, so drift's code
never changed. rlang 1.3.0 was installed 2026-08-06, between the #38 pin (2026-07-11) and the #51
re-pin (2026-09-01), which is exactly why only the older goldens fail.

3. The passing golden is not a control. It holds only because #51 re-pinned it after the
upgrade. Its contemporaneous #36-era value fails too — so "the fetch key is fine, therefore the
shared members are fine" is the available wrong inference, and this body's FAIL 2 count would
have reinforced it.

4. "Is the key stable going forward?" — not with rlang::hash(), by rlang's own statement.
That answer is what selected the fix, and it also settles the pinning question: no rlang pin,
because after the fix rlang is not in the key path at all.

5. All three candidate fixes shipped, with fix 1 reframed. Not "robust to the drifting member"
but keyed by content: a canonical string hashed by its bytes with
digest::digest(algo = "xxhash64", serialize = FALSE). Fix 2 (versioned cache directory) shipped as
<cache>/v2/<source>/. Fix 3 (detect and report) shipped as dft_cache_info()'s
n_files_superseded / size_mb_superseded plus dft_cache_clear(scheme = "superseded").

6. The failure count in this body is stale. It reports FAIL 2 | PASS 407; main at the time
of the fix was FAIL 1 | PASS 522 — only the cube golden. The fetch golden passes, for the reason
in point 3.

7. The stated cost overstates what was being paid. "Each cube costs 10–30 min to rebuild" is
true of Sentinel-2 cubes but the cache held zero cube entries — it was entirely io-lulc fetch
entries. Measured re-fetch is ~10 s per entry and flat in entry size, so the real one-time cost
is minutes, not hours. The cube figure remains the forward-looking reason this mattered: the next
rlang bump would have destroyed those silently.