Skip to content

fix: authenticate the CARTO basemap with the suite key - #10

Merged
tgilbert14 merged 2 commits into
mainfrom
claude/carto-basemap-key
Aug 31, 2026
Merged

fix: authenticate the CARTO basemap with the suite key#10
tgilbert14 merged 2 commits into
mainfrom
claude/carto-basemap-key

Conversation

@tgilbert14

@tgilbert14 tgilbert14 commented Aug 29, 2026

Copy link
Copy Markdown
Owner

Status: complete and validated ✅

This PR opened source-only, because editing global.R and server.R invalidates two generated authority files that AGENTS.md forbids hand-editing:

  • manifest.json
  • release/production-identity.json — its runtime_payload_sha256 hashes global.R, ui.R and server.R

Both are now regenerated through the sanctioned path, and CI is green.

What

CARTO began watermarking unauthenticated basemaps.cartocdn.com raster tiles with "API KEY REQUIRED" on 2026-08-26. The national picker on the splash renders defaced on first load — it is the only map in this app, and the first thing every visitor sees. The request still returns HTTP 200 with a valid PNG, so nothing errored or logged.

Full suite diagnosis: docs/SUITE-BASEMAP-INCIDENT-2026-08.md · tgilbert14/NEON-Driver-Cascade#70. Proven first in Ground Beetle #22 — merged, deployed, live map confirmed correct.

How

Adds add_suite_basemap() to global.R and routes both tile call sites through it (server.R:940, :963):

  • With CARTO_BASEMAP_KEY set (Connect Cloud content variable): the exact same Positron tiles as before, keyed, via addTiles()addProviderTiles() cannot carry the key because the bundled CartoDB template has no {apikey} placeholder. Explicit OpenStreetMap + CARTO attribution, as CARTO's terms require.
  • Without the key: falls back to Esri's keyless grey canvas, so the map degrades to a clean basemap rather than a defaced one. Capped at maxNativeZoom = 16, that canvas's real content limit.

Worth knowing for future sweeps

This app used the object form leaflet::providers$CartoDB.Positron, not a quoted provider name — so a search for "CartoDB misses it entirely and would have left this app fully watermarked. The suite-wide sweep is:

grep -rn --include='*.R' -E 'CartoDB[."$]|cartocdn' .

How the authority was regenerated

refresh-data.yml was dispatched on this branch with skip_download=true. All four jobs succeeded against head 96e6ae7: the offline source/science/producer contracts, an independent rebuild of the derived data and preliminary manifest, the exact Pages and Connect release identity, the Living Poster contract, and the full candidate verification. Its publish_candidate job wrote the validated tree to automation/invert-data-refresh (cc322fd).

The three authority files were taken verbatim from that branch. Before committing, every other file was byte-compared: automation/invert-data-refresh differed from this branch in exactly those three paths, so all 34 data/sites/*.rds, data/release_contract.rds, both receipts, data/site_index.rds, data/cross_site.rds, data/search_index.rds and data-sample/demo.rds are byte-identical. No scientific bytes moved.

runtime_payload_sha256    89d9fdab… -> 69d67a7e…   (hashes global.R, ui.R, server.R)
manifest_contract_sha256  f6045525… -> f6ce6cc8…
release_id                eb247cf7… -> 8e998862…

Verified

  • CI green on this head: gate ✅ · produce_candidate ✅ · validate_candidate ✅ — including the "Reject a stale committed identity on pull requests" step that failed before the regeneration.
  • The key returns clean tiles where unkeyed requests are watermarked — proven at CDN origin misses (the CDN caches keyed responses separately, so a cached tile is not a valid test).
  • grep -E 'addProviderTiles\(|providers\$' outside the helper: zero remaining matches.

Before merging (owner)

Connect Cloud → this app's content settings → Variables → add CARTO_BASEMAP_KEY (the cb1_… key). Until it is set, the deployed app shows the clean Esri fallback rather than CARTO — correct, but visibly greyer than Ground Beetle.

CARTO began watermarking unauthenticated basemaps.cartocdn.com raster
tiles with "API KEY REQUIRED" on 2026-08-26, so the national picker on
the splash renders defaced on first load — the only map in this app, and
the first thing every visitor sees. The tile request still returns HTTP
200 with a valid PNG, so nothing errors or logs; full suite diagnosis in
NEON-Driver-Cascade docs/SUITE-BASEMAP-INCIDENT-2026-08.md.

Adds an add_suite_basemap() helper to global.R and routes both tile call
sites through it. With CARTO_BASEMAP_KEY set (a Connect Cloud content
variable) it serves the exact same Positron tiles as before, keyed, via
addTiles() — addProviderTiles() cannot carry the key because the bundled
CartoDB template has no {apikey} placeholder. The key is a public
rate-limited identifier, not a credential: it rides in the client-side
tile URL by design, and the env var exists to keep it out of git and make
rotation a Connect setting rather than a release.

Note this app used the object form leaflet::providers$CartoDB.Positron
rather than a quoted provider name, so a search for the quoted string
misses it entirely; the suite-wide sweep is
grep -rn --include='*.R' -E 'CartoDB[."$]|cartocdn'.

Without the key it falls back to Esri's keyless grey canvas so the map
degrades to a clean basemap rather than a defaced one, capped at that
canvas's real maxNativeZoom of 16.

SOURCE ONLY — NOT YET MERGEABLE. Editing global.R and server.R
invalidates two generated authority files: manifest.json and
release/production-identity.json, whose runtime_payload_sha256 hashes
global.R, ui.R and server.R. AGENTS.md forbids hand-editing or casually
regenerating them, this repo has no CI to produce them, and no R runtime
was available in the patch environment. They must be regenerated in the
clean validator before this merges.

Copy link
Copy Markdown
Owner Author

CI ran after all, and it failed exactly where the PR description predicted

Failing check: validate_candidate"Reject a stale committed identity on pull requests"
(run 33229723311)

identity_paths=(manifest.json release/production-identity.json docs/release.json)
git diff --exit-code -- "${identity_paths[@]}"

I'd said this repo has no CI because it has no ci.yml — the validator actually lives in refresh-data.yml and runs on PRs. Correcting that; it doesn't change the conclusion, and it's now confirmed by the run rather than predicted.

Two things the run positively establishes

  • gate and produce_candidate both passed. I downloaded the invert-produced-96e6ae7b artifact and byte-compared all 36 files against this branch: 36 identical, 0 differing. The producer changed no scientific bytes — this really is only the identity files.
  • There is nothing for me to shuttle. The validator regenerates the three identity files at "Build and verify the exact Pages and Connect release identity", but the stale-identity gate is the very next step, so the job dies before "Upload validated candidate" ever runs. The only artifact is the producer's, which doesn't contain them. I confirmed this against the workflow rather than assuming it.

To unblock, on this branch

Rscript --vanilla scripts/build_cross_site.R
Rscript --vanilla scripts/build_search_index.R
INV_MANIFEST_PHASE=prestamp Rscript --vanilla scripts/write_manifest.R
INV_RELEASE_IDENTITY_MODE=write INV_WRITE_PAGES_RELEASE=1 Rscript --vanilla scripts/write_release_identity.R
INV_MANIFEST_PHASE=final Rscript --vanilla scripts/write_manifest.R
INV_RELEASE_IDENTITY_MODE=verify INV_WRITE_PAGES_RELEASE=1 Rscript --vanilla scripts/write_release_identity.R
cmp -s release/production-identity.json docs/release.json
git commit -am "build: regenerate the manifest and release identity for the basemap change"

That is the validator's own sequence, lifted verbatim from the two steps above. The prestamp → identity → final-manifest → verify ordering matters: the identity binds the manifest contract, and the final manifest then records the identity file.

What I deliberately did not do

refresh-data.yml also has a workflow_dispatch path, and its stale-identity gate is pull_request-only — so dispatching it would sail past this failure. I didn't, because its publisher force-pushes the shared automation/invert-data-refresh branch used for real data refreshes and requires the promotion commit to be a direct child of the dispatched SHA. That would put a basemap fix into your data-refresh lane and wouldn't update this PR anyway. It's available to you if you'd rather go that route, but it seemed the wrong tool for a two-line map change.

The same helper is merged, deployed and confirmed live in Ground Beetle #22, and is green in Plant Diversity, Small Mammal, Vegetation Structure and Water Chemistry. I'll pick this up as soon as there's a new commit.


Generated by Claude Code

The generated authority for this PR's source, produced by the pinned
validator rather than by hand, per AGENTS.md.

Source: refresh-data.yml run 33409207678 (workflow_dispatch, skip_download),
whose four jobs all succeeded against this branch's head 96e6ae7 -- the
offline source/science/producer contracts, an independent rebuild of the
derived data and preliminary manifest, the exact Pages and Connect release
identity, the Living Poster contract, and the full candidate verification.
Its publish job wrote the validated tree to automation/invert-data-refresh
(cc322fd).

These three files are taken verbatim from that branch. Every other file was
byte-compared first: automation/invert-data-refresh differs from this branch
in exactly manifest.json, release/production-identity.json and
docs/release.json, and this tree is now byte-identical to it.

runtime_payload_sha256 moves because it hashes global.R, ui.R and server.R,
two of which this PR edits; manifest_contract_sha256 and release_id follow.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01738HeGPEU9NEoczL8RXnu1
@tgilbert14 tgilbert14 changed the title fix: authenticate the CARTO basemap with the suite key (needs validator regen before merge) fix: authenticate the CARTO basemap with the suite key Aug 31, 2026
@tgilbert14
tgilbert14 marked this pull request as ready for review August 31, 2026 16:22
@tgilbert14
tgilbert14 merged commit c7dbf9a into main Aug 31, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants