Skip to content

Serve /api/city stale-while-revalidate, instrument the intro drop-off, stop the 404 noise - #109

Merged
wachen merged 2 commits into
mainfrom
fix/city-swr-and-noise
Aug 10, 2026
Merged

Serve /api/city stale-while-revalidate, instrument the intro drop-off, stop the 404 noise#109
wachen merged 2 commits into
mainfrom
fix/city-swr-and-noise

Conversation

@wachen

@wachen wachen commented Aug 10, 2026

Copy link
Copy Markdown
Owner

Three fixes that came straight out of the Cloudflare numbers for Jul 12 - Aug 10.

1. /api/city never blocks a visitor on the sheet again

/api/city averaged 2.7s and twice died at 8.3s with a 502 in the Aug 3-10 window. Cause: past the 5-minute freshness window, handleCity discarded the cached tally and made the visitor wait on the Apps Script round-trip. A perfectly usable cached entry was sitting there both times.

It now serves that entry immediately and refreshes in the background. Only a cold cache (first hit per colo per day) blocks.

One subtlety worth flagging for review: naively returning { ...cached, stale: true } on this path would have made /city/ show its "Live tally unavailable right now" banner (src/boot-city.jsx) on every request more than 5 minutes old. That banner means "upstream is down", not "a few minutes behind". So stale now needs CITY_STALE_MS (30 min) of failed refreshes, which is unreachable while background refreshes are working.

2. Two funnel events on the screen where the funnel actually leaks

The 7-day funnel was 74 game_started → 15 mode_chosen → 9 submit_attempted → 9 submit_succeeded. Submissions never fail once attempted, but four out of five people who pick a mode never reach the board, and nothing recorded said whether they bounced off the five-field intro gate on sight or gave up partway through it.

  • intro_engaged fires once on the first keystroke in any field. Splits bounced-on-sight from gave-up-partway.
  • intro_blocked fires when Start was pressed but validation refused, carrying which required fields were missing. This is the actionable one: it names the field that costs the conversion.

Field names only, never values. The Worker re-derives the list from INTRO_FIELDS rather than echoing it, so a forged beacon can only ever produce a subset in a fixed order.

I did not rename game_started/mode_chosen, even though they fire in the opposite order to what the names suggest (game_started = picked a mode, mode_chosen = cleared the intro gate). Renaming mid-event-week would orphan the existing baseline. Worth doing after the burn.

3. The 404 noise

45% of all requests and 82% of Worker invocations are scanner traffic. Some of it was self-inflicted: favicon.ico and apple-touch-icon-precomposed.png were 404ing because clients request those exact paths whether or not the HTML points at them, and sitemap.xml was missing. Adds those three plus llms.txt, with day-long cache headers.

favicon.ico is a real multi-size ICO (16/32/48, PNG-embedded) generated from apple-touch-icon.png. No robots.txt here on purpose: Cloudflare currently serves a synthetic one, and shadowing it could drop AI Crawl Control's directives.

The remaining scanner traffic is a WAF rule, which is dashboard config rather than a code change, so it is not in this PR.

Verification

  • bun test — 132 pass, 0 fail. The 4 new /api/city tests fail by timeout against the pre-fix handler, and that timeout is precisely the blocked visitor the 502s came from.
  • bun run scripts/build.js && git status --porcelain -- dist — clean.
  • test/boot-smoke.mjs — all four pages pass.
  • Both new beacons confirmed end to end in Chromium, for board and form mode, including that a second field edit does not re-emit intro_engaged.

Note for reviewers: this rebased onto #108, which unified the two Intro render branches and moved trackEvent into beacon.js as window.sendEvent. The auto-merge silently put mode="form" on both paths; it is now mode={board ? 'board' : 'form'}, which is what the two-mode browser check above pins.

🤖 Generated with Claude Code

https://claude.ai/code/session_01B94AdnTqEvYxFjRLXQUMBb

…; stop the 404 noise

Cloudflare analytics for Aug 3-10 turned up three things worth fixing.

/api/city averaged 2.7s and twice died at 8.3s with a 502, while a perfectly
usable cached tally sat in the colo cache unused: past the 5-minute freshness
window the handler threw the cached entry away and made the visitor wait on the
Apps Script round-trip. It now serves that entry immediately and refreshes in
the background, so only a cold cache (first hit per colo per day) blocks. The
"Live tally unavailable" banner would have started firing on every request over
5 minutes old, which is not what it means, so it now needs 30 minutes of failed
refreshes (CITY_STALE_MS) and again indicates a real upstream outage.

The funnel showed 74 mode picks producing 15 starts, with nothing recorded to
say whether those 59 bounced off the five-field intro gate on sight or gave up
partway through it. intro_engaged fires once on the first keystroke in any
field; intro_blocked fires when Start was pressed but validation refused, and
carries which required fields were empty or invalid. Field names only, never
values, re-derived server-side from an allowlist so a forged beacon can only
produce a subset of INTRO_FIELDS.

45% of all requests and 82% of Worker invocations were scanner traffic, and
some of that noise was self-inflicted: favicon.ico and
apple-touch-icon-precomposed.png were 404ing because clients request those
exact paths whether or not the HTML points at them, and sitemap.xml was
missing. Adds all three plus llms.txt, with day-long cache headers.

Verified: 132 tests pass, and the 4 new /api/city ones fail by timeout against
the pre-fix handler — that timeout is the blocked visitor the 502s came from.
boot-smoke passes on all four pages. Both new beacons confirmed end to end in
Chromium for board and form mode, including that a second field edit does not
re-emit intro_engaged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B94AdnTqEvYxFjRLXQUMBb
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 10, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
green-radius-game 73e4ec5 Commit Preview URL

Branch Preview URL
Aug 10 2026, 11:20 PM

…tro_engaged

Found while reviewing #109, and the first one is a live outage class on main
rather than anything this branch introduced.

#108 moved the funnel transport into beacon.js as window.sendEvent and called it
directly from nine places. beacon.js is not guaranteed to load: a network blip,
or an ad blocker matching its filename, which is a common filter-list target.
The direct call then throws from a click handler, so with beacon.js blocked the
mode picker's first click threw "window.sendEvent is not a function" and the
game could not be started at all. It reported nothing, because the client-error
beacon lives in that same blocked file — which is why no telemetry ever showed
it. Confirmed in headless Chromium: intro inputs = 0, one uncaught page error.

This branch would have widened it, since intro_engaged put the same unguarded
call in the keystroke path of every intro field, where the throw happens before
set(v) and would have swallowed each character.

Fixed at the root: all ten call sites (green-radius.jsx, src/boot-result.jsx)
now go through trackEvent in src/core.jsx, which loads first on all four pages
and no-ops when the transport is absent. The helper is deliberately NOT named
sendEvent — top-level bindings in these classic scripts land on window, so that
name would overwrite window.sendEvent with the wrapper and recurse forever.
test/boot-smoke.mjs gains a beacon.js-blocked case; it fails with the pre-fix
unguarded form (verified) and passes now.

Also: intro_engaged fired per Intro mount, and Intro unmounts on Back, so a
player who stepped back and returned counted as two engaged players in the very
funnel this PR adds. Now once per page load. And llms.txt claimed the game
"takes about 10 to 20 minutes", a figure supported by nothing in the repo, so it
is gone rather than published as fact on the live domain.

Docs: docs/architecture.md rewrites the funnel section around trackEvent and
documents the /api/city stale-while-revalidate flow plus why stale:true waits
for CITY_STALE_MS; CLAUDE.md gains both gotchas.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B94AdnTqEvYxFjRLXQUMBb
@wachen

wachen commented Aug 10, 2026

Copy link
Copy Markdown
Owner Author

Review pass (high effort) — 5 findings, 3 fixed

The review turned up a live production bug on main that this PR did not introduce but would have widened.

Fixed: blocked beacon.js makes the game unplayable

#108 moved the funnel transport into beacon.js as window.sendEvent and called it directly from nine places. beacon.js is not guaranteed to load — a network blip, or an ad blocker matching its filename, which is a common filter-list target. The direct call then throws from a click handler, so the mode picker's first click dies and the game cannot be started at all. Confirmed in headless Chromium:

sendEvent defined? undefined
reached intro screen? inputs = 0
JS errors: ["pageerror: window.sendEvent is not a function"]

It reports nothing, because the client-error beacon lives in that same blocked file. That is why no telemetry ever surfaced it, and it is live right now.

This branch would have made it worse: intro_engaged put the same unguarded call in the keystroke path of every intro field, where the throw happens before set(v) and would have swallowed each typed character.

Fixed at the root — all ten call sites now route through trackEvent (src/core.jsx, first script on all four pages), which no-ops when the transport is absent. test/boot-smoke.mjs gains a beacon.js-blocked case, verified to fail against the unguarded form:

[FAIL] / (beacon.js blocked)
    - uncaught page error: window.sendEvent is not a function

One trap worth recording: the helper must not be named sendEvent. Top-level bindings in these classic scripts land on window, so that name would overwrite window.sendEvent with the wrapper and recurse forever. Both gotchas are now in CLAUDE.md.

Fixed: intro_engaged double-counted players

engagedRef was per-mount, and Intro unmounts on Back — so a player who stepped back and returned counted as two engaged players in the very funnel this PR adds. Now once per page load; verified one beacon across a Back-and-return.

Fixed: unsourced claim in llms.txt

"takes about 10 to 20 minutes" was supported by nothing in the repo. Removed rather than published as fact on the live domain.

Accepted, not fixed

  • favicon.ico is an opaque square (generated from apple-touch-icon.png, which has no alpha) while favicon.svg is a transparent circle. Only affects clients that ignore the SVG link. A brand-colored square is a normal favicon treatment.
  • sitemap.xml lastmod is hardcoded to 2026-08-10 and will rot. It is an advisory hint; automating it means adding build machinery for no real gain.

Verification

bun test 132 pass / 0 fail · compile gate clean · boot-smoke passes all five cases including the new one. Docs updated: docs/architecture.md (funnel section rewritten around trackEvent; /api/city SWR flow and the CITY_STALE_MS rationale) and CLAUDE.md.

🤖 Generated with Claude Code

@wachen
wachen merged commit 3ed073c into main Aug 10, 2026
3 checks passed
@wachen
wachen deleted the fix/city-swr-and-noise branch August 10, 2026 23:21
wachen added a commit that referenced this pull request Aug 12, 2026
"Manage your robots.txt" is being turned off, so this file is served alone
rather than merged into. Cloudflare's managed block emitted a blanket
"Disallow: /" for nine AI crawlers, which asks them not to fetch at all:
strictly stronger than "do not train on this", and it left the llms.txt
added in #109 unreachable by the agents it was written for.

Declaring Content-Signal: search=yes,ai-train=no,use=reference states the
intended position instead. Crawl, index, quote, link back: yes. Train: no.
Enforcement (Block AI training bots) is also off now, so the preference and
the enforcement finally agree.

Verified: robots.txt has exactly one User-agent group, one Content-Signal,
one Sitemap, and zero Disallow rules, so no colliding "*" group; all 4
sitemap locs resolve on disk; no dist drift; 132 tests pass; boot-smoke
passes all 5 cases.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B94AdnTqEvYxFjRLXQUMBb
wachen added a commit that referenced this pull request Aug 12, 2026
…Fs (#110) (#110)

/robots.txt was the Worker's largest single source of invocations: 26 of the
42 404s still reaching it in the 24h after the scanner WAF rule landed. With
no file at that path, Cloudflare probed the origin to decide whether to merge
with or create one, and the Worker answered 404 about once an hour. Crawlers
never saw those 404s, since Cloudflare synthesized a file for them, but what
it synthesized carried no Sitemap line, so the sitemap.xml added in #109 had
no discovery path at all.

The new file is served as a static asset, so the path stops invoking the
Worker, and it declares Content-Signal: search=yes,ai-train=no,use=reference.
That replaces the blanket "Disallow: /" for nine AI crawlers that Cloudflare's
managed block had been emitting, which asked them not to fetch anything and
left llms.txt unreachable by the agents it was written for. Crawl, index,
quote, and link back are allowed; training is not. The "Manage your robots.txt"
and "Block AI training bots" zone settings were both turned off to match.

/result/ now sends X-Robots-Tag: noindex, keeping individual camps' names and
scores out of search results while still letting link unfurlers fetch the page
for the OG share card. A robots.txt Disallow would have blocked that fetch and
broken previews.

sitemap.xml gains the two print-and-play PDFs and refreshed lastmod dates.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B94AdnTqEvYxFjRLXQUMBb
wachen added a commit that referenced this pull request Aug 12, 2026
Docs only, no behaviour change. Three classes of drift, each verified against
the code rather than taken on trust.

Wrong route count: CLAUDE.md said "seven dynamic routes" and listed seven, but
worker/index.js has eight — POST /api/admin/visit was missing (drift since
#106), and README.md's layout table listed five. Both now agree with
docs/architecture.md, which already said eight.

The undocumented run_worker_first trap that made #110 ship a no-op: opting
/result/ into the Worker also opts it OUT of the asset layer, so _headers never
applies there, silently and with no error. The same mechanism explains why
Rocket Loader being enabled on the zone never reordered this app's load-bearing
script order — Cloudflare's HTML rewriters are skipped for Worker-served
responses too. Rocket Loader was turned off and 0-RTT turned on (2026-08-12).

No crawler surface documented: architecture.md never mentioned robots.txt,
sitemap.xml or llms.txt, despite documenting analogous owner-side Cloudflare
toggles elsewhere. Adds a section covering all three plus the two zone settings
that interact with robots.txt and why both are off. CONTRIBUTING.md and
README.md layout tables gained the five crawler/icon files from #109 and #110
that neither had listed.

Also folds in two docs-only edits that had been sitting uncommitted: CLAUDE.md's
three per-page bullets condensed into one "Three read-only view pages" bullet
plus a curl-in-the-cloud-pulse gotcha, and a docs/admin-setup.md pointer naming
docs/apps-script/Code.gs as the paste-in source of truth. Shipping them here
avoids a conflict, since the CLAUDE.md edit sat on the line adjacent to the
route-count fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B94AdnTqEvYxFjRLXQUMBb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant