diff --git a/.gitignore b/.gitignore index 16de9119..3dcd20e3 100644 --- a/.gitignore +++ b/.gitignore @@ -28,6 +28,8 @@ dist-sourcemaps/ /playwright-report/ /blob-report/ /playwright/.cache/ +/badseo/test-results/ +/badseo/playwright-report/ .tanstack .logs/ **/.pnpm-store/ diff --git a/badseo/README.md b/badseo/README.md index 67339088..c175f797 100644 --- a/badseo/README.md +++ b/badseo/README.md @@ -57,12 +57,109 @@ things we're trying to break (it insists on injecting a ``, etc.). npx wrangler dev # serves on http://localhost:8787 ``` +## Google Analytics 4 and consent + +The production environment is configured with the public GA4 measurement ID +`G-7MXV9FH7SS`, but the committed `GA4_ADMIN_VERIFIED` safety gate remains +`false` until the GA Admin checklist below is complete. Once enabled, analytics +uses global, opt-in **Basic Consent Mode**: no Google Analytics tag, Analytics +request, or cookieless ping is loaded before an affirmative choice. + +badseo.dev does not need a Google-certified CMP because it uses GA4 only and +does not use AdSense, Ad Manager, or AdMob. Google permits a custom consent +solution for Analytics. The first-party implementation here avoids loading a +second consent vendor while still keeping an operator-accessible audit trail: + +1. The browser queues all four Consent Mode v2 defaults as denied. +2. **Accept analytics** posts a pseudonymous receipt to the same-origin + `/analytics-consent` endpoint. +3. The Worker rate-limits and validates the request, then writes an append-only + record to `CONSENT_LOG`. The record contains the server time, exact notice + and choices, notice/privacy versions, measurement ID, and grant state. It + does not include the visitor's IP address or user agent. +4. Only a successful KV write lets a newly accepted choice grant + `analytics_storage` and load GA. Later page views rely on the unexpired, + matching-version, matching-measurement-ID necessary browser receipt. A + timeout, quota error, or storage outage during a new grant leaves analytics + off and offers a retry. Same-origin tabs synchronize grants and withdrawals. +5. Withdrawal denies consent and removes `_ga` cookies immediately. Its ledger + event is retried on the next page view if the first write fails and the + browser can retain the necessary retry marker. If browser storage itself is + unavailable, the page still attempts the write immediately and remains + analytics-off regardless of its outcome. + +The necessary browser receipt is treated as expired after 400 days and removed +on the next visit. Operator-side consent events expire after 830 days, covering +that possible grant period plus the configured 14-month GA4 user-level and +event-level retention window. Do not export those records; update the policy +and implement an equivalent per-event deletion schedule before introducing any +separate evidence store. Have privacy counsel revisit the evidence period if +the analytics retention or legal requirements change. Bump the canonical +notice version in `src/consent.ts` whenever the vendor, purpose, choices, or +material wording changes; a measurement-ID change automatically invalidates +stored grants. Never rewrite an old notice version. + +Wrangler automatically provisions separate local and production KV namespaces +from `wrangler.jsonc`. The root Worker is deliberately named `badseo-dev`; the +production environment retains its existing implicit identity, +`badseo-production`, so a plain preview deploy cannot overwrite the production +Worker or its bindings. Production deploys must use `--env production` because +bindings and variables do not inherit into named environments. + +On the first production deploy, Wrangler writes the provisioned KV namespace ID +back into `wrangler.jsonc`. Review and commit that generated ID immediately so +ledger continuity is explicit in later diffs. Monitor consent-endpoint 429/503 +responses and the KV write quota; add an account-level WAF/rate-limit rule for +`/analytics-consent` if automated abuse appears. To inspect the operator-side +ledger after deployment: + +```bash +npx wrangler kv key list \ + --binding CONSENT_LOG \ + --env production \ + --remote \ + --prefix analytics-consent/ +``` + +Before production deployment, finish these GA Admin settings: + +1. Keep all four **Account Data Sharing Settings** off. +2. In Enhanced Measurement, keep page views, scrolls, and outbound clicks on; + turn the other events off. +3. Set user-level and event-level data retention to 14 months. +4. Leave Google Signals and advertising personalization off. Do not set a + User-ID, link Google Ads, or import user data. +5. Review the badseo.dev policy at `/privacy`; it deploys with this Worker, so + the consent notice and policy cannot drift across separate applications. +6. Verify the default and update signals with Tag Assistant, then confirm one + consented page view in GA Realtime. +7. Only after steps 1–6 pass, set `GA4_ADMIN_VERIFIED` to `"true"` in the + production environment and run the production dry-run again. + +For local browser testing, inject a non-production ID without editing the +Wrangler config: + +```bash +npx wrangler dev \ + --var GA4_MEASUREMENT_ID:G-TEST123 \ + --var GA4_ADMIN_VERIFIED:true +``` + +The browser consent UI is injected by `/analytics.js`, not included in the raw +page HTML. This keeps its text and controls out of the crawler fixtures' word +counts, heading checks, image checks, and duplicate-content hashes. + +The Worker fails closed if the GA Admin verification gate is not `"true"`, or +if the measurement ID, KV binding, or rate-limit binding is absent: +`/analytics.js` only removes stale choices and `_ga` cookies, and no consent UI +or Google request is produced. + ## Run the end-to-end audit The harness drives the **real** OpenSEO crawl + issue-detection functions (imported straight from `../src`) against a running badseo.dev, then asserts every -fixture triggers exactly the issues it declares — and that the homepage, catalog, -and support pages come back clean. +fixture triggers exactly the issues it declares — and that the homepage, +catalog, privacy policy, and support pages come back clean. ```bash # with `wrangler dev` running in another terminal: @@ -127,6 +224,7 @@ directly. First-time setup: the `production` env in `wrangler.jsonc` binds the custom domains `badseo.dev` and `www.badseo.dev`, so the zone must be on the Cloudflare -account before the first deploy. (The routes live under `production` so that -plain `wrangler dev` still serves on localhost.) To preview on a `*.workers.dev` -URL without the custom domain, deploy the top-level env: `npx wrangler deploy`. +account before the first deploy. Production disables `workers.dev` and preview +URLs so the real GA ID is served only on those custom domains. To preview on the +separate `badseo-dev` `*.workers.dev` Worker without the custom domain or real +measurement ID, deploy the top-level env with `npx wrangler deploy`. diff --git a/badseo/e2e/consent.spec.ts b/badseo/e2e/consent.spec.ts new file mode 100644 index 00000000..ec759452 --- /dev/null +++ b/badseo/e2e/consent.spec.ts @@ -0,0 +1,491 @@ +/* eslint-disable max-lines -- End-to-end lifecycle scenarios stay together so shared consent helpers and ordering remain visible. */ +import { + expect, + test, + type BrowserContext, + type Request as PlaywrightRequest, + type Route, +} from "@playwright/test"; + +const STORAGE_KEY = "badseo:analytics-consent:v2"; + +interface ConsentRequestBody { + action?: string; + receiptId?: string; +} + +interface StoredConsentRecord { + choice?: string; + measurementId?: string; + receiptId?: string; +} + +function parseJson(raw: string | null): unknown { + if (!raw) return null; + try { + return JSON.parse(raw) as unknown; + } catch { + return null; + } +} + +function readStringProperty( + value: unknown, + key: "action" | "choice" | "measurementId" | "receiptId", +): string | undefined { + if (value === null || typeof value !== "object") return undefined; + switch (key) { + case "action": + return "action" in value && typeof value.action === "string" + ? value.action + : undefined; + case "choice": + return "choice" in value && typeof value.choice === "string" + ? value.choice + : undefined; + case "measurementId": + return "measurementId" in value && typeof value.measurementId === "string" + ? value.measurementId + : undefined; + case "receiptId": + return "receiptId" in value && typeof value.receiptId === "string" + ? value.receiptId + : undefined; + } +} + +function consentRequestBody(request: PlaywrightRequest): ConsentRequestBody { + const parsed = parseJson(request.postData()); + return { + action: readStringProperty(parsed, "action"), + receiptId: readStringProperty(parsed, "receiptId"), + }; +} + +function parseStoredConsent(raw: string | null): StoredConsentRecord | null { + const parsed = parseJson(raw); + if (parsed === null || typeof parsed !== "object") return null; + return { + choice: readStringProperty(parsed, "choice"), + measurementId: readStringProperty(parsed, "measurementId"), + receiptId: readStringProperty(parsed, "receiptId"), + }; +} + +function requireStoredConsent(raw: string | null): StoredConsentRecord { + const record = parseStoredConsent(raw); + if (!record) throw new Error("Expected a valid stored consent record"); + return record; +} + +async function mockGoogleTag(context: BrowserContext): Promise<void> { + await context.route("https://www.googletagmanager.com/**", (route: Route) => + route.fulfill({ + status: 200, + contentType: "application/javascript", + body: [ + 'document.cookie="_ga=test-client; Path=/; SameSite=Lax";', + 'document.cookie="_ga_TEST123=test-session; Path=/; SameSite=Lax";', + ].join(""), + }), + ); +} + +test("serves a local privacy policy and canonicalizes the consent origin", async ({ + request, +}) => { + const home = await request.get("/"); + const homeHtml = await home.text(); + expect(homeHtml).toContain('<script src="/analytics.js" defer></script>'); + expect(homeHtml).not.toContain("fonts.googleapis.com"); + + const privacy = await request.get("/privacy"); + const privacyHtml = await privacy.text(); + expect(privacy.status()).toBe(200); + expect(privacyHtml).toContain("<title>Privacy policy | badseo.dev"); + expect(privacyHtml).toContain("

Privacy policy

"); + expect(privacyHtml).toContain('href="/privacy"'); + + const sitemap = await request.get("/sitemap.xml"); + expect(await sitemap.text()).toContain( + "http://127.0.0.1:18787/privacy", + ); + + const redirect = await request.get("/privacy?source=redirect-test", { + headers: { host: "www.badseo.dev" }, + maxRedirects: 0, + }); + expect(redirect.status()).toBe(308); + expect(redirect.headers().location).toBe( + "https://badseo.dev/privacy?source=redirect-test", + ); + + const insecureApex = await request.get("/privacy?source=http-test", { + headers: { host: "badseo.dev" }, + maxRedirects: 0, + }); + expect(insecureApex.status()).toBe(308); + expect(insecureApex.headers().location).toBe( + "https://badseo.dev/privacy?source=http-test", + ); +}); + +test("synchronizes grants and withdrawals across open tabs", async ({ + context, +}) => { + await mockGoogleTag(context); + const googleRequests: string[] = []; + context.on("request", (request) => { + if ( + request.url().includes("googletagmanager.com") || + request.url().includes("google-analytics.com") + ) { + googleRequests.push(request.url()); + } + }); + + const pageA = await context.newPage(); + const pageB = await context.newPage(); + await pageA.goto("/?campaign=private#fragment", { + waitUntil: "networkidle", + }); + await pageB.goto("/", { waitUntil: "networkidle" }); + + await expect( + pageA.getByRole("button", { name: "Accept analytics" }), + ).toBeVisible(); + await expect( + pageB.getByRole("button", { name: "Accept analytics" }), + ).toBeVisible(); + expect(googleRequests).toEqual([]); + + const [grantResponse] = await Promise.all([ + pageA.waitForResponse( + (response) => + response.url().endsWith("/analytics-consent") && + consentRequestBody(response.request()).action === "grant", + ), + pageA.getByRole("button", { name: "Accept analytics" }).click(), + ]); + expect(grantResponse.status()).toBe(201); + await expect(pageA.locator("#badseo-google-analytics")).toHaveCount(1); + await expect(pageB.locator("#badseo-google-analytics")).toHaveCount(1); + + const storedGrant = requireStoredConsent( + await pageA.evaluate((key) => localStorage.getItem(key), STORAGE_KEY), + ); + expect(storedGrant).toMatchObject({ + choice: "granted", + measurementId: "G-TEST123", + }); + expect(storedGrant.receiptId).toMatch(/^[0-9a-f-]{36}$/i); + + await pageB.getByRole("button", { name: "Cookie settings" }).click(); + const pageAReloaded = pageA.waitForEvent("load"); + const [withdrawResponse] = await Promise.all([ + pageB.waitForResponse( + (response) => + response.url().endsWith("/analytics-consent") && + consentRequestBody(response.request()).action === "withdraw", + ), + pageB.getByRole("button", { name: "Reject" }).click(), + ]); + expect(withdrawResponse.status()).toBe(201); + await pageAReloaded; + await pageA.waitForLoadState("networkidle"); + + await expect(pageA.locator("#badseo-google-analytics")).toHaveCount(0); + await expect(pageB.locator("#badseo-google-analytics")).toHaveCount(0); + expect( + (await context.cookies()).filter((cookie) => cookie.name.startsWith("_ga")), + ).toEqual([]); +}); + +test("a newer rejection supersedes an in-flight grant", async ({ context }) => { + const googleRequests: string[] = []; + context.on("request", (request) => { + if ( + request.url().includes("googletagmanager.com") || + request.url().includes("google-analytics.com") + ) { + googleRequests.push(request.url()); + } + }); + + const pageA = await context.newPage(); + const pageB = await context.newPage(); + await pageA.goto("/", { waitUntil: "networkidle" }); + await pageB.goto("/", { waitUntil: "networkidle" }); + + await pageA.evaluate(() => { + const nativeFetch = window.fetch.bind(window); + window.fetch = async (input, init) => { + const response = await nativeFetch(input, init); + const body = typeof init?.body === "string" ? init.body : ""; + if (body.includes('"action":"grant"')) { + await new Promise((resolve) => window.setTimeout(resolve, 500)); + } + return response; + }; + }); + + const compensatingWithdrawal = pageA.waitForResponse( + (response) => + response.url().endsWith("/analytics-consent") && + consentRequestBody(response.request()).action === "withdraw", + ); + await pageA.getByRole("button", { name: "Accept analytics" }).click(); + await pageA.waitForTimeout(100); + await pageB.getByRole("button", { name: "Reject" }).click(); + + expect((await compensatingWithdrawal).status()).toBe(201); + await pageA.waitForTimeout(150); + await expect(pageA.locator("#badseo-google-analytics")).toHaveCount(0); + await expect(pageB.locator("#badseo-google-analytics")).toHaveCount(0); + expect(googleRequests).toEqual([]); + + const storedChoice = parseStoredConsent( + await pageA.evaluate((key) => localStorage.getItem(key), STORAGE_KEY), + )?.choice; + expect(storedChoice).toBe("denied"); +}); + +test("a rejection during withdrawal preflight prevents a stale grant", async ({ + context, +}) => { + const grantRequests: string[] = []; + context.on("request", (request) => { + if ( + request.url().endsWith("/analytics-consent") && + consentRequestBody(request).action === "grant" + ) { + grantRequests.push(request.postData() ?? ""); + } + }); + + const pageA = await context.newPage(); + const pageB = await context.newPage(); + await pageA.goto("/", { waitUntil: "networkidle" }); + await pageB.goto("/", { waitUntil: "networkidle" }); + + const pendingReceiptId = "123e4567-e89b-42d3-a456-426614174001"; + await pageB.evaluate((receiptId) => { + localStorage.setItem( + `badseo:analytics-withdrawal:v1:${receiptId}`, + JSON.stringify({ + requestId: "123e4567-e89b-42d3-a456-426614174002", + measurementId: "G-TEST123", + createdAt: new Date().toISOString(), + }), + ); + + const nativeFetch = window.fetch.bind(window); + window.fetch = async (input, init) => { + const response = await nativeFetch(input, init); + const body = typeof init?.body === "string" ? init.body : ""; + if (body.includes('"action":"withdraw"')) { + await new Promise((resolve) => window.setTimeout(resolve, 500)); + } + return response; + }; + }, pendingReceiptId); + + const withdrawal = pageB.waitForResponse( + (response) => + response.url().endsWith("/analytics-consent") && + consentRequestBody(response.request()).action === "withdraw", + ); + await pageB.getByRole("button", { name: "Accept analytics" }).click(); + await withdrawal; + await pageB.waitForTimeout(100); + await pageA.getByRole("button", { name: "Reject" }).click(); + await pageB.waitForTimeout(550); + + expect(grantRequests).toEqual([]); + await expect(pageA.locator("#badseo-google-analytics")).toHaveCount(0); + await expect(pageB.locator("#badseo-google-analytics")).toHaveCount(0); + const state = await pageB.evaluate( + ({ key, receiptId }) => ({ + consent: localStorage.getItem(key), + pending: localStorage.getItem( + `badseo:analytics-withdrawal:v1:${receiptId}`, + ), + }), + { key: STORAGE_KEY, receiptId: pendingReceiptId }, + ); + expect(parseStoredConsent(state.consent)?.choice).toBe("denied"); + expect(state.pending).toBeNull(); +}); + +test("retries a failed withdrawal from its per-receipt marker", async ({ + context, +}) => { + await mockGoogleTag(context); + const page = await context.newPage(); + await page.goto("/", { waitUntil: "networkidle" }); + + await Promise.all([ + page.waitForResponse( + (response) => + response.url().endsWith("/analytics-consent") && + consentRequestBody(response.request()).action === "grant", + ), + page.getByRole("button", { name: "Accept analytics" }).click(), + ]); + await expect(page.locator("#badseo-google-analytics")).toHaveCount(1); + + let failFirstWithdrawal = true; + await page.route("**/analytics-consent", async (route) => { + const action = consentRequestBody(route.request()).action; + if (action === "withdraw" && failFirstWithdrawal) { + failFirstWithdrawal = false; + await route.fulfill({ + status: 503, + contentType: "application/json", + body: '{"error":"simulated outage"}', + }); + return; + } + await route.continue(); + }); + + await page.getByRole("button", { name: "Cookie settings" }).click(); + const failedWrite = page.waitForResponse( + (response) => + response.url().endsWith("/analytics-consent") && + response.status() === 503, + ); + const retriedWrite = page.waitForResponse( + (response) => + response.url().endsWith("/analytics-consent") && + response.status() === 201 && + consentRequestBody(response.request()).action === "withdraw", + ); + await page.getByRole("button", { name: "Reject" }).click(); + + await failedWrite; + await retriedWrite; + await page.waitForLoadState("networkidle"); + const pendingKeys = await page.evaluate(() => + Object.keys(localStorage).filter((key) => + key.startsWith("badseo:analytics-withdrawal:v1:"), + ), + ); + expect(pendingKeys).toEqual([]); + await expect(page.locator("#badseo-google-analytics")).toHaveCount(0); +}); + +test("concurrent re-grants sharing a receipt do not append a withdrawal", async ({ + context, +}) => { + await mockGoogleTag(context); + const pageA = await context.newPage(); + await pageA.goto("/", { waitUntil: "networkidle" }); + + await Promise.all([ + pageA.waitForResponse( + (response) => + response.url().endsWith("/analytics-consent") && + consentRequestBody(response.request()).action === "grant", + ), + pageA.getByRole("button", { name: "Accept analytics" }).click(), + ]); + await pageA.getByRole("button", { name: "Cookie settings" }).click(); + await Promise.all([ + pageA.waitForResponse( + (response) => + response.url().endsWith("/analytics-consent") && + consentRequestBody(response.request()).action === "withdraw", + ), + pageA.getByRole("button", { name: "Reject" }).click(), + ]); + await pageA.waitForLoadState("networkidle"); + + const deniedRecord = requireStoredConsent( + await pageA.evaluate((key) => localStorage.getItem(key), STORAGE_KEY), + ); + expect(deniedRecord.choice).toBe("denied"); + expect(deniedRecord.receiptId).toMatch(/^[0-9a-f-]{36}$/i); + + const pageB = await context.newPage(); + await pageB.goto("/", { waitUntil: "networkidle" }); + await pageB.evaluate(() => { + const nativeFetch = window.fetch.bind(window); + window.fetch = async (input, init) => { + const response = await nativeFetch(input, init); + const body = typeof init?.body === "string" ? init.body : ""; + if (body.includes('"action":"grant"')) { + await new Promise((resolve) => window.setTimeout(resolve, 500)); + } + return response; + }; + }); + + const laterWithdrawals: string[] = []; + context.on("request", (request) => { + const body = consentRequestBody(request); + if ( + request.url().endsWith("/analytics-consent") && + body.action === "withdraw" && + body.receiptId + ) { + laterWithdrawals.push(body.receiptId); + } + }); + + await pageA.getByRole("button", { name: "Cookie settings" }).click(); + await pageB.getByRole("button", { name: "Cookie settings" }).click(); + await pageB.getByRole("button", { name: "Accept analytics" }).click(); + await pageB.waitForTimeout(100); + await pageA.getByRole("button", { name: "Accept analytics" }).click(); + await pageB.waitForTimeout(700); + + await expect(pageA.locator("#badseo-google-analytics")).toHaveCount(1); + await expect(pageB.locator("#badseo-google-analytics")).toHaveCount(1); + expect(laterWithdrawals).toEqual([]); + const activeRecord = requireStoredConsent( + await pageA.evaluate((key) => localStorage.getItem(key), STORAGE_KEY), + ); + expect(activeRecord).toMatchObject({ + choice: "granted", + receiptId: deniedRecord.receiptId, + }); +}); + +test("invalidates grants for another property and remains operable when short", async ({ + context, +}) => { + await context.addInitScript((key) => { + localStorage.setItem( + key, + JSON.stringify({ + choice: "granted", + version: 2, + noticeVersion: "2026-07-10.2", + privacyPolicyVersion: "2026-07-10", + measurementId: "G-OLDPROPERTY", + receiptId: "123e4567-e89b-42d3-a456-426614174000", + updatedAt: new Date().toISOString(), + }), + ); + }, STORAGE_KEY); + + const page = await context.newPage(); + await page.setViewportSize({ width: 390, height: 320 }); + await page.goto("/", { waitUntil: "networkidle" }); + + await expect(page.locator("#badseo-google-analytics")).toHaveCount(0); + await expect(page.getByRole("button", { name: "Reject" })).toBeVisible(); + expect( + await page.evaluate((key) => localStorage.getItem(key), STORAGE_KEY), + ).toBeNull(); + + const panel = page.locator(".analytics-consent"); + const box = await panel.boundingBox(); + expect(box).not.toBeNull(); + expect(box!.y).toBeGreaterThanOrEqual(0); + expect(box!.y + box!.height).toBeLessThanOrEqual(320); + expect( + await panel.evaluate((element) => getComputedStyle(element).overflowY), + ).toBe("auto"); +}); diff --git a/badseo/package.json b/badseo/package.json index 75925372..e1e77c0c 100644 --- a/badseo/package.json +++ b/badseo/package.json @@ -10,7 +10,8 @@ "build": "tsc --noEmit", "deploy": "npm run build && wrangler deploy --env production", "audit": "tsx scripts/run-audit.ts", - "test:e2e": "tsx scripts/run-audit.ts" + "test:e2e": "tsx scripts/run-audit.ts", + "test:consent": "../node_modules/.bin/playwright test --config=playwright.config.ts" }, "devDependencies": { "@cloudflare/workers-types": "^4.20250109.0", diff --git a/badseo/playwright.config.ts b/badseo/playwright.config.ts new file mode 100644 index 00000000..4ecdffff --- /dev/null +++ b/badseo/playwright.config.ts @@ -0,0 +1,32 @@ +import { defineConfig } from "@playwright/test"; +import { fileURLToPath } from "node:url"; + +const badseoDir = fileURLToPath(new URL(".", import.meta.url)); + +export default defineConfig({ + testDir: "./e2e", + timeout: 30_000, + expect: { timeout: 8_000 }, + fullyParallel: false, + workers: 1, + reporter: [["list"]], + use: { + baseURL: "http://127.0.0.1:18787", + browserName: "chromium", + headless: true, + launchOptions: process.env.PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH + ? { executablePath: process.env.PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH } + : undefined, + screenshot: "only-on-failure", + trace: "retain-on-failure", + }, + webServer: { + command: + "../node_modules/.bin/wrangler dev --config wrangler.jsonc --var GA4_MEASUREMENT_ID:G-TEST123 --var GA4_ADMIN_VERIFIED:true --port 18787", + cwd: badseoDir, + url: "http://127.0.0.1:18787", + reuseExistingServer: false, + timeout: 45_000, + gracefulShutdown: { signal: "SIGTERM", timeout: 5_000 }, + }, +}); diff --git a/badseo/scripts/run-audit.ts b/badseo/scripts/run-audit.ts index ac14f03e..40f74bad 100644 --- a/badseo/scripts/run-audit.ts +++ b/badseo/scripts/run-audit.ts @@ -46,7 +46,7 @@ interface CrawlLink { async function warmup(): Promise { // Prime the dev server so healthy pages don't read as slow on a cold start. - const paths = new Set(["/", "/catalog"]); + const paths = new Set(["/", "/catalog", "/privacy"]); for (const f of allFixtures) for (const p of fixturePaths(f)) paths.add(p); await Promise.all( [...paths].map((p) => @@ -332,9 +332,10 @@ async function main() { }); }; - // Homepage + catalog must be clean. + // Non-fixture content pages must be clean. check("Homepage", "/", [], false); check("Catalog", "/catalog", [], false); + check("Privacy policy", "/privacy", [], false); const byCategory = new Map(); for (const f of allFixtures) { diff --git a/badseo/src/analytics.ts b/badseo/src/analytics.ts new file mode 100644 index 00000000..c643bd6b --- /dev/null +++ b/badseo/src/analytics.ts @@ -0,0 +1,788 @@ +/* eslint-disable max-lines, max-lines-per-function -- This module emits one self-contained browser script; splitting the template would obscure execution order across consent defaults, persistence, and tag loading. */ +// Privacy-first Google Analytics bootstrap for badseo.dev. +// +// A new analytics grant never loads the Google tag until the Worker has saved +// an operator-accessible consent event. Later page views use the unexpired, +// matching-version necessary receipt in browser storage. The consent interface +// is created in the browser so it does not change the raw HTML signals that +// badseo.dev's crawler fixtures exercise. + +import { + ANALYTICS_CONSENT_NOTICE, + CONSENT_RETENTION_DAYS, + CONSENT_STORAGE_KEY, + CONSENT_STORAGE_VERSION, + LEGACY_CONSENT_STORAGE_KEYS, +} from "./consent"; + +const MEASUREMENT_ID_PATTERN = /^G-[A-Z0-9]+$/; +const CONSENT_WITHDRAWAL_STORAGE_PREFIX = "badseo:analytics-withdrawal:v1:"; + +export function isValidGa4MeasurementId( + value: string | undefined, +): value is string { + return value !== undefined && MEASUREMENT_ID_PATTERN.test(value); +} + +export function buildAnalyticsScript(measurementId: string): string { + // The id is validated before this function is called and JSON-encoded before + // insertion so a misconfigured Worker binding cannot become script content. + const encodedMeasurementId = JSON.stringify(measurementId); + + return `(() => { + "use strict"; + + const measurementId = ${encodedMeasurementId}; + const consentEndpoint = "/analytics-consent"; + const notice = ${JSON.stringify(ANALYTICS_CONSENT_NOTICE)}; + const storageKey = ${JSON.stringify(CONSENT_STORAGE_KEY)}; + const withdrawalStoragePrefix = ${JSON.stringify( + CONSENT_WITHDRAWAL_STORAGE_PREFIX, + )}; + const legacyStorageKeys = ${JSON.stringify(LEGACY_CONSENT_STORAGE_KEYS)}; + const storageVersion = ${CONSENT_STORAGE_VERSION}; + const preferenceMaxAgeMs = ${CONSENT_RETENTION_DAYS} * 24 * 60 * 60 * 1000; + const googleTagElementId = "badseo-google-analytics"; + const uuidPattern = /^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i; + let analyticsLoaded = false; + let consentPanel = null; + let consentStatus = null; + let consentRecord = null; + let settingsButton = null; + let restoreSettingsFocus = false; + + window.dataLayer = window.dataLayer || []; + window.gtag = window.gtag || function () { + window.dataLayer.push(arguments); + }; + + // Consent Mode v2 defaults must be queued before any config or event. This + // is Basic Consent Mode: the Google tag remains completely unloaded while + // consent is unknown or denied, so no pre-consent ping is sent to Google. + window.gtag("consent", "default", { + analytics_storage: "denied", + ad_storage: "denied", + ad_user_data: "denied", + ad_personalization: "denied" + }); + window.gtag("js", new Date()); + + function clearStoredConsent() { + consentRecord = null; + try { + window.localStorage.removeItem(storageKey); + for (const legacyKey of legacyStorageKeys) { + window.localStorage.removeItem(legacyKey); + } + } catch { + // Storage may be unavailable in hardened/private browser contexts. + } + } + + function readConsentRecord() { + try { + const stored = window.localStorage.getItem(storageKey); + if (!stored) return null; + const record = JSON.parse(stored); + const updatedAt = Date.parse(record && record.updatedAt); + const hasValidChoice = + record && (record.choice === "granted" || record.choice === "denied"); + const hasValidReceipt = + !record.receiptId || uuidPattern.test(record.receiptId); + const grantHasReceipt = + record.choice !== "granted" || uuidPattern.test(record.receiptId || ""); + const grantMatchesMeasurement = + record.choice !== "granted" || record.measurementId === measurementId; + const isCurrent = + Number.isFinite(updatedAt) && + Date.now() - updatedAt >= 0 && + Date.now() - updatedAt <= preferenceMaxAgeMs; + + if ( + record.version === storageVersion && + record.noticeVersion === notice.noticeVersion && + record.privacyPolicyVersion === notice.privacyPolicyVersion && + hasValidChoice && + hasValidReceipt && + grantHasReceipt && + grantMatchesMeasurement && + isCurrent + ) { + return record; + } + } catch { + // Fail closed and ask again if the stored record cannot be parsed. + } + clearStoredConsent(); + return null; + } + + function persistConsentRecord(record) { + consentRecord = record; + try { + window.localStorage.setItem(storageKey, JSON.stringify(record)); + for (const legacyKey of legacyStorageKeys) { + window.localStorage.removeItem(legacyKey); + } + return true; + } catch { + // Never leave an older grant behind if an overwrite fails (for example, + // because storage is full). Callers must fail closed when this returns + // false so a grant is never used without its withdrawal receipt. + clearStoredConsent(); + return false; + } + } + + function readConsentStorageSnapshot() { + try { + return window.localStorage.getItem(storageKey); + } catch { + return null; + } + } + + function createRandomUuid() { + if (typeof window.crypto.randomUUID === "function") { + return window.crypto.randomUUID(); + } + const bytes = new Uint8Array(16); + window.crypto.getRandomValues(bytes); + bytes[6] = (bytes[6] & 0x0f) | 0x40; + bytes[8] = (bytes[8] & 0x3f) | 0x80; + const hex = Array.from(bytes, (byte) => byte.toString(16).padStart(2, "0")); + return [ + hex.slice(0, 4).join(""), + hex.slice(4, 6).join(""), + hex.slice(6, 8).join(""), + hex.slice(8, 10).join(""), + hex.slice(10).join("") + ].join("-"); + } + + function cleanPageUrl(value) { + if (!value) return ""; + try { + const url = new URL(value, window.location.origin); + if (url.protocol !== "http:" && url.protocol !== "https:") return ""; + return url.origin + url.pathname; + } catch { + return ""; + } + } + + async function postConsentEvent(action, receiptId, options) { + const requestId = options && options.requestId + ? options.requestId + : createRandomUuid(); + const keepalive = Boolean(options && options.keepalive); + const eventMeasurementId = + options && options.measurementId + ? options.measurementId + : measurementId; + const controller = keepalive ? null : new AbortController(); + const timeout = controller + ? window.setTimeout(() => controller.abort(), 8000) + : null; + + try { + const response = await window.fetch(consentEndpoint, { + method: "POST", + credentials: "same-origin", + cache: "no-store", + keepalive, + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + action, + receiptId, + requestId, + noticeVersion: notice.noticeVersion, + privacyPolicyVersion: notice.privacyPolicyVersion, + measurementId: eventMeasurementId + }), + signal: controller ? controller.signal : undefined + }); + if (response.status === 409) { + const error = new Error("Consent notice changed"); + error.name = "ConsentVersionMismatch"; + throw error; + } + if (!response.ok) throw new Error("Consent record was not saved"); + const result = await response.json(); + if ( + !result || + result.receiptId !== receiptId || + result.action !== action || + !Number.isFinite(Date.parse(result.recordedAt)) + ) { + throw new Error("Consent record response was invalid"); + } + return result; + } finally { + if (timeout !== null) window.clearTimeout(timeout); + } + } + + function readPendingWithdrawals() { + const pending = []; + try { + for (let index = 0; index < window.localStorage.length; index++) { + const key = window.localStorage.key(index); + if (!key || !key.startsWith(withdrawalStoragePrefix)) continue; + + const receiptId = key.slice(withdrawalStoragePrefix.length); + const raw = window.localStorage.getItem(key); + let entry = null; + try { + entry = raw ? JSON.parse(raw) : null; + } catch { + // Invalid marker is removed below. + } + + const createdAt = Date.parse(entry && entry.createdAt); + const isCurrent = + Number.isFinite(createdAt) && + Date.now() - createdAt >= 0 && + Date.now() - createdAt <= preferenceMaxAgeMs; + const isValid = + uuidPattern.test(receiptId) && + entry && + uuidPattern.test(entry.requestId || "") && + /^G-[A-Z0-9]+$/.test(entry.measurementId || "") && + isCurrent; + + if (isValid) { + pending.push({ + key, + receiptId, + requestId: entry.requestId, + measurementId: entry.measurementId, + createdAt: entry.createdAt + }); + } else { + window.localStorage.removeItem(key); + index -= 1; + } + } + } catch { + // Storage may be unavailable. The caller still keeps analytics denied. + } + return pending; + } + + function queuePendingWithdrawal(receiptId, eventMeasurementId) { + const entry = { + key: withdrawalStoragePrefix + receiptId, + receiptId, + requestId: createRandomUuid(), + measurementId: eventMeasurementId, + createdAt: new Date().toISOString() + }; + try { + window.localStorage.setItem( + entry.key, + JSON.stringify({ + requestId: entry.requestId, + measurementId: entry.measurementId, + createdAt: entry.createdAt + }) + ); + return { entry, persisted: true }; + } catch { + return { entry, persisted: false }; + } + } + + function removePendingWithdrawalMarker(entry) { + try { + const currentRaw = window.localStorage.getItem(entry.key); + const current = currentRaw ? JSON.parse(currentRaw) : null; + if (current && current.requestId === entry.requestId) { + window.localStorage.removeItem(entry.key); + } + } catch { + // The server event or newer browser decision remains authoritative. + } + } + + async function sendPendingWithdrawal(entry) { + try { + await postConsentEvent("withdraw", entry.receiptId, { + keepalive: true, + requestId: entry.requestId, + measurementId: entry.measurementId + }); + removePendingWithdrawalMarker(entry); + return true; + } catch { + // A persisted marker stays available for the next page view. + return false; + } + } + + async function flushPendingWithdrawals() { + const pending = readPendingWithdrawals(); + const activeReceipt = + consentRecord && consentRecord.choice === "granted" + ? consentRecord.receiptId + : null; + const toSend = []; + for (const entry of pending) { + if ( + activeReceipt === entry.receiptId && + consentRecord.measurementId === entry.measurementId + ) { + // A newer grant for the same receipt supersedes this older retry. + removePendingWithdrawalMarker(entry); + } else { + toSend.push(entry); + } + } + const results = await Promise.all(toSend.map(sendPendingWithdrawal)); + return results.every(Boolean); + } + + function recordWithdrawalWithRetry(receiptId, eventMeasurementId) { + let queued; + try { + queued = queuePendingWithdrawal(receiptId, eventMeasurementId); + } catch { + return; + } + + if (queued.persisted) { + void sendPendingWithdrawal(queued.entry); + } else { + // Storage failure cannot prevent an immediate best-effort write. + void postConsentEvent("withdraw", receiptId, { + keepalive: true, + requestId: queued.entry.requestId, + measurementId: eventMeasurementId + }).catch(() => {}); + } + } + + function enableAnalytics() { + if (analyticsLoaded) return; + analyticsLoaded = true; + + window.gtag("consent", "update", { + analytics_storage: "granted", + ad_storage: "denied", + ad_user_data: "denied", + ad_personalization: "denied" + }); + window.gtag("config", measurementId, { + allow_google_signals: false, + allow_ad_personalization_signals: false, + page_location: cleanPageUrl(window.location.href), + page_referrer: cleanPageUrl(document.referrer) + }); + + const googleTag = document.createElement("script"); + googleTag.id = googleTagElementId; + googleTag.async = true; + googleTag.src = + "https://www.googletagmanager.com/gtag/js?id=" + + encodeURIComponent(measurementId); + document.head.appendChild(googleTag); + } + + function deleteAnalyticsCookies() { + const names = document.cookie + .split(";") + .map((cookie) => cookie.trim().split("=")[0]) + .filter((name) => name === "_ga" || name.startsWith("_ga_")); + const host = window.location.hostname; + const domains = new Set([host, "." + host]); + const parts = host.split("."); + if (parts.length > 1) domains.add("." + parts.slice(-2).join(".")); + + for (const name of names) { + document.cookie = name + "=; Max-Age=0; Path=/; SameSite=Lax"; + for (const domain of domains) { + document.cookie = + name + + "=; Max-Age=0; Path=/; Domain=" + + domain + + "; SameSite=Lax"; + } + } + } + + function applyDeniedConsent() { + window.gtag("consent", "update", { + analytics_storage: "denied", + ad_storage: "denied", + ad_user_data: "denied", + ad_personalization: "denied" + }); + deleteAnalyticsCookies(); + } + + function hideConsentPanel() { + if (!consentPanel) return; + consentPanel.remove(); + consentPanel = null; + consentStatus = null; + document.documentElement.classList.remove("analytics-consent-open"); + if (restoreSettingsFocus && settingsButton) settingsButton.focus(); + restoreSettingsFocus = false; + } + + function setPanelBusy(busy) { + if (!consentPanel) return; + consentPanel.setAttribute("aria-busy", String(busy)); + for (const button of consentPanel.querySelectorAll("button")) { + button.disabled = busy; + } + } + + function showConsentError(message) { + if (!consentStatus) return; + consentStatus.textContent = message; + consentStatus.hidden = false; + } + + async function chooseAnalytics(choice) { + // Another same-origin tab may have changed the preference since this page + // loaded. Decisions must use the current receipt, not cached page state. + consentRecord = readConsentRecord(); + + if (choice === "granted") { + const grantStorageSnapshot = readConsentStorageSnapshot(); + setPanelBusy(true); + if (consentStatus) consentStatus.hidden = true; + + try { + const withdrawalsSaved = await flushPendingWithdrawals(); + if (!withdrawalsSaved) { + throw new Error("A previous withdrawal is still being saved"); + } + + // A newer cross-tab choice made during withdrawal preflight wins + // before this tab creates any new server grant. + if (readConsentStorageSnapshot() !== grantStorageSnapshot) { + const latestRecord = readConsentRecord(); + consentRecord = latestRecord; + hideConsentPanel(); + if (latestRecord && latestRecord.choice === "granted") { + enableAnalytics(); + } else { + applyDeniedConsent(); + } + return; + } + + const receiptId = + consentRecord && uuidPattern.test(consentRecord.receiptId || "") + ? consentRecord.receiptId + : createRandomUuid(); + const result = await postConsentEvent("grant", receiptId); + + // A newer decision from another tab always wins. The server may have + // recorded this grant already, so append a compensating withdrawal, + // leave the newer browser record untouched, and never enable GA from + // this superseded response. + if (readConsentStorageSnapshot() !== grantStorageSnapshot) { + const latestRecord = readConsentRecord(); + consentRecord = latestRecord; + hideConsentPanel(); + if (latestRecord && latestRecord.choice === "granted") { + // A newer valid grant already authorizes this measurement ID. Do + // not append a withdrawal for a receipt that may be shared by both + // concurrent re-grants. + if (latestRecord.receiptId !== receiptId) { + recordWithdrawalWithRetry(receiptId, measurementId); + } + enableAnalytics(); + } else { + recordWithdrawalWithRetry(receiptId, measurementId); + applyDeniedConsent(); + } + return; + } + + const preferenceSaved = persistConsentRecord({ + choice: "granted", + version: storageVersion, + noticeVersion: notice.noticeVersion, + privacyPolicyVersion: notice.privacyPolicyVersion, + measurementId, + receiptId, + updatedAt: result.recordedAt + }); + if (!preferenceSaved) { + throw new Error("Browser storage could not save the consent receipt"); + } + hideConsentPanel(); + enableAnalytics(); + } catch (error) { + if (error && error.name === "ConsentVersionMismatch") { + showConsentError( + "The consent notice changed while this page was open. Reloading it now." + ); + window.setTimeout(() => window.location.reload(), 250); + return; + } + showConsentError( + "Analytics stayed off because we could not save your choice. Please try again." + ); + setPanelBusy(false); + } + return; + } + + const wasLoaded = analyticsLoaded; + const wasGranted = + wasLoaded || Boolean(consentRecord && consentRecord.choice === "granted"); + const receiptId = + consentRecord && uuidPattern.test(consentRecord.receiptId || "") + ? consentRecord.receiptId + : null; + const receiptMeasurementId = + consentRecord && consentRecord.measurementId + ? consentRecord.measurementId + : measurementId; + const pendingWithdrawal = Boolean(wasGranted && receiptId); + // Withdrawal takes effect before any fallible receipt or storage work. + applyDeniedConsent(); + clearStoredConsent(); + + persistConsentRecord({ + choice: "denied", + version: storageVersion, + noticeVersion: notice.noticeVersion, + privacyPolicyVersion: notice.privacyPolicyVersion, + measurementId: receiptMeasurementId, + ...(receiptId ? { receiptId } : {}), + updatedAt: new Date().toISOString() + }); + hideConsentPanel(); + if (pendingWithdrawal && receiptId) { + recordWithdrawalWithRetry(receiptId, receiptMeasurementId); + } + + // Once gtag.js has loaded it cannot be unloaded. Reload into the persisted + // denied state so subsequent interactions send no requests to Google. + if (wasLoaded) window.setTimeout(() => window.location.reload(), 150); + } + + function makeChoiceButton(label, choice, variant) { + const button = document.createElement("button"); + button.type = "button"; + button.className = + "analytics-consent__choice analytics-consent__choice--" + variant; + button.textContent = label; + button.addEventListener("click", () => { + void chooseAnalytics(choice); + }); + return button; + } + + function showConsentPanel(fromSettings) { + if (consentPanel) { + if (fromSettings) { + const firstChoice = consentPanel.querySelector("button"); + if (firstChoice) firstChoice.focus(); + } + return; + } + + restoreSettingsFocus = fromSettings; + const panel = document.createElement("section"); + panel.className = "analytics-consent"; + panel.setAttribute("role", "region"); + panel.setAttribute("aria-labelledby", "analytics-consent-title"); + panel.setAttribute("aria-describedby", "analytics-consent-description"); + + const copy = document.createElement("div"); + copy.className = "analytics-consent__copy"; + + const eyebrow = document.createElement("span"); + eyebrow.className = "analytics-consent__eyebrow"; + eyebrow.textContent = notice.category.toUpperCase(); + + const title = document.createElement("strong"); + title.id = "analytics-consent-title"; + title.className = "analytics-consent__title"; + title.textContent = notice.title; + + const description = document.createElement("p"); + description.id = "analytics-consent-description"; + description.className = "analytics-consent__description"; + description.append(notice.description + " "); + const privacyLink = document.createElement("a"); + privacyLink.href = notice.privacyUrl; + privacyLink.textContent = notice.privacyLinkLabel; + description.appendChild(privacyLink); + + const status = document.createElement("p"); + status.className = "analytics-consent__status"; + status.setAttribute("role", "status"); + status.hidden = true; + consentStatus = status; + + copy.append(eyebrow, title, description, status); + + const choices = document.createElement("div"); + choices.className = "analytics-consent__choices"; + choices.append( + makeChoiceButton(notice.choices.reject, "denied", "reject"), + makeChoiceButton(notice.choices.accept, "granted", "accept") + ); + + panel.append(copy, choices); + consentPanel = panel; + document.documentElement.classList.add("analytics-consent-open"); + document.body.appendChild(panel); + + if (fromSettings) { + const firstChoice = panel.querySelector("button"); + if (firstChoice) firstChoice.focus(); + } + } + + function addSettingsButton() { + const footerLinks = document.querySelector(".foot-links"); + if (settingsButton) return; + const button = document.createElement("button"); + button.type = "button"; + button.className = "analytics-settings"; + button.textContent = "Cookie settings"; + button.addEventListener("click", () => showConsentPanel(true)); + if (footerLinks) { + footerLinks.appendChild(button); + } else { + // A few deliberate crawler fixtures render without the shared footer + // (notably the no-outgoing-links page). Inject a browser-only fallback so + // withdrawal remains available without changing their raw link graph. + button.classList.add("analytics-settings--floating"); + document.body.appendChild(button); + } + settingsButton = button; + } + + function handleConsentStorageChange(event) { + if (event.key !== storageKey) return; + + const nextRecord = readConsentRecord(); + consentRecord = nextRecord; + + if (nextRecord && nextRecord.choice === "granted") { + hideConsentPanel(); + enableAnalytics(); + return; + } + + // A rejection, withdrawal, invalidation, or removal in another tab takes + // effect here immediately. A loaded Google tag requires a clean reload. + applyDeniedConsent(); + hideConsentPanel(); + if (analyticsLoaded) { + window.setTimeout(() => window.location.reload(), 0); + } else if (!nextRecord && document.readyState !== "loading") { + showConsentPanel(false); + } + } + + window.addEventListener("storage", handleConsentStorageChange); + + try { + for (const legacyKey of legacyStorageKeys) { + window.localStorage.removeItem(legacyKey); + } + } catch { + // Storage may be unavailable; the current page still fails closed. + } + + consentRecord = readConsentRecord(); + void flushPendingWithdrawals(); + if (consentRecord && consentRecord.choice === "granted") { + enableAnalytics(); + } else { + // Remove identifiers left by an expired, invalid, or previously withdrawn + // grant even though the Google tag remains unloaded. + deleteAnalyticsCookies(); + } + + function initializeInterface() { + addSettingsButton(); + if (!consentRecord) showConsentPanel(false); + } + + if (document.readyState === "loading") { + document.addEventListener("DOMContentLoaded", initializeInterface, { + once: true + }); + } else { + initializeInterface(); + } +})();`; +} + +/** + * Remove state left by a previously-enabled GA deployment. This script makes + * disabling any required binding a privacy-safe off switch instead of leaving + * old grants and first-party identifiers behind until they expire. + */ +export function buildDisabledAnalyticsScript(): string { + return `(() => { + "use strict"; + try { + for (const key of ${JSON.stringify([ + CONSENT_STORAGE_KEY, + ...LEGACY_CONSENT_STORAGE_KEYS, + ])}) { + window.localStorage.removeItem(key); + } + for (let index = window.localStorage.length - 1; index >= 0; index--) { + const key = window.localStorage.key(index); + if (key && key.startsWith(${JSON.stringify( + CONSENT_WITHDRAWAL_STORAGE_PREFIX, + )})) { + window.localStorage.removeItem(key); + } + } + } catch { + // Storage may be unavailable; cookie cleanup can still continue. + } + + const names = document.cookie + .split(";") + .map((cookie) => cookie.trim().split("=")[0]) + .filter((name) => name === "_ga" || name.startsWith("_ga_")); + const host = window.location.hostname; + const domains = new Set([host, "." + host]); + const parts = host.split("."); + if (parts.length > 1) domains.add("." + parts.slice(-2).join(".")); + + for (const name of names) { + document.cookie = name + "=; Max-Age=0; Path=/; SameSite=Lax"; + for (const domain of domains) { + document.cookie = + name + + "=; Max-Age=0; Path=/; Domain=" + + domain + + "; SameSite=Lax"; + } + } +})();`; +} + +export function analyticsScriptResponse( + measurementId: string | undefined, + consentLedgerAvailable: boolean, +): Response { + const headers = { + "content-type": "application/javascript; charset=utf-8", + "cache-control": "no-store", + "x-content-type-options": "nosniff", + }; + + if (!isValidGa4MeasurementId(measurementId) || !consentLedgerAvailable) { + return new Response(buildDisabledAnalyticsScript(), { headers }); + } + + return new Response(buildAnalyticsScript(measurementId), { headers }); +} diff --git a/badseo/src/consent-log.ts b/badseo/src/consent-log.ts new file mode 100644 index 00000000..1b461e00 --- /dev/null +++ b/badseo/src/consent-log.ts @@ -0,0 +1,235 @@ +import { + ANALYTICS_CONSENT_NOTICE, + CONSENT_EVIDENCE_RETENTION_SECONDS, + CONSENT_SCHEMA_VERSION, +} from "./consent"; + +const MAX_REQUEST_BYTES = 1024; +const UUID_PATTERN = + /^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i; + +type ConsentAction = "grant" | "withdraw"; + +interface ConsentRequestBody { + action: ConsentAction; + receiptId: string; + requestId: string; + noticeVersion: string; + privacyPolicyVersion: string; + measurementId: string; +} + +function jsonResponse(body: unknown, status: number): Response { + return Response.json(body, { + status, + headers: { + "cache-control": "no-store", + "x-content-type-options": "nosniff", + }, + }); +} + +function isConsentRequestBody(value: unknown): value is ConsentRequestBody { + if (!value || typeof value !== "object" || Array.isArray(value)) return false; + return ( + "action" in value && + (value.action === "grant" || value.action === "withdraw") && + "receiptId" in value && + typeof value.receiptId === "string" && + UUID_PATTERN.test(value.receiptId) && + "requestId" in value && + typeof value.requestId === "string" && + UUID_PATTERN.test(value.requestId) && + "noticeVersion" in value && + typeof value.noticeVersion === "string" && + "privacyPolicyVersion" in value && + typeof value.privacyPolicyVersion === "string" && + "measurementId" in value && + typeof value.measurementId === "string" && + /^G-[A-Z0-9]+$/.test(value.measurementId) + ); +} + +async function readLimitedRequestBody( + request: Request, +): Promise { + if (!request.body) return ""; + + const reader = request.body.getReader(); + const decoder = new TextDecoder(); + let bytesRead = 0; + let text = ""; + + try { + while (true) { + const result: unknown = await reader.read(); + if ( + !result || + typeof result !== "object" || + !("done" in result) || + typeof result.done !== "boolean" + ) { + return null; + } + if (result.done) break; + if (!("value" in result) || !(result.value instanceof Uint8Array)) { + return null; + } + bytesRead += result.value.byteLength; + if (bytesRead > MAX_REQUEST_BYTES) { + await reader.cancel("Request body is too large"); + return null; + } + text += decoder.decode(result.value, { stream: true }); + } + text += decoder.decode(); + return text; + } finally { + reader.releaseLock(); + } +} + +function requestOrigin(request: Request, url: URL): string { + const host = request.headers.get("host") ?? url.host; + return `${url.protocol}//${host}`; +} + +export async function recordAnalyticsConsent( + request: Request, + url: URL, + env: Env, +): Promise { + if (request.method !== "POST") { + return new Response(null, { + status: 405, + headers: { + allow: "POST", + "cache-control": "no-store", + }, + }); + } + + const origin = request.headers.get("origin"); + const fetchSite = request.headers.get("sec-fetch-site"); + if ( + origin !== requestOrigin(request, url) || + (fetchSite !== null && fetchSite !== "same-origin") + ) { + return jsonResponse({ error: "Forbidden" }, 403); + } + + const contentType = request.headers + .get("content-type") + ?.split(";", 1)[0] + .trim() + .toLowerCase(); + if (contentType !== "application/json") { + return jsonResponse({ error: "Expected application/json" }, 415); + } + + const contentLength = request.headers.get("content-length"); + if (contentLength !== null) { + if (!/^\d+$/.test(contentLength)) { + return jsonResponse({ error: "Invalid Content-Length" }, 400); + } + if (Number(contentLength) > MAX_REQUEST_BYTES) { + return jsonResponse({ error: "Request body is too large" }, 413); + } + } + + // Rate-limit before reading any body bytes. The IP is used only by + // Cloudflare's short-lived counter and is never written to the consent log. + const rateLimitKey = request.headers.get("cf-connecting-ip") ?? "unknown"; + const rateLimit = await env.CONSENT_RATE_LIMITER.limit({ key: rateLimitKey }); + if (!rateLimit.success) { + return jsonResponse({ error: "Too many consent requests" }, 429); + } + + const rawBody = await readLimitedRequestBody(request); + if (rawBody === null) + return jsonResponse({ error: "Request body is too large" }, 413); + + let parsed: unknown; + try { + parsed = JSON.parse(rawBody); + } catch { + return jsonResponse({ error: "Invalid JSON" }, 400); + } + if (!isConsentRequestBody(parsed)) { + return jsonResponse({ error: "Invalid consent record" }, 400); + } + + // A page can remain open across a deployment. Never attest that it showed + // the new notice when its browser script actually rendered an older one. + if ( + parsed.action === "grant" && + (parsed.noticeVersion !== ANALYTICS_CONSENT_NOTICE.noticeVersion || + parsed.privacyPolicyVersion !== + ANALYTICS_CONSENT_NOTICE.privacyPolicyVersion || + parsed.measurementId !== env.GA4_MEASUREMENT_ID) + ) { + return jsonResponse( + { error: "Consent notice changed", code: "consent_version_mismatch" }, + 409, + ); + } + + const recordedAt = new Date().toISOString(); + const eventId = crypto.randomUUID(); + const record = { + schemaVersion: CONSENT_SCHEMA_VERSION, + eventId, + requestId: parsed.requestId, + receiptId: parsed.receiptId, + action: parsed.action, + analyticsGranted: parsed.action === "grant", + recordedAt, + measurementId: parsed.measurementId, + ...(parsed.action === "grant" + ? { notice: ANALYTICS_CONSENT_NOTICE } + : { + noticeReference: { + noticeVersion: parsed.noticeVersion, + privacyPolicyVersion: parsed.privacyPolicyVersion, + measurementId: parsed.measurementId, + }, + }), + }; + + // Every attempt receives a new server event id and therefore a unique key. + // A retried browser request may produce a clearly marked duplicate with the + // same requestId, but it can never overwrite an earlier grant or withdrawal. + const key = [ + "analytics-consent", + `v${CONSENT_SCHEMA_VERSION}`, + parsed.receiptId, + `${recordedAt}-${eventId}`, + ].join("/"); + + try { + await env.CONSENT_LOG.put(key, JSON.stringify(record), { + expirationTtl: CONSENT_EVIDENCE_RETENTION_SECONDS, + }); + } catch (error) { + console.error( + JSON.stringify({ + event: "analytics_consent_write_failed", + action: parsed.action, + message: error instanceof Error ? error.message : "Unknown error", + }), + ); + return jsonResponse( + { error: "Analytics consent could not be recorded" }, + 503, + ); + } + + return jsonResponse( + { + receiptId: parsed.receiptId, + action: parsed.action, + recordedAt, + }, + 201, + ); +} diff --git a/badseo/src/consent.ts b/badseo/src/consent.ts new file mode 100644 index 00000000..5d0ae583 --- /dev/null +++ b/badseo/src/consent.ts @@ -0,0 +1,38 @@ +// Versioned, canonical consent copy shared by the browser UI and the +// operator-side consent ledger. Never edit an existing version in place: bump +// `noticeVersion` when the wording, vendor, or purpose materially changes so a +// stored record continues to prove exactly what a visitor saw. + +export const CONSENT_SCHEMA_VERSION = 1; +export const CONSENT_STORAGE_VERSION = 2; +export const CONSENT_STORAGE_KEY = "badseo:analytics-consent:v2"; +export const LEGACY_CONSENT_STORAGE_KEYS = [ + "badseo:analytics-consent:v1", +] as const; + +export const CONSENT_RETENTION_DAYS = 400; +// Keep operator-side evidence long enough to cover the 400-day browser grant +// plus the configured 14-month GA4 event-retention window. This is separate +// from the shorter browser preference and should be revisited with counsel if +// either downstream retention or the legal evidence requirement changes. +export const CONSENT_EVIDENCE_RETENTION_DAYS = 830; +export const CONSENT_EVIDENCE_RETENTION_SECONDS = + CONSENT_EVIDENCE_RETENTION_DAYS * 24 * 60 * 60; + +export const ANALYTICS_CONSENT_NOTICE = { + noticeVersion: "2026-07-10.2", + privacyPolicyVersion: "2026-07-10", + site: "https://badseo.dev", + category: "Analytics / optional", + title: "Help us see which broken pages people use.", + description: + "If accepted, Google Analytics, provided by Google, sets cookies to measure visits, page use, and outbound clicks. It stays off unless you accept.", + provider: "Google Analytics 4 (Google LLC)", + purpose: "Measure visits, page use, and outbound links on badseo.dev.", + choices: { + reject: "Reject", + accept: "Accept analytics", + }, + privacyUrl: "/privacy", + privacyLinkLabel: "Privacy details", +} as const; diff --git a/badseo/src/index.ts b/badseo/src/index.ts index 5197f986..d4a8fd21 100644 --- a/badseo/src/index.ts +++ b/badseo/src/index.ts @@ -7,7 +7,10 @@ import { STYLESHEET } from "./styles"; import { OPENSEO_LOGO_PNG_BASE64 } from "./logo"; import { renderHome, renderCatalog } from "./pages"; +import { renderPrivacy } from "./privacy"; import { renderShell, redirect } from "./lib"; +import { analyticsScriptResponse, isValidGa4MeasurementId } from "./analytics"; +import { recordAnalyticsConsent } from "./consent-log"; import { TRAILING_SLASH_CANONICAL } from "./fixtures/redirects"; import { allFixtures, @@ -56,7 +59,7 @@ function sitemapXml(origin: string): Response { // click-depth — which would propagate down the chain and stop the deep-page // fixture from ever reaching depth >= 5. Keeping the catalog link-only means // the chain gets real, incrementing depths. - const paths = new Set(["/"]); + const paths = new Set(["/", "/privacy"]); for (const fixture of sitemapFixtures) { for (const path of fixturePaths(fixture)) paths.add(path); } @@ -96,7 +99,7 @@ function notFoundPage(): Response { } export default { - async fetch(request: Request): Promise { + async fetch(request: Request, env: Env): Promise { const url = new URL(request.url); // Derive the origin from the Host header, not url.origin: under `wrangler // dev` with custom-domain routes configured, url.origin resolves to the @@ -106,6 +109,21 @@ export default { const origin = `${url.protocol}//${host}`; const path = normalizePath(url.pathname); + // Consent preferences use origin-scoped browser storage. Keep one + // canonical production origin so www and apex can never disagree about a + // grant or withdrawal. + const requestHostname = host.split(":", 1)[0].toLowerCase(); + const isProductionHost = + requestHostname === "badseo.dev" || requestHostname === "www.badseo.dev"; + if ( + isProductionHost && + (requestHostname === "www.badseo.dev" || url.protocol !== "https:") + ) { + return redirect(`https://badseo.dev${url.pathname}${url.search}`, 308); + } + + const analyticsEnabled = String(env.GA4_ADMIN_VERIFIED) === "true"; + // Trailing-slash canonical: the non-slash form 301-redirects to the slash // form, which is served as the canonical 200 (via the fixture below, since // normalizePath maps the slash form to the same route). Checked on the RAW @@ -116,6 +134,26 @@ export default { } switch (path) { + case "/analytics.js": + return analyticsScriptResponse( + env.GA4_MEASUREMENT_ID, + Boolean( + analyticsEnabled && env.CONSENT_LOG && env.CONSENT_RATE_LIMITER, + ), + ); + case "/analytics-consent": + if ( + !analyticsEnabled || + !isValidGa4MeasurementId(env.GA4_MEASUREMENT_ID) || + !env.CONSENT_LOG || + !env.CONSENT_RATE_LIMITER + ) { + return new Response(null, { + status: 404, + headers: { "cache-control": "no-store" }, + }); + } + return recordAnalyticsConsent(request, url, env); case "/styles.css": return new Response(STYLESHEET, { headers: { @@ -150,6 +188,8 @@ export default { return html(renderHome()); case "/catalog": return html(renderCatalog()); + case "/privacy": + return html(renderPrivacy()); } const fixture = routeTable.get(path); @@ -160,4 +200,4 @@ export default { return notFoundPage(); }, -}; +} satisfies ExportedHandler; diff --git a/badseo/src/lib.ts b/badseo/src/lib.ts index 0e6cb1b4..b03d3db0 100644 --- a/badseo/src/lib.ts +++ b/badseo/src/lib.ts @@ -58,12 +58,10 @@ export function renderDocument(opts: DocumentOptions): string { head.push(``); if (opts.robotsMeta) head.push(``); - head.push( - '', - '', - '', - ); - head.push(''); + // Version the stylesheet whenever shared chrome changes. The consent UI is + // injected by fresh, no-store JavaScript and must never meet stale CSS. + head.push(''); + head.push(''); if (opts.headExtra) head.push(opts.headExtra); return ` @@ -102,6 +100,7 @@ function footerHtml(): string { Catalog GitHub OpenSEO + Privacy `; } diff --git a/badseo/src/pages.ts b/badseo/src/pages.ts index 7d31c57e..d0f3514e 100644 --- a/badseo/src/pages.ts +++ b/badseo/src/pages.ts @@ -1,6 +1,5 @@ -// The two non-fixture pages: the homepage and the catalog. Both must audit -// CLEAN — they're the crawl entry point and the hub that links every fixture, -// so any accidental issue here would show up in the e2e run. +// The primary non-fixture content pages. They must audit CLEAN because they +// are part of the same crawl as the deliberately broken fixtures. import { renderShell, escapeHtml } from "./lib"; import { AUDIT_ISSUE_TYPES } from "../../src/shared/audit-issues"; import { diff --git a/badseo/src/privacy.ts b/badseo/src/privacy.ts new file mode 100644 index 00000000..dce41955 --- /dev/null +++ b/badseo/src/privacy.ts @@ -0,0 +1,78 @@ +import { renderShell } from "./lib"; + +/** Standalone privacy policy for badseo.dev and its optional GA4 analytics. */ +export function renderPrivacy(): string { + return renderShell({ + title: "Privacy policy | badseo.dev", + metaDescription: + "How badseo.dev uses optional Google Analytics, necessary consent storage, and Cloudflare infrastructure.", + bodyHtml: ``, + }); +} diff --git a/badseo/src/styles.ts b/badseo/src/styles.ts index 4575e468..ef9240bc 100644 --- a/badseo/src/styles.ts +++ b/badseo/src/styles.ts @@ -1,7 +1,7 @@ // Served verbatim at /styles.css. Kept as a module string so the Worker stays -// dependency-free. Palette + type mirror the OpenSEO marketing site (web/): -// a warm "cream" canvas, ink text, one orange accent, Inter, hairline borders, -// small radii, no shadows. +// dependency-free. The warm canvas, ink text, orange accent, hairline borders, +// and compact editorial typography mirror the OpenSEO marketing site without +// making any pre-consent font request to a third party. export const STYLESHEET = ` :root { --canvas: #f5f1ec; @@ -13,12 +13,13 @@ export const STYLESHEET = ` --hairline: #d8d1c8; --hairline-soft: #ebe7e1; --orange: #ff5600; + --orange-ink: #a63a00; --footer-bg: #eee8de; --critical: #d23b1f; --warning: #b26a00; --info: #6b7280; - --mono: "JetBrains Mono", ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; - --sans: Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; + --mono: "SFMono-Regular", Consolas, "Liberation Mono", Menlo, monospace; + --sans: "Avenir Next", Avenir, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; } * { box-sizing: border-box; } @@ -91,6 +92,39 @@ a:hover { text-decoration: underline; text-underline-offset: 3px; } .next { margin-top: 8px; } .next a { font-weight: 500; } +/* ── privacy policy: an editorial, plain-language legal memo ── */ +.legal-page { max-width: 760px; } +.legal-head { padding: 20px 0 12px; } +.legal-kicker { + display: block; color: var(--orange-ink); font-family: var(--mono); + font-size: 11px; font-weight: 600; letter-spacing: 0.11em; +} +.legal-version { + display: flex; gap: 12px; align-items: baseline; + padding-bottom: 18px; border-bottom: 1px solid var(--hairline); + color: var(--ink-muted); font-family: var(--mono); font-size: 12px; +} +.legal-version span { + color: var(--ink-subtle); font-size: 10px; letter-spacing: 0.08em; + text-transform: uppercase; +} +.legal-summary { + margin: 28px 0 42px; padding: 17px 19px 18px; + border: 1px solid var(--hairline); border-top: 4px solid var(--orange); + border-radius: 8px; background: var(--surface); +} +.legal-summary strong { + display: block; margin-bottom: 5px; font-family: var(--mono); + font-size: 11px; letter-spacing: 0.08em; text-transform: uppercase; +} +.legal-summary p { margin: 0; } +.legal-page section { + padding-top: 34px; border-top: 1px solid var(--hairline); +} +.legal-page section + section { margin-top: 34px; } +.legal-page section h2 { margin-top: 0; } +.legal-page section:last-child { padding-bottom: 12px; } + /* ── test panel ── */ .panel { margin-top: 8px; @@ -191,7 +225,113 @@ a:hover { text-decoration: underline; text-underline-offset: 3px; } } .foot-inner a { color: var(--ink-muted); } .foot-inner a:hover { color: var(--ink); } -.foot-links { display: flex; gap: 18px; } +.foot-links { display: flex; gap: 18px; flex-wrap: wrap; align-items: baseline; } + +.analytics-settings { + appearance: none; border: 0; padding: 0; background: transparent; + color: var(--ink-muted); font: inherit; line-height: inherit; cursor: pointer; + text-decoration: none; +} +.analytics-settings:hover { + color: var(--ink); text-decoration: underline; text-underline-offset: 3px; +} +.analytics-settings--floating { + position: fixed; z-index: 60; left: 20px; bottom: 20px; + padding: 8px 10px; border: 1px solid var(--hairline); border-radius: 6px; + background: var(--surface); box-shadow: 3px 3px 0 rgba(17, 17, 17, 0.12); +} +.analytics-settings--floating:hover { background: var(--surface-2); } +.analytics-settings:focus-visible, +.analytics-consent a:focus-visible, +.analytics-consent__choice:focus-visible { + outline: 3px solid var(--orange); outline-offset: 3px; +} + +/* ── optional analytics consent ── */ +.analytics-consent { + position: fixed; z-index: 100; left: 50%; bottom: 18px; + width: min(760px, calc(100vw - 36px)); + transform: translateX(-50%); + display: grid; grid-template-columns: minmax(0, 1fr) auto; + gap: 24px; align-items: end; + padding: 20px 20px 20px 22px; + border: 1px solid var(--ink); border-top: 4px solid var(--orange); + border-radius: 10px; background: var(--surface); color: var(--ink); + box-shadow: 8px 8px 0 rgba(17, 17, 17, 0.18); + animation: analytics-consent-enter 180ms ease-out both; + max-height: calc(100dvh - 20px); overflow-y: auto; + overscroll-behavior: contain; +} +.analytics-consent__copy { min-width: 0; } +.analytics-consent__eyebrow { + display: block; margin-bottom: 6px; color: var(--orange-ink); + font-family: var(--mono); font-size: 10.5px; font-weight: 500; + letter-spacing: 0.1em; +} +.analytics-consent__title { + display: block; font-size: 17px; line-height: 1.3; font-weight: 600; + letter-spacing: -0.012em; +} +.analytics-consent__description { + max-width: 560px; margin: 6px 0 0; color: var(--ink-muted); + font-size: 13.5px; line-height: 1.45; +} +.analytics-consent__description a { + color: var(--ink); text-decoration: underline; + text-decoration-color: var(--hairline); text-underline-offset: 3px; +} +.analytics-consent__description a:hover { text-decoration-color: var(--ink); } +.analytics-consent__status { + margin: 9px 0 0; color: #9e2f18; + font-size: 12.5px; line-height: 1.4; font-weight: 600; +} +.analytics-consent__choices { + display: grid; grid-template-columns: 1fr 1fr; gap: 8px; + min-width: 270px; +} +.analytics-consent__choice { + appearance: none; min-height: 42px; margin: 0; padding: 9px 13px; + border: 1px solid var(--ink); border-radius: 7px; + font: 600 13px/1.2 var(--sans); cursor: pointer; + transition: transform 120ms ease, background-color 120ms ease, color 120ms ease; +} +.analytics-consent__choice:hover { transform: translateY(-1px); } +.analytics-consent__choice:disabled { + cursor: wait; opacity: 0.58; transform: none; +} +.analytics-consent__choice--reject, +.analytics-consent__choice--accept { + background: var(--surface); color: var(--ink); +} +.analytics-consent__choice--reject:hover, +.analytics-consent__choice--accept:hover { background: var(--surface-2); } +.analytics-consent-open .openseo-badge { + visibility: hidden; +} +.analytics-consent-open .analytics-settings--floating { + visibility: hidden; +} +@keyframes analytics-consent-enter { + from { opacity: 0; transform: translate(-50%, 10px); } + to { opacity: 1; transform: translate(-50%, 0); } +} + +@media (max-width: 720px) { + .analytics-consent { + grid-template-columns: 1fr; gap: 16px; align-items: stretch; + padding: 18px; + } + .analytics-consent__choices { min-width: 0; } +} +@media (max-width: 420px) { + .analytics-consent { bottom: 10px; width: calc(100vw - 20px); } + .analytics-consent__choices { grid-template-columns: 1fr; } + .analytics-settings--floating { left: 10px; bottom: 10px; } +} +@media (prefers-reduced-motion: reduce) { + .analytics-consent { animation: none; } + .analytics-consent__choice { transition: none; } +} .openseo-badge { position: fixed; right: 20px; bottom: 20px; z-index: 50; diff --git a/badseo/src/worker-env.d.ts b/badseo/src/worker-env.d.ts new file mode 100644 index 00000000..290914dc --- /dev/null +++ b/badseo/src/worker-env.d.ts @@ -0,0 +1,15 @@ +/* eslint-disable */ +// Generated by Wrangler by running `wrangler types --config=badseo/wrangler.jsonc --env=production --include-runtime=false badseo/src/worker-env.d.ts` (hash: a86cf968e75c0ed194ddfb87c6855162) +interface __BaseEnv_Env { + CONSENT_LOG: KVNamespace; + CONSENT_RATE_LIMITER: RateLimit; + GA4_MEASUREMENT_ID: "G-7MXV9FH7SS"; + GA4_ADMIN_VERIFIED: "false"; +} +declare namespace Cloudflare { + interface GlobalProps { + mainModule: typeof import("./index"); + } + interface Env extends __BaseEnv_Env {} +} +interface Env extends __BaseEnv_Env {} diff --git a/badseo/wrangler.jsonc b/badseo/wrangler.jsonc index 427d0b1d..aab337e8 100644 --- a/badseo/wrangler.jsonc +++ b/badseo/wrangler.jsonc @@ -1,10 +1,25 @@ { "$schema": "node_modules/wrangler/config-schema.json", - "name": "badseo", + // Keep the root/preview Worker distinct from the production custom-domain + // Worker so a plain `wrangler deploy` cannot replace production bindings. + "name": "badseo-dev", "main": "src/index.ts", - "compatibility_date": "2025-06-01", + // Latest compatibility date supported by this repo's pinned workerd. + "compatibility_date": "2026-07-02", "workers_dev": true, "preview_urls": true, + // Wrangler provisions separate local and production namespaces. Consent + // events are append-only and expire after the retention period in code. + "kv_namespaces": [{ "binding": "CONSENT_LOG" }], + "ratelimits": [ + { + "name": "CONSENT_RATE_LIMITER", + "namespace_id": "172901", + // Local browser tests share one loopback address and exercise many + // consent transitions in a minute. Production keeps the strict limit. + "simple": { "limit": 100, "period": 60 }, + }, + ], "observability": { "enabled": true, }, @@ -14,8 +29,25 @@ // Deploy with: wrangler deploy --env production "env": { "production": { - "name": "badseo", - "workers_dev": true, + // Preserve the Worker identity created by the original implicit + // `-production` environment naming convention. + "name": "badseo-production", + "workers_dev": false, + "preview_urls": false, + // Flip the verification gate only after completing and checking every + // GA Admin item documented in README.md. + "vars": { + "GA4_MEASUREMENT_ID": "G-7MXV9FH7SS", + "GA4_ADMIN_VERIFIED": "false", + }, + "kv_namespaces": [{ "binding": "CONSENT_LOG" }], + "ratelimits": [ + { + "name": "CONSENT_RATE_LIMITER", + "namespace_id": "172902", + "simple": { "limit": 5, "period": 60 }, + }, + ], "observability": { "enabled": true }, "routes": [ { "pattern": "badseo.dev", "custom_domain": true },