fix: harden signup against bots and undeliverable recipients

[FrkAk]

Summary

Task Reference: [PYZ-91]

Three defences on the sign-up path.

Turnstile bot protection. Registers the Better Auth captcha plugin on /sign-up/email, /sign-in/email, /send-verification-email, and /request-password-reset. Widget mode managed with appearance: "interaction-only", so ordinary visitors see nothing and only flagged visitors get an interactive challenge. Boot-gated on TURNSTILE_SECRET_KEY: unset, no plugin is registered and no widget renders, so self-host is unchanged.

Recipient deliverability gate (PYZ-364). A DNS-over-HTTPS probe runs on /sign-up/email before the user row is written and rejects domains that cannot receive mail: MX exchanges resolving only to unreachable address space, RFC 7505 null MX, and NXDOMAIN. No MX with a routable A/AAAA is accepted per the RFC 5321 §5.1 implicit-MX fallback. Exchanges are probed in MX preference order under a 4s whole-probe deadline, and only a fully probed exchange list can be asserted undeliverable. Resolver errors, SERVFAIL, timeouts, and unparseable MX answers fail open, and the probe is skipped entirely when signups are disabled.

Routability is decided by lib/auth/ip-routability.ts, which parses an address to bytes once and tests CIDR containment against an IANA-derived table covering both families. Numeric containment is what catches the same address written a different way: ::ffff:7f00:1, 64:ff9b::7f00:1, and 2002:7f00:1:: all embed 127.0.0.1 and all read as routable under textual prefix matching. The table covers loopback, RFC 1918, link-local, CGNAT, the TEST-NET blocks, 240/4, multicast, 198.18/15, 192.88.99/24, 192.0.0/24, and IPv6 unique-local, discard, Teredo, benchmarking, 6to4, NAT64, and both documentation blocks.

Both address families for an exchange resolve concurrently. Serialised, one exchange cost two round trips and a domain whose first exchange is dead cost five, which overran the whole-probe deadline and returned unknown for domains that were merely slow. Settled verdicts are memoized per domain in-process, longer for deliverable than undeliverable; unknown is never cached.

Per-recipient email budget (PYZ-365). Caps auth email per recipient per template per hour, enforced at deliverAuthEmail, the single chokepoint that also covers the server-action senders which bypass Better Auth rate limiting. The budget counts delivered mail: a slot is claimed before dispatch and committed only once the provider accepts, so a provider outage cannot exhaust a recipient's allowance and leave them unable to verify. Caps are per template, with a higher allowance for team invitations so someone joining several teams in an hour is not silently dropped. Keys are SHA-256 digests, so no address reaches KV. Over-budget sends are dropped and logged with a hashed recipient handle. KV outage fails open in both directions.

Rate limiting is unchanged.

Type of change

• Bug fix
• New feature
• Refactor / cleanup
• Documentation

Testing

• Tested locally with bun run dev
• Linting passes (bun run lint)
• Typecheck passes (bun run typecheck)
• Workers smoke passes (bun run build:cf && bun run smoke:cf)

bun test: 2215 pass, 0 fail, ~45s.

New test files: tests/auth/ip-routability.test.ts (every range in the table, the CGNAT and 172/12 boundaries on both sides, and the hex-form embeddings), tests/auth/recipient-domain.test.ts (probe against canned DoH zones, including null MX versus unparseable MX, preference-order probing, concurrent address-family resolution, and the verdict cache), tests/auth/signup-deliverability.test.ts (gate wiring), tests/auth/turnstile-config.test.ts (plugin gating and order, the endpoint list read from the exported constant, CSP, a handler-level tokenless 400 on every protected endpoint, and the verification arms driven through a stubbed siteverify: valid token, hostname mismatch, failed verification, and transport outage), tests/ui/turnstile-state.test.ts (submit gating, the captcha header name, and the copy pairing), tests/security/turnstile-symmetry.test.ts (both directions of the deploy gate), tests/security/middleware.test.ts (CSP scoping), tests/email/budget.test.ts and tests/email/budget-workers.test.ts (caps, key scoping, the read/commit split, TTL floor, fail-open).

Client half verified in a browser against Cloudflare's published testing keys on a local dev run: the widget loads under the production CSP shape, stays invisible in interaction-only, delivers its token via x-captcha-response, and the form recovers after a captcha failure. The forced-interactive key confirms the escalated challenge renders below the submit control and leaves it in place, and that the challenge spans the auth column rather than sitting short of it. The deliverability gate was exercised against live DNS: an NXDOMAIN domain is rejected with the friendly message before any row is written, and a deliverable domain completes sign-up.

Not yet exercised: a provisioned production widget, so managed-mode escalation behaviour remains unobserved; the Workers build, so the KV budget store and the DoH probe have not run on workerd; and a real sub-350px viewport, where the challenge is expected to scroll within its own strip rather than widen the page.

Notes for reviewer

The captcha plugin fails closed. A siteverify outage returns 500, and sign-in is protected, so login fails with it for the whole deployment, with a 10s timeout bounding each blocked request. A Turnstile outage also stops the client widget from minting tokens, so rollback drops both halves: delete the TURNSTILE_SECRET_KEY Worker secret, clear the environment's TURNSTILE_SITE_KEY_* variable, and redeploy. That symmetry is gated on both heads: assert-deploy-ready covers production, and scripts/turnstile-symmetry.ts --env dev runs on the dev deploy, which previously had no gate at all.

The deliverability probe rejects an exchange that resolves only to unreachable address space, and also one that resolves to nothing at all; neither is a delivery target. Parked and anti-spam domains publish MX records pointing at loopback, which resolve normally and are what Cloudflare reports as transport_none.

Role-address prefixes (admin@, info@) are not blocked; prefix matching produces false positives on legitimate customer-domain addresses.

setRecipientDomainResolver exists so the suite stays hermetic. Without it the probe makes live DNS calls on every auth test and rejects the suite's @test.local addresses as NXDOMAIN. tests/setup/preload.ts installs a permissive default through a dynamic import, which must remain dynamic because a static import hoists above the server-only neutralization.

Better Auth 1.6.23 matches captcha endpoints by substring rather than exact path, so the protected list is explicit and exported. Of the four entries, /sign-in/email collides with /sign-in/email-otp and /request-password-reset collides with the emailOTP and phoneNumber variants; Better Auth exempts only the first internally and neither plugin is enabled here. v1.7 switches to exact-and-wildcard matching and drops that exemption, which is a no-op for a list of exact paths.

The plugin forwards the caller's address to siteverify as remoteip, taken from advanced.ipAddress, which masks IPv6 to a /64. IPv6 visitors therefore send a network rather than an address. Cloudflare documents remoteip as optional with no mismatch handling, and 1.6.23 offers no per-call opt-out short of disabling IP tracking globally.

Under appearance: "interaction-only" the widget library styles its own wrapper width: fit-content; display: flex, which prevents size: "flexible" from expanding. The gate replaces that wrapper, and centres the iframe with auto inline margins because Tailwind's preflight makes iframes block-level and immune to text-align.

KV read-modify-write is not atomic and propagates in roughly 60s, so a caller spread across POPs can exceed the nominal budget. A dropped send is silent to the caller by design, so auth_email_budget_exceeded is the only signal that a real user is being capped.

Follow-ups filed separately: PYZ-366, PYZ-367, PYZ-368.

Docs impact

• TURNSTILE_SECRET_KEY: per-env Worker secret. Not unconditionally required: the deploy gates enforce that it and the build-time site key are present together or absent together, on both production and dev.
• TURNSTILE_SITE_KEY: build-time, surfaced to the client as NEXT_PUBLIC_TURNSTILE_SITE_KEY, supplied per environment from the TURNSTILE_SITE_KEY_PROD and TURNSTILE_SITE_KEY_DEV Actions variables. It is part of the build-artefact cache key, so rotating the widget cannot restore a bundle carrying the previous value.
• Self-host is unaffected and Turnstile stays off unless both keys are set. .env.local.example documents the pair as commented entries, including that the dashboard widget must be Managed mode for the invisible-until-needed behaviour.

Compliance impact

Adds a third-party client-side service on the auth pages. Turnstile loads a script and iframe from challenges.cloudflare.com and processes client IP, TLS fingerprint, User-Agent, and sitekey with origin.

The privacy policy's cookie claims were reviewed and hold. Pre-clearance is not enabled, which is the default, so Turnstile issues a one-time token and no cf_clearance cookie is set on this domain; no first-party cookie is set either. Cloudflare separately documents cf_chl_rc_i / cf_chl_rc_ni / cf_chl_rc_m as Challenge Platform cookies, so this is not a zero-cookie integration and the claim is scoped to what is verifiable. The policy's technical-data categories, the security purpose under Art. 6(1)(f), and the Art. 22 statement cover Turnstile as written, so content/legal/privacy.md and lib/legal/versions.ts are untouched and no re-consent is triggered. Enabling pre-clearance later would set cf_clearance and require this section to be revisited.

content/legal/subprocessors.md now names bot protection (Turnstile) in the existing Cloudflare entry. Same entity, same data categories, so this is a list refresh, not a new sub-processor under the Art. 28(2) notice process.