Skip to content

P0: production email delivery dead since 2026-06-18 — invalid RESEND_API_KEY hard-locks signup and password reset #1298

Description

@teetangh

Summary

Production email delivery has been completely broken since 2026-06-18 (~10 weeks). The RESEND_API_KEY on familiarisenow.com is set but invalid — Resend rejects every send with "API key is invalid".

Because requireEmailVerification: true (lib/auth.ts:97), a credential signup receives no session until the emailed link is clicked. With email dead, this is not a degraded experience — it is a hard, inescapable lockout that blocks the entire booking/checkout funnel at the auth wall. Password reset runs through the same transport, so existing users who mistype a password are also permanently locked out.

18 of 249 production users are currently stuck at emailVerified: false with no way out.

Evidence

From the production database (Supabase project pzmbxqdgibfkhjwzeprf, table FailedEmail):

SELECT "emailType", status, count(*) AS n, min("createdAt"), max("createdAt")
FROM "FailedEmail" GROUP BY 1,2 ORDER BY n DESC;
emailType status count attempts lastError
EMAIL_VERIFICATION DEAD_LETTER 16 5 API key is invalid
WELCOME DEAD_LETTER 7 5 API key is invalid
ACCOUNT_LINKED DEAD_LETTER 1 5 API key is invalid
PAYMENT_SUCCESS DEAD_LETTER 1 5 API key is invalid
  • First failure: 2026-06-18 07:29:32
  • Most recent: 2026-08-23 05:01:37
  • Every row has attempts: 5 and status: DEAD_LETTER — the retry worker has exhausted itself. Nothing will self-heal.
  • fromAddress on every row is @familiarise.com.

User state:

SELECT count(*) total,
       count(*) FILTER (WHERE "emailVerified") verified,
       count(*) FILTER (WHERE NOT "emailVerified") unverified
FROM users;
-- total: 249 | verified: 231 | unverified: 18

The key is definitively present but invalid, not absent: the message got as far as being rendered and handed to Resend (which is what populates FailedEmail). The absent-key path returns earlier — see Fix 3 below.

The lockout loop

  1. User clicks "Book now" on an expert profile → bounced to /auth/signin?callbackUrl=/checkout/...
  2. Chooses sign-up → account row is created at emailVerified: false, no session issued (requireEmailVerification: true)
  3. UI shows "Check your email" — the email never sends
  4. Any sign-in attempt returns EMAIL_NOT_VERIFIED
  5. The page offers "resend the link below" → hits the same dead key → goto 3

No exit. And the recovery route (password reset) is equally dead, so this also traps users whose accounts are verified.

Reproduction

Prerequisite: production, or any environment with an invalid RESEND_API_KEY. Not reproducible on localhost if a valid key is present.

  1. Go to https://familiarisenow.com/explore/experts, open any expert profile
  2. Pick a slot, click Book now → redirected to /auth/signin?callbackUrl=/checkout/plans/consultation/<id>?startsAt=...
  3. Click Sign up, register with a fresh email address
  4. Observe "Check your email" panel. No email arrives (verify: a new FailedEmail row appears with lastError = "API key is invalid")
  5. Attempt to sign in with those credentials → EMAIL_NOT_VERIFIED, "Verify your email" banner
  6. Click the in-page resend → still nothing
  7. Try Forgot password → no email either

Confirm server-side with:

SELECT "emailType", "fromAddress", "lastError", attempts, "createdAt"
FROM "FailedEmail" ORDER BY "createdAt" DESC LIMIT 5;

Suggested fixes

Fix 1 — Rotate RESEND_API_KEY in Netlify production (confidence: CERTAIN this is the root cause; requires dashboard access)

Generate a fresh key in Resend, set it in Netlify production env, redeploy. This alone restores delivery if Fix 2 is not also in play.

Fix 2 — Reconcile the sender domain (confidence: MEDIUM-HIGH that this is a second, latent blocker)

Every sender hardcodes @familiarise.com:

lib/email.ts:21, 143, 201, 267, 324, 400, 484, 571, 634, 661 (plus :782 support@familiarise.com)

…but the live site is familiarisenow.com. If familiarise.com is not a verified sending domain on the Resend account, sends will keep failing after the key rotation — just with a different error message. I have not been able to inspect the Resend account, hence the confidence caveat.

Verify the domain in Resend before rotating, so this isn't debugged twice. Then move the addresses to a single env-driven constant rather than 11 hardcoded string literals.

Fix 3 — Make an unconfigured key fail loudly (confidence: CERTAIN, from code reading)

lib/email.ts:255-264 (and the identical block in sendPasswordResetEmail) returns { success: false } before sentMessage is assigned when the key is missing:

const resend = getResendClient();
if (!resend) {
  console.error("RESEND_API_KEY is not configured. Cannot send verification email.");
  return { success: false, error: "Email service not configured" };  // <-- no dead-letter, no Sentry
}

Since recordFailedEmail is only reached via the catch and is guarded by if (sentMessage), an absent key produces no FailedEmail row and no Sentry event — a totally silent outage. Better Auth's hook awaits a resolved value and treats the send as successful, so signup reports success.

This is not the current failure mode (our key is invalid, not missing), but it is the same class of bug and would hide the next occurrence completely. Suggest: build sentMessage first, or throw so the existing catch dead-letters it.

Fix 4 — Alert on delivery failure (confidence: MEDIUM — needs verification)

recordFailedEmail does call Sentry.captureException with tags: { subsystem: "email" }, level: "warning" (lib/email.ts:47-54). Yet no email-subsystem issue appears in Sentry for familiarise_web over 90 days, despite 25 dead-lettered rows. Either the captures aren't arriving or my searches missed them — worth confirming directly. A 10-week total email outage should not be discoverable only by manually querying a table.

Suggest a cheap alert on FailedEmail rows in DEAD_LETTER, plus a startup assertion that the key is valid.

Fix 5 — Unstick the 18 affected users (confidence: HIGH on the approach)

After Fixes 1-2 land, these users still cannot get in on their own — their old verification links are long expired.

⚠️ Do NOT bulk-replay the DEAD_LETTER rows. Verification tokens expire in 1 hour (emailVerification.expiresIn) and reset tokens in 30 minutes (resetPasswordTokenExpiresIn), so every queued message contains a dead link. Replaying them sends 25 emails that all fail on click.

Instead: prompt affected users to re-request verification, or flip emailVerified for the ones whose addresses are known-good.

Ruled out during investigation

For the next person, so this ground isn't re-covered:

  • Checkout-resume / callbackUrl handling is fine. The #booking-journey train (7b4973ab), the auth-flicker fix (be043a7c), the P1 audit fixes (c69378b3), and the checkout dead-end fix (b7223228) are all deployed on origin/prod (e9a664eb, release 2026-08-27). utils/purchase-intent.ts and lib/safe-callback-url.ts both exist there. Nothing to backport.
  • Email-verification encoding chain is correct. Better Auth encodes the nested callbackURL properly (node_modules/better-auth/dist/api/routes/email-verification.mjs:28-29, redirect at :259), so /checkout/...?startsAt=X&endsAt=Y survives verification intact.
  • No role gate at checkout. app/api/checkout/route.ts:33 calls only requireApiAuth().
  • OAuth path recovers correctly via requireOnboarded() + x-pathname (lib/auth-guard.ts:77-92).

Separate pre-existing gaps found along the way

Worth their own issues; not the cause of this outage:

  1. Forgot-password drops callbackUrl entirely. app/auth/signin/page.tsx:494 (bare link), app/auth/forgot-password/page.tsx:43 (redirectTo hardcoded) and :127, app/auth/reset-password/page.tsx:83 and :194. A guest who resets mid-checkout loses their purchase context. Also forgot-password/page.tsx:30 sends an authenticated visitor to /dashboard.
  2. requireNotOnboarded() redirects to a hardcoded /dashboard (lib/auth-guard.ts:167) with no callbackUrl preservation, unlike its requireOnboarded() sibling (:77-92). An asymmetry that will drop purchase intent again.
  3. utils/purchase-intent.ts uses sessionStorage (tab-scoped), but email verification forces a hop through the user's email client, which opens a new tab. For the request-for-approval flow — whose slot lives only in that stash — the intent cannot survive signup by design. The direct-Buy flow is unaffected (slot is in the URL).

Severity

Launch blocker. No new user has been able to complete signup on production since 2026-06-18.

Metadata

Metadata

Assignees

No one assigned

    Labels

    authAuthentication, sessions, RBAC, account lifecyclebugSomething isn't workingcriticalCritical priority itemsinfrastructureInfrastructure, deployment, and DevOpslaunch: pre-mvpGates launch — money, data, or a failure we would not detectpriority: criticalproductionProduction deployment and readiness

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions