feat(ops): migrate Postgres to Neon.tech — verified end-to-end (#50)

.env.example

@@ -21,6 +21,11 @@ DB_NAME=elevareai
 DB_USER=postgres
 DB_PASSWORD=postgres
 
+# Neon.tech Postgres (production/#50 migration): set DATABASE_URL to the
+# Neon POOLED connection string with ?sslmode=require; it overrides the
+# DB_* parts above. Never commit a real value.
+# DATABASE_URL=postgresql://USER:PASSWORD@ep-xxx-pooler.REGION.aws.neon.tech/DB?sslmode=require
+
 # AI provider (OpenRouter)
 # Free-tier model with no rate limits. Base URL defaults to https://openrouter.ai/api/v1
 # (set OPENROUTER_BASE_URL environment variable to override).

README.md

@@ -489,36 +489,44 @@ npm run dev
 
 ### Deploy to Render (Free Tier)
 
+> **Note:** Postgres is now hosted on [Neon.tech](https://neon.tech) (a
+> persistent free tier), not Render's free Postgres. Render's Blueprint now
+> provisions only the web service and static site below; `DATABASE_URL` is
+> set manually in the dashboard to point at Neon. See
+> `_docs/RUNBOOK-neon-migration.md` for the full migration/setup runbook.
+
 1. **Push to GitHub** — Commit and push this repository to GitHub
 2. **Create Blueprint** — Log in to [Render Dashboard](https://dashboard.render.com), select **New → Blueprint**, and connect this repository
 3. **Auto-Provisioning** — Render deploys via `render.yaml`:
-   - **elevareai-db** — Free PostgreSQL instance
    - **elevareai-api** — FastAPI backend (Python web service)
    - **elevareai-frontend** — React/Vite static site
 4. **Set Manual Secrets** — In Render dashboard, add to **elevareai-api** environment variables:
+   - `DATABASE_URL` — your Neon pooled connection string (see `_docs/RUNBOOK-neon-migration.md`)
    - `OPENROUTER_API_KEY` (format: `sk-or-v1-...`) — your OpenRouter API key
    - `DEMO_PASSWORD` — demo account password (only if seeding demo data)
    - *JWT_SECRET is auto-generated by Render*
 5. **Verify Service URLs** — Confirm default URLs from `render.yaml`:
    - API: `https://elevareai-api.onrender.com`
    - Frontend: `https://elevareai-frontend.onrender.com`
    - If you renamed services or added custom domains, update `ALLOWED_ORIGINS` (API) and `VITE_API_BASE_URL` (Frontend) in the dashboard
-6. **Seed Demo Data** — Follow [Database Recreation Runbook](#database-recreation-render-free-postgresql) to populate demo accounts
+6. **Seed Demo Data** — Follow `_docs/RUNBOOK-neon-migration.md` to populate demo accounts against Neon
 
 ### Free-Tier Behavior Notes
 - **Pre-demo Warm-up**: Web services spin down after ~15 min idle; first request after idle takes ~50s. Before a demo, wake the backend by running:
   ```bash
   curl https://elevareai-api.onrender.com/health
   ```
   Wait for `{"status":"healthy","database":"connected"}` (may take ~50s). Repeat if needed. Then load the frontend at `https://elevareai-frontend.onrender.com` for a smooth demo experience.
-- **Postgres Expiry**: Free databases delete 30 days after creation, regardless of activity — see runbook for recovery procedure
+- **Database on Neon (persistent)**: Previously hosted on Render's free Postgres (which expired 30 days after creation), now migrated to Neon.tech with no expiry. See `_docs/RUNBOOK-neon-migration.md` for setup and `_docs/RUNBOOK-db-expiry-recovery.md` for the legacy recovery procedure if reverting.
 - **AI Latency**: Free OpenRouter model takes ~20s per response
 
 ### Legacy Deployment Paths
 AWS deployment guides (ECS/Cognito/SES) are in `_docs/guides/` for reference only — use Render above for current deployments.
 
 ### Database Recreation (Render Free PostgreSQL)
 
+> **Legacy (pre-Neon):** This procedure describes recreating a Render PostgreSQL database from an External URL and applies only if you revert to Render Postgres or are maintaining a legacy Render-based deployment. The current database is hosted on Neon.tech (persistent, no expiry). See `_docs/RUNBOOK-neon-migration.md` for the current setup, and `_docs/RUNBOOK-db-expiry-recovery.md` if you need the full legacy Render-Postgres recovery procedure.
+
 Render's free PostgreSQL databases are deleted 30 days after creation, regardless of activity. Restore a demo-ready database in ~5 minutes:
 
 1. Create new PostgreSQL database on [Render Dashboard](https://dashboard.render.com) and copy the External Database URL

_docs/ARCHITECTURE.md

@@ -44,7 +44,7 @@ Math practice is the exception: it is generated **deterministically with SymPy**
 
 **Frontend.** The frontend (`examples/frontend-starter/`) is a React 18 + Vite SPA using React Query for server-state, React Router for navigation, react-markdown for rendering AI responses, recharts for progress visualizations, and axios for HTTP.
 
-**Hosting.** The app is deployed on Render's free tier using a `render.yaml` Blueprint (infrastructure-as-code) that provisions three resources: a free Postgres database (`elevareai-db`), the FastAPI backend as a Python web service (`elevareai-api`), and the React frontend as a static site (`elevareai-frontend`). The database is wired to the backend via `DATABASE_URL` (Render's `connectionString` property), which the app normalizes from `postgres://` to `postgresql://` for SQLAlchemy 2.x. Two secrets — `OPENROUTER_API_KEY` and `DEMO_PASSWORD` — must be entered manually in the Render dashboard after the first deploy; `JWT_SECRET` is auto-generated by Render; CORS is controlled via `ALLOWED_ORIGINS`. Free-tier caveats are documented directly in `render.yaml`: web services spin down after ~15 minutes of inactivity (cold starts up to ~1 minute), and free Postgres databases are deleted 30 days after creation.
+**Hosting.** The app is deployed on Render's free tier using a `render.yaml` Blueprint (infrastructure-as-code) that provisions two resources: the FastAPI backend as a Python web service (`elevareai-api`) and the React frontend as a static site (`elevareai-frontend`). Postgres is hosted separately on [Neon.tech](https://neon.tech) (a persistent free tier, no expiry) rather than Render's free Postgres — see `_docs/RUNBOOK-neon-migration.md` for the migration/setup runbook. The database is wired to the backend via `DATABASE_URL` (a manually-set secret pointing at Neon's pooled connection string), which the app normalizes from `postgres://` to `postgresql://` for SQLAlchemy 2.x. Three secrets — `DATABASE_URL`, `OPENROUTER_API_KEY`, and `DEMO_PASSWORD` — must be entered manually in the Render dashboard after the first deploy; `JWT_SECRET` is auto-generated by Render; CORS is controlled via `ALLOWED_ORIGINS`. Free-tier caveats for the web services are documented directly in `render.yaml`: they spin down after ~15 minutes of inactivity (cold starts up to ~1 minute).
 
 **Cost.** Running cost is effectively $0/month. A one-time $10 OpenRouter credit raises the free-tier request cap to roughly 1,000 requests/day, comfortably covering normal usage.
 
@@ -74,7 +74,7 @@ The current design reflects a deliberate migration away from a broken, AWS-coupl
 - **Concise answers, generous timeouts, non-blocking calls.** Prompts explicitly ask for concise (~250-word) plain-text answers; the OpenRouter client has a 60-second timeout; and blocking LLM calls are offloaded off the async event loop (`run_in_threadpool`) in the QA, practice, and summarizer code paths, keeping the API responsive under concurrent load and avoiding upstream proxy timeouts.
 - **Deterministic math generation.** Math practice items are generated with SymPy rather than the LLM — correct-by-construction, free, and independently verifiable, with no risk of a hallucinated wrong answer being presented as correct.
 - **No credentials in the repo.** All secrets are supplied via `.env` locally or the Render dashboard in production; nothing sensitive is committed.
-- **Idempotent demo-seed script.** A seed script using deterministic UUIDs can turn an empty database into a fully demo-ready state in one command. This directly mitigates Render's free-tier Postgres 30-day deletion — there is a documented runbook for recreating and reseeding the database when it expires.
+- **Idempotent demo-seed script.** A seed script using deterministic UUIDs can turn an empty database into a fully demo-ready state in one command. This originally mitigated Render's free-tier Postgres 30-day deletion (see `_docs/RUNBOOK-db-expiry-recovery.md`); now that Postgres lives on Neon (no expiry, see `_docs/RUNBOOK-neon-migration.md`), the same script is still used for initial setup and demo re-seeding.
 
 ## 5. Testing Approach
 

_docs/RUNBOOK-db-expiry-recovery.md

@@ -1,5 +1,8 @@
 # Database Expiry Recovery Runbook
 
+> If migrated to Neon (#50), this Render-expiry recovery no longer applies;
+> see `RUNBOOK-neon-migration.md`.
+
 **Scope:** Render free-tier PostgreSQL databases expire 30 days after creation. This runbook restores a demo-ready database end-to-end.
 
 ## When to Use

_docs/RUNBOOK-neon-migration.md

@@ -0,0 +1,140 @@
+# Neon.tech Postgres Migration Runbook (#50)
+
+> **STATUS: MIGRATION MECHANICS VERIFIED, CUTOVER PENDING.** On 2026-07-24,
+> steps 1 and 3 (connect via the pooled SSL endpoint, run `setup_db.py` to
+> migrate all 18 tables, seed demo data, and log in as `demo@elevare.ai`
+> with the seeded bcrypt password) were executed and verified end-to-end
+> against a real Neon project. What remains is the owner's Render-dashboard
+> cutover: set `DATABASE_URL` in the Render dashboard (step 2), redeploy and
+> verify `/health` (step 4), confirm demo login/data in production (step 5),
+> and only then remove the Render database (step 6).
+
+**Scope:** Move ElevareAI's Postgres database off Render's free-tier
+Postgres (which expires 30 days after creation, see
+`_docs/RUNBOOK-db-expiry-recovery.md`) onto Neon.tech, which has no such
+expiry. No application code changes are required -- `get_database_url()` in
+`src/config/settings.py` already passes `DATABASE_URL` through verbatim
+(only normalizing `postgres://` to `postgresql://`), so a Neon DSN with
+`?sslmode=require` works unchanged.
+
+## GOTCHA -- do not add unknown keys to `.env`
+
+**Do NOT add `NEON_EXTERNAL_DATABASE_URL` (or any other non-schema key) to
+`.env`.** `src/config/settings.py`'s `Settings` rejects unknown keys by default
+(pydantic-settings' standard behavior; the app does not set `extra="allow"`),
+so an active (uncommented) unknown key crashes the app at startup with a
+`pydantic_core.ValidationError` (`extra_forbidden`) -- and since the
+seed/migration path (`scripts/seed_demo_data.py` -> `scripts/setup_db.py`)
+and the test suite both import through `src/config/settings.py`, an unknown
+key breaks those too, not just the running app. This was hit during
+verification.
+
+For local Neon runs, set the canonical `DATABASE_URL=` key directly. On
+Render, set `DATABASE_URL` in the dashboard (see step 2). If you want to
+keep a Neon connection string in `.env` for reference without activating
+it, comment the line out (leading `#`), the same pattern already used for
+`RENDER_EXTERNAL_DATABASE_URL` above it.
+
+## Ordering -- read this before starting
+
+Steps 1-5 are additive/reversible: at every point up through step 5, the
+existing Render `elevareai-db` is untouched and still there as a fallback.
+**Step 6 (removing the Render database) is destructive and irreversible.**
+Do not perform step 6 until steps 4 and 5 have both succeeded against Neon.
+If you remove the Render database before confirming Neon is serving traffic
+correctly, and something is wrong with the Neon setup, there is no
+fallback -- the data is gone.
+
+## Steps
+
+### 1. Provision a Neon project
+
+- Create a Neon project (https://neon.tech).
+- From the Neon dashboard, copy the **pooled** connection string (the host
+  contains `-pooler`, e.g. `ep-xxx-pooler.us-east-2.aws.neon.tech`) -- not
+  the direct/unpooled one. Ensure it includes `?sslmode=require`.
+- Format: `postgresql://USER:PASSWORD@ep-xxx-pooler.REGION.aws.neon.tech/DB?sslmode=require`
+- Treat this string as a secret; do not commit it anywhere.
+
+### 2. Set DATABASE_URL in the Render dashboard
+
+- In the Render dashboard, go to `elevareai-api` -> Environment.
+- Set `DATABASE_URL` to the Neon pooled connection string from step 1.
+- This PR's `render.yaml` already changes `DATABASE_URL` from a
+  `fromDatabase:` binding to `sync: false` (manual secret) and removes the
+  `databases:` block -- but do NOT apply/sync that blueprint change (or
+  delete the Render database) until step 6. Setting the dashboard env var
+  now is independent of the blueprint sync and does not touch the existing
+  Render database.
+
+### 3. Run migrations and seed data against Neon
+
+Run this from wherever you run the seed script, with the environment
+pointed at Neon. Reference: `_docs/RUNBOOK-db-expiry-recovery.md` step 4
+uses the same script.
+
+```bash
+python scripts/seed_demo_data.py
+```
+
+This runs `scripts/setup_db.py` (schema/migrations) followed by idempotent
+demo-account seeding.
+
+**`DB_*` vs `DATABASE_URL` (resolved):** `scripts/setup_db.py`'s
+`get_db_connection_string()` now prefers `DATABASE_URL` when set, returning
+it verbatim (aside from normalizing `postgres://` to `postgresql://`), the
+same precedence `src/config/database.py`'s `get_database_url()` uses. This
+DSN is used to build the SQLAlchemy `engine`, which `run_migration()` then
+uses via `_psycopg2_params_from_url()` to construct its own `psycopg2`
+connection -- that helper explicitly carries the DSN's query params (e.g.
+`sslmode`, `channel_binding`) into the `psycopg2.connect()` kwargs, so
+Neon's SSL requirements are honored explicitly rather than left to
+psycopg2's default of `sslmode=prefer`. `DB_*` remains a fallback for local
+dev when `DATABASE_URL` is unset. See `tests/test_setup_db_connection.py`
+for coverage.
+
+**Already done:** migration and seeding have already been run against the
+Neon database directly via its pooled URL (see status banner above). Once
+Render's `DATABASE_URL` points at that same Neon database, the data is
+already present -- step 4 below is a connection-string cutover (redeploy +
+verify), not a re-seed, unless you specifically want fresh data.
+
+### 4. Redeploy and verify /health
+
+- Trigger a manual deploy of `elevareai-api` in the Render dashboard (or
+  push a commit) so it picks up the new `DATABASE_URL`.
+- Verify:
+
+```bash
+curl https://elevareai-api.onrender.com/health
+```
+
+Expected: `{"status":"healthy","database":"connected"}`
+
+### 5. Verify demo login and data
+
+- Open the frontend, log in as `demo@elevare.ai` with the `DEMO_PASSWORD`
+  value set in the Render dashboard.
+- Confirm demo data (goals, sessions, practice items) is visible.
+
+### 6. Remove the Render database -- ONLY AFTER 4 AND 5 SUCCEED
+
+> ⚠️ **This step is destructive.** Removing the `databases:` block (applying
+> this PR's `render.yaml` via a Blueprint sync) and/or deleting the Render
+> `elevareai-db` instance permanently deletes that database. Do this LAST,
+> never before Neon is confirmed serving traffic in steps 4-5, or you lose
+> the database with no fallback.
+
+Once Neon is confirmed working:
+- Sync the Blueprint (applies this PR's `render.yaml`, which no longer
+  declares `elevareai-db`), or manually delete the `elevareai-db` instance
+  in the Render dashboard.
+- `elevareai-db` is no longer referenced by anything once `DATABASE_URL`
+  points at Neon.
+
+### 7. Relationship to the #34 expiry-recovery runbook
+
+This migration supersedes the 30-day Render-Postgres expiry problem that
+`_docs/RUNBOOK-db-expiry-recovery.md` (#34) addresses. That runbook now
+carries a pointer at the top noting it no longer applies once migrated to
+Neon.

render.yaml

@@ -1,13 +1,19 @@
 # render.yaml — Render Blueprint for ElevareAI
 #
-# Deploys three resources:
-#   1. elevareai-db        — free Postgres instance
-#   2. elevareai-api       — FastAPI backend (Python web service)
-#   3. elevareai-frontend  — React/Vite frontend (static site)
+# Deploys two resources:
+#   1. elevareai-api       — FastAPI backend (Python web service)
+#   2. elevareai-frontend  — React/Vite frontend (static site)
+#
+# Database: Postgres is hosted on Neon.tech (not Render) as of the #50
+# migration. DATABASE_URL is set manually in the Render dashboard (see below).
 #
 # ---------------------------------------------------------------------------
 # MANUAL STEPS REQUIRED AFTER "New Blueprint" DEPLOY (cannot be automated):
 #   - elevareai-api: OPENROUTER_API_KEY  -> set in Render dashboard (secret)
+#   - elevareai-api: DATABASE_URL        -> set in Render dashboard (secret)
+#     to the Neon POOLED connection string with ?sslmode=require, e.g.
+#     postgresql://USER:PASSWORD@ep-xxx-pooler.REGION.aws.neon.tech/DB?sslmode=require
+#     Never commit the actual value. See _docs/RUNBOOK-neon-migration.md.
 #   - elevareai-api: DEMO_PASSWORD       -> set in Render dashboard (secret,
 #     only needed if/when you run the demo-account seed script; harmless to
 #     set even if unused)
@@ -25,18 +31,16 @@
 # FREE TIER NOTES:
 #   - Free web services spin down after ~15 minutes of inactivity; the next
 #     request will incur a cold-start delay (up to ~1 minute).
-#   - Free Postgres databases on Render expire 30 days after creation and
-#     are deleted (not just suspended). Before expiry, upgrade the plan or
-#     back up/migrate data. See the deployment runbook in the README for the
-#     recovery/reseed procedure.
+#   - Postgres is on Neon's free tier, not Render's: the Render-Postgres
+#     30-day-expiry issue this file used to warn about no longer applies.
+#     Neon's free tier is persistent (no expiry) but autosuspends the
+#     compute endpoint after a period of inactivity; the first query after
+#     autosuspend incurs a cold-start delay (roughly half a second per
+#     Neon's docs), which the app's pool_pre_ping=True setting absorbs by
+#     transparently retrying/replacing a stale pooled connection rather than
+#     surfacing an error.
 # ---------------------------------------------------------------------------
 
-databases:
-  - name: elevareai-db
-    plan: free
-    databaseName: elevareai
-    user: elevareai
-
 services:
   # ---------------------------------------------------------------------
   # Backend API (FastAPI / uvicorn)
@@ -49,15 +53,17 @@ services:
     startCommand: "uvicorn src.api.main:app --host 0.0.0.0 --port $PORT"
     healthCheckPath: /health
     envVars:
-      # Database connection — Render's Blueprint spec only supports
-      # connectionString/connectionPoolString/user/password/database for
-      # fromDatabase (not host/port). The app reads DATABASE_URL verbatim
-      # via src/config/settings.py get_database_url(), normalizing the
-      # postgres:// scheme Render uses to postgresql:// for SQLAlchemy 2.x.
+      # Database connection — Postgres is hosted on Neon.tech, not Render,
+      # so this is a manual secret rather than a `fromDatabase:` binding.
+      # Set in the Render dashboard to the Neon POOLED connection string
+      # with ?sslmode=require, e.g.
+      # postgresql://USER:PASSWORD@ep-xxx-pooler.REGION.aws.neon.tech/DB?sslmode=require
+      # Never commit the actual value. The app reads DATABASE_URL verbatim
+      # via src/config/settings.py get_database_url(), normalizing only the
+      # postgres:// scheme to postgresql:// for SQLAlchemy 2.x; sslmode and
+      # the pooler host pass through unchanged.
       - key: DATABASE_URL
-        fromDatabase:
-          name: elevareai-db
-          property: connectionString
+        sync: false
 
       # OpenRouter (LLM provider) — API key must be entered manually in the
       # Render dashboard after the first deploy; it is never committed here.