updating crawler watchdog branch to test

.github/workflows/test.yml

@@ -0,0 +1,40 @@
+name: test
+
+# Fast, hermetic checks only. The maintenance-cycle harness
+# (`npm run e2e:maintenance`) is deliberately excluded: it spawns real
+# crawler and scanner processes against a mock Bungie server and runs for
+# minutes, which is not what a per-push gate is for.
+#
+# NOTE ON ARCHITECTURE: this runs on x86_64, while production is ARM64
+# (Oracle A1 Flex). better-sqlite3 is a native module, so CI compiles and
+# tests a different binary than production runs. Native-level differences
+# in better-sqlite3 will not be caught here.
+
+on:
+  push:
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22
+          cache: npm
+
+      # `ci`, not `install`: the lockfile is the source of truth for
+      # deployments, and a CI run that silently resolved different versions
+      # would be testing something production never sees.
+      - run: npm ci
+
+      - run: npm run lint
+
+      # Nothing else typechecks the repo — `npm test` transpiles without
+      # checking, and a full `next build` is too slow for this gate.
+      - run: npx tsc --noEmit
+
+      - run: npm test

.gitignore

@@ -70,4 +70,13 @@ localhost.key
 # Sentry Config File
 .env.sentry-build-plugin
 
-certificates
+certificates
+
+#handoff docs
+docs/handoffs/
+
+#tickets
+docs/tickets/
+
+# claude plans and skills
+.claude/

CLAUDE.md

@@ -0,0 +1,60 @@
+# Destiny Farm Finder
+
+Real-time Destiny 2 raid completion tracker. SQLite database fed by background crawlers against the Bungie API; Next.js app reads from the DB and serves live leaderboards + active fireteam sessions.
+
+## Note
+
+- NEVER overwrite files in ~/.claude/plans/ 
+
+## Commands
+
+`package.json` has the full list. The non-obvious ones:
+
+- `npm run start` — **web app only.** The crawler/scanner/discovery are separate PM2 processes (`ecosystem.config.js`) and must be started independently.
+- `npm run setup-manifest` — writes `data/manifest-cache.json` for a human to read. Changes nothing about runtime behaviour; see the raid-detection convention below.
+- `npm run e2e:maintenance` — slow, spawns real crawler/scanner processes against a mock Bungie server. Deliberately outside `npm test` and CI.
+
+Scripts run via `tsx` using `tsconfig.scripts.json`. Next.js app and scripts compile separately.
+
+## Conventions that will burn you if missed
+
+- **Player identity is `Name#Code`.** Always store and display `bungie_global_display_name` in full. Partial names were a real bug — see recent commits.
+- **Raid detection is a hardcoded table, not a runtime cache lookup.** `RAID_DEFINITIONS` in `src/lib/bungie/manifest.ts` is a literal map of raid key → name/slug/activity hashes, flattened into a hash→key `Map` at import time. Nothing reads `data/manifest-cache.json` at runtime. Adding a new raid means **editing `manifest.ts`** — `npm run setup-manifest` only writes the cache file for a human to read.
+- **Dedicated scanner key pool** (`BUNGIE_SCANNER_API_KEY`, `_2`) — the scanner rotates only within its own keys, and each key gets its own RPS budget. Don't share scanner keys across other processes.
+- **All SQL lives in `src/lib/db/queries.ts`.** API routes call it directly — no ORM, no service layer.
+- **API cache headers are only half the story** — Cloudflare cache rules (not in this repo) rewrite them, and a rule with no Browser TTL falls back to a 4h zone default. Check `cf-cache-status` and the header prod actually returns before touching `src/lib/http/cache.ts`. See `docs/decisions.md`.
+
+## Env vars
+
+Core: `BUNGIE_API_KEY`, `BUNGIE_SCANNER_API_KEY`, `BUNGIE_SCANNER_API_KEY_2`, `BUNGIE_SCANNER_API_KEY_3`, `BUNGIE_SCANNER_API_KEY_4`, `BUNGIE_DISCOVERY_API_KEY`, `ADMIN_STATS_USERNAME` / `ADMIN_STATS_PASSWORD`, `SEED_PLAYERS`.
+
+Web: `NEXT_PUBLIC_SITE_URL` (default `https://destinyfarmfinder.qzz.io`) — sets `metadataBase` for social-share unfurls **and** the same-origin allowlist for the client-write guard. `NEXT_PUBLIC_BUNGIE_PUBLIC_API_KEY` — public key used by browser-side Bungie calls (profile + LinkedProfiles resolution). `PAGE_TOKEN_SECRET` (optional, server-only) — when set, enables the short-lived HMAC page-token check on the client-write endpoints (`active-session-update`, `players/identity`, `queue-crawl`); when unset, those endpoints still enforce the same-origin check but skip the token layer. See `src/lib/http/request-auth.ts`.
+
+Tuning — the scripts below list every knob and its default. These are the ones whose *value* is load-bearing for a reason the code doesn't explain:
+
+- `SQLITE_BUSY_TIMEOUT_MS` (30000) — how long any process waits on a competing write lock before SQLITE_BUSY. The wait blocks that process's event loop.
+- `CRAWLER_MEMBER_RESOLVE_CONCURRENCY` (4) — member resolution runs concurrently rather than one-at-a-time; sequential resolution of the full `CRAWLER_MEMBER_RESOLVE_LIMIT` could run ~25 × fetch-timeout ≈ 12.5 min and trip the poll watchdog on its own.
+- `BUNGIE_GAME_SERVER_BACKOFF_SEC` (2) — self-imposed per-key pause when Bungie returns ErrorCode 1672 `DestinyThrottledByGameServer`, which arrives with `ThrottleSeconds: 0`.
+- `CRAWLER_CLEANUP_BATCH_SIZE` (500) — expired PGCRs deleted per cleanup transaction, sized to keep each write-lock hold sub-second; one whole-backlog DELETE would hold the write lock for minutes and freeze the crawler's event loop. `CRAWLER_CLEANUP_YIELD_MS` (25) pauses between batches so the session/crawl loops and the scanner interleave.
+
+Active-session display (`/active-sessions` + the StatsBar/OG count): `ACTIVE_SESSION_DISPLAY_LIMIT` (default 600 — max **fireteams** rendered; counted in fireteams, never rows) and `ACTIVE_SESSION_ROW_SCAN_LIMIT` (default 3000 — max raw per-player rows scanned before dedupe). These are two different units and must stay separate: `active_sessions` is keyed by `membership_id`, so one fireteam yields up to 6 rows, and capping rows silently drops the longest-running raids. See `docs/adr/0001-fireteam-denominated-display-cap.md`. Verify with `npx tsx scripts/verify-active-session-limit.ts` (needs the crawler running — 900s freshness window).
+
+Active-session poll backoff (players found offline/private are snoozed instead of re-polled every cycle; maintained by `recordSessionCheck` on `players.next_session_eligible_at`): `SESSION_OFFLINE_BACKOFF_BASE_SEC` (default 120), `SESSION_OFFLINE_BACKOFF_CAP_SEC` (default 960 ≈ 16 min), `SESSION_PRIVACY_BACKOFF_SEC` (default 21600 = 6h).
+
+Full list: `scripts/start-crawler.ts`, `scripts/start-scanner.ts`, `scripts/discover.ts`.
+
+## Active-session loop resilience
+
+The active-session poll loop shares an event loop and a singleton Bungie client with the crawl loop, and a single hung request used to park it indefinitely (observed: overnight stalls) while the crawl heartbeat stayed green. Four guards close this off: a per-poll watchdog, a `finally`-anchored reschedule, a completion-anchored `session_heartbeat`, and treating a timeout as distinct from "offline" so a Bungie storm can't delete live fireteams. Don't remove one without reading [ADR 0005](docs/adr/0005-active-session-loop-resilience.md).
+
+## Backups
+
+There is **no continuous replication.** Backups are manual snapshots: `npx tsx scripts/backup-db.ts` checkpoints the WAL and `VACUUM INTO`s a dated copy alongside the live DB (`data/raid-tracker.backup-YYYY-MM-DD.db`). It needs ~1 DB size of free disk. Restoring is a file move: stop the processes, swap the snapshot into `data/raid-tracker.db` (remove stale `-wal`/`-shm` files), restart. Crawled data is re-crawlable, which is why this is tolerable.
+
+## Testing
+
+Vitest. **`tests/README.md` is the how-to** — read it before writing or changing a test; it covers the two ground rules (mock `fetch` and only `fetch`; `npm test` stays hermetic), the layout, colocate-vs-`tests/`, fixtures-vs-builders, and the naming conventions. The rationale is in [ADR 0003](docs/adr/0003-tests-run-against-a-real-sqlite-file.md) and [ADR 0004](docs/adr/0004-mock-only-at-the-network-boundary.md); the build-out is in `docs/handoffs/testing-framework-handoff.md`. CI runs `npm test` only.
+
+**Never reorder `setupFiles` in `vitest.config.ts`** — `tests/setup/test-db-path.ts` must stay first or the suite binds to the live dev database and `resetTestDb()` deletes from it. `getDb()` enforces this; `tests/README.md` explains why.
+
+Application code was not changed to make anything testable — if a test seems to require that, question it first.

CONTEXT.md

@@ -0,0 +1,71 @@
+# Destiny Farm Finder
+
+Tracks Destiny 2 raid activity in near real time: which players are raiding right now, and who
+has completed what. A set of background crawlers observes the Bungie API and writes to SQLite;
+the web app only reads.
+
+## Language
+
+**Fireteam**:
+A group of players playing a Destiny activity together. The unit users care about — every card on
+the active-sessions page is one fireteam.
+_Avoid_: party, group, team, squad, lobby
+
+**Active Session**:
+A fireteam observed to be inside a raid right now. Ceases to be active when the crawler confirms
+the raid ended, or when the observation goes stale.
+_Avoid_: live session, current activity, in-progress raid
+
+**Roster**:
+The players making up a fireteam, as reported by Bungie. May be incomplete — Bungie does not
+always disclose every member — so a roster of one usually means limited visibility rather than a
+genuine solo run.
+_Avoid_: party members, participants, players in session
+
+**Tracked Player**:
+A player the system knows about and will poll for activity. Identified by `Name#Code`; a player
+becomes tracked by being discovered in a raid alongside someone already tracked.
+_Avoid_: user, account, member
+
+**Raid**:
+Destiny's six-player endgame activity, and the only activity type the leaderboards and the
+active-sessions list cover. Other activities are observed but never displayed.
+_Avoid_: activity (too broad), instance
+
+**Full Clear**:
+A raid played from the first encounter through the final boss, as opposed to joining at a
+checkpoint. Only Bungie's own report that the activity began at the start establishes this — no
+other signal is authoritative, and one that looks like it is has been wrong since Bungie stopped
+publishing it.
+_Avoid_: complete run, fresh run
+
+**Checkpoint Run**:
+A raid entered partway through, at a saved encounter. Observed and stored like any other run, but
+never counted toward a leaderboard. The majority of raids we see.
+_Avoid_: partial run, CP run
+
+**Completion**:
+One full clear finished by a particular player, counted once per raid instance however many
+characters they brought to it. The unit every leaderboard ranks by. A player being present for a
+cleared raid is not enough — they must have finished it themselves.
+_Avoid_: clear, kill, run
+
+**Heartbeat**:
+The crawler's periodic signal that it is still observing. Its age is the Data Freshness; once it
+lapses, the site reports itself as no longer live.
+_Avoid_: ping, health check, status
+
+**Data Freshness**:
+How long ago the crawler last confirmed it was working — the age of what the site knows. The only
+freshness that says anything about whether a leaderboard or an active session can be trusted.
+_Avoid_: last updated, uptime
+
+**Page Freshness**:
+How long ago a browser tab last fetched — the age of what is on screen. Says nothing about whether
+the data behind it is current: a tab can be seconds old and still be displaying hours-old data.
+_Avoid_: last updated, refresh time
+
+**Farm**:
+Repeatedly replaying a single raid encounter or checkpoint for rewards, rather than progressing
+through the raid. The activity the site is named for.
+_Avoid_: grind, rerun

docs/adr/0001-fireteam-denominated-display-cap.md

@@ -0,0 +1,24 @@
+# Active-session limits are denominated in fireteams, not rows
+
+`active_sessions` is keyed by `membership_id`, so a single fireteam produces up to six rows — one
+per tracked player in it. The read path originally capped those raw rows (`ORDER BY started_at
+DESC LIMIT 200`) and deduped into fireteams afterwards, which meant the limit was spent on
+duplicates and, because it sorted by start time, evicted the longest-running raids first. In
+practice ~1000 live rows rendered ~110 cards and nothing older than about five minutes was ever
+visible. We now scan a generous bound of raw rows, dedupe into fireteams, and only then apply the
+user-facing cap — which is counted in fireteams.
+
+## Consequences
+
+- Two separate limits exist and must not be collapsed into one: `ACTIVE_SESSION_ROW_SCAN_LIMIT`
+  (raw rows, default 3000) and `ACTIVE_SESSION_DISPLAY_LIMIT` (fireteams, default 600).
+  Re-introducing a single `LIMIT` in SQL restores the bug.
+- The row scan is ordered by `checked_at DESC`, not `started_at DESC`. It is served by
+  `idx_active_sessions_checked_at`, and if the bound is ever hit it sheds the *stalest* rows —
+  the ones closest to ageing out — instead of the longest-running raids.
+- Dedupe happens before name enrichment, so the display-name lookup covers only the fireteams
+  actually rendered rather than every row scanned.
+- The row bound has roughly 2x headroom over what the crawler can produce: at
+  `CRAWLER_SESSION_POLLING_LIMIT` rows per cycle across the 900s freshness window, at most ~1500
+  rows can be fresh simultaneously. Raising the polling limit materially should prompt a review
+  of this bound.

docs/adr/0002-session-count-reports-true-total.md

@@ -0,0 +1,26 @@
+# The active-session count reports the true total, not the number of cards shown
+
+`countActiveRaidSessions` feeds the nav StatsBar and the OG share cards, while
+`/api/active-sessions` feeds the page. These deliberately no longer agree: the count reports every
+live fireteam, whereas the list is capped at `ACTIVE_SESSION_DISPLAY_LIMIT`. The headline number
+answers "how busy is Destiny right now", which is the question a share card and a stats bar are
+actually asking; capping it to whatever happened to fit on screen would understate real activity
+and undersell the site.
+
+This is a deliberate exception to the invariant that `dedupe.ts` was written to protect — that the
+count and the list collapse duplicate rows identically. That invariant still holds: both go through
+`getDedupedActiveSessions`, so they can never disagree about *what a fireteam is*. Only the cap
+differs.
+
+## Consequences
+
+- `/api/active-sessions` returns `total` (all live fireteams) alongside `shown` (those under the
+  cap), so the page can disclose the difference rather than hiding sessions silently.
+- A discrepancy between the StatsBar number and the visible card count is expected when the cap
+  bites, and is not a bug to be "fixed" by capping the count.
+- The server logs a warning whenever the cap bites, since the default (600) sits close to observed
+  prod volume and the gap would otherwise be invisible.
+- The StatsBar counts link to `/leaderboard` and `/active-sessions` without carrying a time range or
+  clearing raid filters. The destination honours whatever view the user saved, so the same expected
+  discrepancy extends to navigation: clicking "full clears · last 24h" can land a reader on their
+  own saved 7-day board, and that is not a broken link.

docs/adr/0003-tests-run-against-a-real-sqlite-file.md

@@ -0,0 +1,69 @@
+# Tests run against a real SQLite file, not `:memory:`
+
+The test suite gives each test file its own throwaway database in a `mkdtemp`
+directory, pointed at by `RAID_TRACKER_DB_PATH`, rather than using SQLite's
+`:memory:` database. `:memory:` looks like the obvious choice — faster, no
+cleanup — so the reason for not using it needs recording.
+
+## Why
+
+SQLite cannot put an in-memory database into WAL mode. `PRAGMA journal_mode = WAL`
+returns `memory` for `:memory:` and `wal` for a file, silently:
+
+```
+:memory:  journal_mode = WAL  ->  'memory'
+file      journal_mode = WAL  ->  'wal'
+```
+
+Production runs WAL. The entire justification for testing against a real database
+rather than a mock is that it validates the real SQL under real semantics, so
+running the suite under a different journal mode gives up most of what the
+approach was bought for.
+
+A temp *directory* rather than just a temp file, because `DATA_DIR` in
+`src/lib/maintenance/state.ts` derives from `dirname(RAID_TRACKER_DB_PATH)`.
+Relocating the database therefore relocates `maintenance-state.json` for free.
+That is not incidental: `getDb()` calls `isDbQuiesceActive()` on *every*
+invocation, which reads that file from disk — so a suite pointed at the real data
+directory would throw `DatabaseMaintenanceError` from every test if it happened
+to run while a maintenance vacuum was in progress.
+
+The path is set in a Vitest `setupFile` rather than inside a helper, because
+`DB_PATH` is a module-level constant resolved at import time. Setting it before
+the test file's own imports run is what lets test files use ordinary static
+imports instead of `await import()` throughout.
+
+That ordering is load-bearing, and breaking it fails *silently*: `DB_PATH` resolves
+to the real data directory, and the suite's `DELETE FROM` and `VACUUM` paths operate
+on live data without erroring. So `getDb()` refuses to open anything but the
+throwaway path while `VITEST` is set — `tests/setup/test-db-path.ts` publishes the
+directory it minted as `DFF_TEST_DB_SENTINEL`, and `assertDbPathAllowed()` in
+`src/lib/db/index.ts` requires an exact match. Keyed on `VITEST` rather than on the
+sentinel alone so that a suite where the setup file never ran at all — the case where
+every other protection has already failed — still refuses rather than going quiet.
+
+The tempting alternative was to remove the hazard instead of detecting it: have the
+setup file `await import('@/lib/db')` immediately after setting the env var, pinning
+`DB_PATH` before anything else can. Rejected because its correctness depends on the
+import being *dynamic* — a static `import` hoists above the assignment and does
+nothing — so a routine tidy-up reverts it, silently, which is the property that made
+the original hazard dangerous in the first place.
+
+## Consequences
+
+- Test databases cost a `mkdtemp` plus `initializeSchema()` per test file —
+  roughly 5–15 ms on tmpfs. At this suite's size that is a few milliseconds
+  overall, well below the value of matching production semantics.
+- Temp directories leak into the system temp dir if a test process is killed
+  before `afterAll` runs. Harmless, and the OS clears them.
+- A misconfigured suite fails on the first `getDb()` call with `Refusing to open …`
+  rather than quietly using the real database. `tests/setup/test-db-path.test.ts`
+  pins the invariant, including that `VITEST` is actually set — an inert guard is
+  the only failure mode here that hides, since one that wrongly refuses breaks
+  every test file at once.
+- `openMaintenanceDb()` is deliberately *not* guarded: nothing in the suite reaches
+  it today. Its callers (`src/lib/bungie/maintenance.ts`) `VACUUM` through it, so a
+  test that exercises them should add the check.
+- The schema under test is the production schema by construction: `getDb()` runs
+  `initializeSchema()`, including the `ended_at` migration guard and the Phase 3
+  indexes. There is no second schema definition that can drift.