Migrate dashboard runtime from Supabase to SQLite

[zepdevelopers]

Summary

This PR moves Zeude dashboard runtime operational storage from Supabase to SQLite while keeping ClickHouse for analytics.

It also adds:

• a one-time Supabase -> SQLite migration path
• single-host server installation via install-server.sh
• updated release packaging so the dashboard can keep serving client install assets and server install assets

What Changed

• Introduced an operational DB boundary under dashboard/src/lib/db/*
• Added SQLite connection/bootstrap/migration support with better-sqlite3
• Moved runtime operational routes off direct Supabase access
  • auth/session/OTT/logout
  • config sync
  • invite claim
  • user skill preferences
  • status/install reporting
  • admin CRUD routes
  • prompt and skill-suggestion ingestion validation paths
  • skill rules
  • leaderboard operational lookups
  • name resolution
• Added npm run migrate:supabase-to-sqlite
• Added scripts/install-server.sh
• Updated Docker / compose defaults for single-host SQLite deployment
• Kept releases/install.sh flow intact and fixed dashboard URL summary output in install logs

Architecture

flowchart LR
    subgraph ClientMachine[Developer Machine]
        CLI[claude / codex shim]
        LocalCfg[~/.zeude + local CLI config]
        RealCLI[real claude / codex binary]
        CLI -->|sync on startup| Dashboard
        CLI --> RealCLI
        CLI --> LocalCfg
    end

    subgraph ServerHost[Single Host Deployment]
        Dashboard[Next.js Dashboard]
        SQLite[(SQLite\noperational state)]
        CH[(ClickHouse\nanalytics)]
        Releases[/releases/* assets/]
        Dashboard <-->|operational reads/writes| SQLite
        Dashboard <-->|analytics queries + writes| CH
        Dashboard --> Releases
    end

    Supabase[(Legacy Supabase)] -->|one-time migration| SQLite

Loading

Runtime Model After This PR

• SQLite becomes the source of truth for operational data:
  • users
  • sessions
  • one-time tokens
  • invites
  • skills
  • hooks
  • MCP servers
  • agents
  • cohort members
  • install status
• ClickHouse remains the analytics store
• Supabase is retained only as a migration source / compatibility boundary

Validation

Completed locally:

• cd zeude/dashboard && npm test
• cd zeude/dashboard && npx tsc --noEmit
• cd zeude/dashboard && npm run migrate:sqlite
• cd zeude/dashboard && npm run dev
• curl -s http://127.0.0.1:3000/api/health
• bash zeude/scripts/build-release.sh
• bash -n zeude/scripts/install.sh
• bash -n zeude/scripts/install-server.sh
• DRY_RUN=1 bash zeude/scripts/install-server.sh
• temp-home install test for releases/install.sh

Observed limitation in this environment:

• non-dry-run install-server.sh could not complete here because the Docker daemon socket was unavailable to the session

Follow-up Manual Checks

Recommended before merge:

• run install-server.sh end-to-end on a host with Docker daemon access
• run npm run migrate:supabase-to-sqlite -- --dry-run against a real Supabase dataset
• run the real migration once and compare row counts between source and SQLite target
• do one smoke deploy from the produced release assets

Commits

• 368f4f9 Migrate dashboard runtime from Supabase to SQLite
• 6e1844f Add single-host server installer
• 52e22f4 Fix install script dashboard summary output

[Q00]

Thanks for the SQLite migration work. I found a few merge blockers before this is safe to land.

1. High: the new single-host install path builds the wrong image.
   scripts/install-server.sh and the generated systemd unit both start dashboard/docker-compose.yaml, which builds dashboard/Dockerfile. That image never copies public/releases; only the root Dockerfile does. The PR description says the dashboard should continue serving both client install assets and server install assets, but a fresh single-host install will ship an image without /releases/*.

2. High: the SQLite bind mount is likely unwritable on a fresh install.
   The container runs as nextjs (uid 1001), while docker-compose bind-mounts ${ZEUDE_DATA_DIR} onto /var/lib/zeude. The installer creates the host data directory, but it never chowns that host path to uid 1001. On first boot, SQLite can fail with permission denied when it tries to create zeude.db.

3. High: the documented fresh-host install command fails for non-root users before Docker starts.
   The README tells users to run bash scripts/install-server.sh, but the script defaults to /opt/zeude and /var/lib/zeude and creates those paths with plain mkdir -p before any sudo path is used. On a normal Linux host, that exits with permission errors before the stack ever comes up.

4. Medium: DATABASE_PROVIDER is still exposed, but runtime selection ignores it.
   The env schema still accepts DATABASE_PROVIDER, but getOperationalDb() only switches on NODE_ENV === 'test'. If the rollout assumes any staged fallback or compatibility path through Supabase, the app will silently boot against a fresh SQLite file instead of either honoring the setting or failing fast.

Validation from my side:

• npm test passed
• npx tsc --noEmit passed
• CLICKHOUSE_PASSWORD=dev next build passed
• npm run lint still fails, including newly touched files such as scripts/migrate-from-supabase.cjs, scripts/migrate-sqlite.cjs, src/lib/db/supabase-adapter.ts, and src/app/api/config/[agentKey]/route.ts

Requesting changes.

[Q00]

Follow-up review after the latest update: the install-path issues look addressed, but I still see two merge blockers.

1. High: DATABASE_PROVIDER=supabase now gets selected, but the Supabase operational adapter is still a stub.
   src/lib/db/index.ts now honors DATABASE_PROVIDER, which is good. But src/lib/db/supabase-adapter.ts still has critical methods that are effectively non-functional: users.findByAgentKey() always returns null, session/token lookups are stubbed out, token creation is a no-op, and team-filtered lookups like skills.listActiveForTeam() still return empty arrays. That means a deployment configured with DATABASE_PROVIDER=supabase will break auth/config/session behavior immediately. If Supabase is meant to remain a supported runtime mode, this adapter needs to be completed; otherwise the provider should be treated as migration-only and fail fast instead of pretending to be supported.

2. High: the invite-claim race is still present.
   src/lib/db/sqlite/repositories.ts still does SELECT ... WHERE used_at IS NULL followed by a separate UPDATE inside claimInviteTx(), and src/app/api/invite/[token]/route.ts still relies on that helper for exclusivity. Under concurrent requests, two acceptances can still observe the invite as unused before either update lands. This needs to be a single conditional UPDATE ... WHERE token = ? AND used_at IS NULL AND expires_at > ?, then check changes to know whether the claim actually succeeded.

Re-check from my side:

• npm test passed (10 files, 183 tests)
• npx tsc --noEmit passed
• the install-path fixes look good in the latest diff
• npm run lint still fails, including in newly touched files such as src/lib/db/supabase-adapter.ts and src/app/api/config/[agentKey]/route.ts

[Q00]

PR Review: Supabase → SQLite Migration

63 files, +3264/-1375 — Large migration with clear direction and solid implementation overall. The repository layer abstraction is clean and the migration tooling is well thought out.

Posting review based on the self-hosted, single-host threat model (operator = admin, small data scale, no external attacker surface for operational DB).

---

Verdict: Ready to merge with follow-ups noted below.

---

HIGH — Address in next PR

1. Duplicated schema DDL across 3 files

migrator.ts, migrate-sqlite.cjs, and migrate-from-supabase.cjs contain identical CREATE TABLE statements copy-pasted in three places. The next schema change (adding a column, changing a constraint) will almost certainly break one of them silently.

Suggestion: extract DDL into a single shared source (e.g. a .sql file or a shared constant) and reference it from all three locations.

2. Missing indexes on FK columns used in WHERE/JOIN

These columns are queried on every request but lack indexes:

• zeude_sessions.user_id
• zeude_one_time_tokens.user_id
• zeude_mcp_install_status.user_id
• zeude_hook_install_status.user_id
• zeude_cohort_members.user_id, zeude_cohort_members.cohort_key

Session validation (every authenticated request) and install-status upserts (every client sync) will degrade noticeably once user count exceeds ~1000.

---

MEDIUM — Address in subsequent PR

3. No cleanup for expired sessions/tokens

zeude_sessions and zeude_one_time_tokens only check expiry on read — rows are never deleted. After 6+ months of operation this table will grow unbounded. A periodic cleanup job or TTL-based delete would help.

4. N+1 query regression in name-resolution.ts

The old Supabase code used .in('email', ...) for batch lookups. The new SQLite code loops over emails individually with db.users.findByEmail(email). For leaderboard and analytics pages resolving many users, this creates N queries per N unresolved emails.

Suggestion: add a findByEmails(emails: string[]) batch method.

5. No runtime validation for SESSION_SECRET in production

install-server.sh auto-generates a strong secret, so the happy path is fine. However, if someone deploys manually and omits SESSION_SECRET, it defaults to '' in production — sessions would be unsigned and forgeable. A startup warning (if (!secret) console.error(...)) would catch this.

---

LOW / INFO

Item	Assessment
buildUpdateSql column-name interpolation	All callers use hardcoded table/column names — no user-input path exists. Low risk in self-hosted context.
No chmod 600 on env file in install-server.sh	Recommended but low risk on single-user host
No FK constraints in DDL	PRAGMA foreign_keys = ON is set but no actual FOREIGN KEY clauses. Acceptable for self-hosted single-host.
Duplicate _sqlite_migrations table creation	Harmless — both MIGRATIONS[0] and migrateSqlite() body create it
Alpine → bookworm-slim Dockerfile change	Required for better-sqlite3 native build. Acceptable tradeoff.
docker-compose.yaml context change (. → ..)	Just needs a .dockerignore check
No .close() on singleton SQLite connection	WAL mode cleanup on graceful shutdown. Typically fine in Next.js standalone.
DATABASE_PROVIDER env var unused	Set in Dockerfile but never read in code. Harmless dead config.

---

Summary

Severity	Count	Key items
CRITICAL	0	—
HIGH	2	Schema DDL duplication, missing FK indexes
MEDIUM	3	Expired data cleanup, N+1 query, SESSION_SECRET validation
LOW/INFO	8	Various minor items

Ship it. I'd prioritize the two HIGH items as a follow-up PR before the next schema change lands.