U5: Semantic cache + pgvector + indexes

[jaturapornchai]

Summary

• Add pgvector extension + semantic_cache table (ivfflat cosine index) with graceful fallback when the extension is unavailable.
• Add hot-path performance indexes: idx_health_cooldown_active (partial), idx_gateway_recent, idx_exam_passed_recent (partial).
• New src/lib/semantic-cache.ts library: embed via local Ollama nomic-embed-text, cosine-similarity lookup, idempotent store, stats helper. All operations no-op silently on failure.

Library is not yet wired into v1/chat/completions — follow-up unit.

Test plan

• rtk npx next build — 0 errors, 0 warnings
• Deploy and verify [migrate] pgvector + semantic_cache ready log line (or graceful warning if ext missing)
• Confirm new indexes exist via \d health_logs / \d gateway_logs / \d exam_attempts

[Copilot]

Pull request overview

Adds an optional Postgres-backed semantic cache using pgvector (Ollama embeddings + cosine similarity) and introduces several new “hot path” DB indexes to improve query performance, while aiming to fail gracefully when pgvector/Ollama are unavailable.

Changes:

• Add src/lib/semantic-cache.ts with embedding, similarity lookup, idempotent storage, and cache stats helpers.
• Extend DB migrations to provision pgvector + semantic_cache (ivfflat cosine index) with fallback logging.
• Add new performance indexes on health_logs, gateway_logs, and exam_attempts.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
src/lib/semantic-cache.ts	New semantic cache library: embeds via Ollama and queries/stores in semantic_cache.
src/lib/db/migrate.ts	Migration adds pgvector + semantic_cache table/indexes and additional performance indexes.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

embedQuery accepts any embedding length, but the DB column is vector(768). If Ollama returns a different dimension (model change, config, API change), every store/lookup will fail and silently no-op. Add an explicit embedding.length === 768 (and optionally finite-number) validation before returning/using the embedding so failures are fast and diagnosable.

[Copilot]

The hit counter update keys on query, but query is not guaranteed unique (you store query.slice(0, 2000) while query_hash is computed from the full query). This UPDATE can increment multiple rows if different long queries share the same first 2000 chars. Update by a unique identifier instead (e.g., select id or query_hash and use that in the WHERE clause).

[Copilot]

This partial index predicate uses now(), which is STABLE (not IMMUTABLE). Postgres will reject it with "functions in index predicate must be marked IMMUTABLE", so the index will never be created (and you'll log a warning every startup). Use an immutable predicate (e.g., cooldown_until IS NOT NULL) or replace with a normal index and keep the time comparison in the query.

Suggested change

	WHERE cooldown_until > now()
	WHERE cooldown_until IS NOT NULL

[Copilot]

These indexes are created with plain CREATE INDEX, which can take an ACCESS EXCLUSIVE lock and block writes on large tables during startup migrations. Since runMigrations() isn't running inside an explicit transaction block, consider CREATE INDEX CONCURRENTLY (or moving index creation to an offline/one-time migration step) to reduce production impact.

Suggested change

	CREATE INDEX IF NOT EXISTS idx_semantic_cache_embedding
	ON semantic_cache USING ivfflat (embedding vector_cosine_ops)
	WITH (lists = 100)
	`;
	await sql`CREATE INDEX IF NOT EXISTS idx_semantic_cache_used ON semantic_cache(last_used_at DESC)`;
	console.log("[migrate] pgvector + semantic_cache ready");
	} catch (err) {
	console.warn("[migrate] pgvector not available — semantic cache disabled:", String(err).slice(0, 150));
	}
	// ─── Performance indexes ─────────────────────────────────────────────────────
	// Note: CREATE INDEX CONCURRENTLY cannot run inside transaction,
	// so we use normal CREATE INDEX IF NOT EXISTS here
	await sql`
	CREATE INDEX IF NOT EXISTS idx_health_cooldown_active
	ON health_logs(model_id, cooldown_until)
	WHERE cooldown_until > now()
	`.catch((err) => console.warn("[migrate] idx_health_cooldown_active:", String(err).slice(0, 100)));
	await sql`
	CREATE INDEX IF NOT EXISTS idx_gateway_recent
	ON gateway_logs(created_at DESC, status)
	`.catch((err) => console.warn("[migrate] idx_gateway_recent:", String(err).slice(0, 100)));
	await sql`
	CREATE INDEX IF NOT EXISTS idx_exam_passed_recent
	CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_semantic_cache_embedding
	ON semantic_cache USING ivfflat (embedding vector_cosine_ops)
	WITH (lists = 100)
	`;
	await sql`CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_semantic_cache_used ON semantic_cache(last_used_at DESC)`;
	console.log("[migrate] pgvector + semantic_cache ready");
	} catch (err) {
	console.warn("[migrate] pgvector not available — semantic cache disabled:", String(err).slice(0, 150));
	}
	// ─── Performance indexes ─────────────────────────────────────────────────────
	// runMigrations() is not inside an explicit transaction, so prefer
	// CREATE INDEX CONCURRENTLY IF NOT EXISTS to reduce write blocking during startup
	await sql`
	CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_health_cooldown_active
	ON health_logs(model_id, cooldown_until)
	WHERE cooldown_until > now()
	`.catch((err) => console.warn("[migrate] idx_health_cooldown_active:", String(err).slice(0, 100)));
	await sql`
	CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_gateway_recent
	ON gateway_logs(created_at DESC, status)
	`.catch((err) => console.warn("[migrate] idx_gateway_recent:", String(err).slice(0, 100)));
	await sql`
	CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_exam_passed_recent