Repository navigation
WP-04: AI provider registry + prompts - #6
Merged
Merged
Conversation
lib/ai: one generic OpenAI-compatible adapter + §8.1 preset table (glm-coding default, glm-open, groq fallback, openai, openrouter, anthropic native Messages, custom), selection order app_settings.ai → env → defaults, GLM multi-endpoint retry + working-endpoint cache ported from engine/glm.js, §8.1 call policy (60s timeout, 1 retry on 429/5xx with 2s backoff → fallback provider → deterministic builder labelled fallback), JSON mode + Zod validation + one repair call, §7.3 quote-by-reference resolution before validation, §8.4 post-check wiring (neverSay scan → regenerate with feedback → phrase replace, religion gate, foreign-URL strip), §7.1 usage_events logging with verified per-model prices (0 when unknown). lib/prompts: shared §8.2 system prompt (verbatim skeleton), 12 per-artifact prompts with PROMPT_VERSION, 60k-token transcript budget (first 40% + last 20% + top-ranked middle), weak-model hardening (temp 0.4 long / 0.2 analyze, flat schemas, inline JSON examples). Voice (§7.4): profile schema + defaults, learn + preview functions. Admin AI API (§7.5): GET/PUT /api/admin/ai + POST /api/admin/ai/test, owner-gated via requireOwner (OWNER_EMAILS + Clerk currentUser, deny- by-default 503 when unconfigured); app_settings store is an injected interface with a documented WP-02 seam. U8 U10 U18 covered by unit tests; live smoke test included but CI-skipped (tests/unit/live.ai.test.ts). E2E_PORT makes the Playwright port configurable for parallel worktrees. Decisions logged in docs/relaunch/DECISIONS.md.
# Conflicts: # docs/relaunch/DECISIONS.md # web/package.json # web/playwright.config.ts # web/pnpm-lock.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope (WP-04 only)
web/lib/ai/+web/lib/prompts/+ admin AI API routes per spec §8.1, §8.2, §7.4, §7.5. No admin UI (WP-09), no orchestration routes (WP-06). Branched frommain@4b19927(WP-03); origin/main (WP-01 + WP-02) merged into this branch and the two documented WP-02 seams are wired for real — details under "WP-02 integration" below.Deliverables
web/lib/ai/): one generic OpenAI-compatible adapter + config presets exactly per §8.1 (glm-codingdefault,glm-open,groqfallback,openai,openrouter,anthropicnative Messages,custom); selection orderapp_settings.ai→ env → defaults; GLM multi-endpoint retry + working-endpoint cache ported fromengine/glm.js; §8.1 call policy — primary (60 s timeout, 1 retry on 429/5xx with 2 s backoff) → fallback provider → deterministic builder (lib/engine) with statusfallback; JSON viaresponse_format json_objectwhere supported + Zod (§7.2 schemas) + one repair call; §7.3 quote-by-reference resolution (claims capped per kind, verbatim offsets, candidate backfill) BEFORE validation; §7.1usage_eventslogging with est. cost fromlib/ai/prices.ts(vendor-verified prices, 0 when unknown).web/lib/prompts/): all 12 files per §8.2 +PROMPT_VERSION; shared system prompt verbatim skeleton; per-artifact rules with §7.2 counts; ≤2 voice samples ≤400 words ("style reference only"); 60 k-token transcript budget (first 40% + last 20% + top-ranked middle, recordstruncated); weak-model hardening (0.4 long / 0.2 analyze, flat schemas, inline JSON examples).learnVoiceProfile(fast model, one repair, default profile on failure) andpreviewVoice.GET/PUT /api/admin/ai,POST /api/admin/ai/test— thin route wrappers over framework-free handlers; owner-only viarequireOwner(ClerkcurrentUser+OWNER_EMAILS).WP-02 integration (merged mid-flight, seams completed)
resolveAppSettingsStore()now returns a real Drizzle store over WP-02'sapp_settingstable (getby key,putupsert);nullonly whenDATABASE_URLis unset (keyless preview) → GET serves env/default view, PUT 503settings_store_unavailable; unreachable DB → PUT 503settings_store_write_failed.requireOwnerclassifies with WP-02'slib/env.isClerkConfigured(): keyless previews are always denied (503) — app pages may render with a preview user there, but AI settings switching must never work under fake auth. U18 tests inject the gate + an in-memory store at the same documented interfaces.Acceptance evidence
U8 — no banned word survives (
tests/unit/ai.generate.test.ts,tests/unit/ai.postcheck-wiring.test.ts)regenerated: true)."We delve into attention some more." → "We into attention some more."), result schema-valid,neverSayclean.U10 — all providers fail → deterministic package labelled fallback (
ai.generate.test.ts"U10" block + live run)[glm-coding, groq]→status: "fallback",provider/model: null,errorCode: "all_providers_failed", bullets are transcript sentences, CTA from show settings; quotes artifact falls back to verbatim candidate refs; chapters without timestamps return{items: []}(no_timestamps), never fabricated times.fallbackin 567 ms.U18 — owner switches provider/model; test button OK/error (
tests/unit/ai.admin.test.ts)PUTpersists a valid selection through the store interface (in-memory store in tests; real Drizzle store wired for runtime); invalid bodies → 400; custom without base URL → 400; unwired store → 503; DB write failure → 503.generateArtifactwith the same store hits the saved provider/model (groq + saved model asserted on the request body).POST testreturns{ok: true, latencyMs, usage}for a working chain and{ok: false, error}on failure; unknown provider → 400; gate 401/403/503 covered.Checks
pnpm lintgreen ·pnpm typecheck0 errors ·pnpm buildgreen (merged tree, incl. WP-02 routes/middleware) ·pnpm test: 248 passed / 2 skipped (live) across 24 files (WP-04 added 92 tests to WP-03's 93; WP-02's suite rides along green) ·pnpm e2eon the merged tree: 18 passed (all 4 spec files, local production build).503 {"error":"owner_not_configured"}).Live provider test — attempted, blocked by key state (no fake results)
Ran
tests/unit/live.ai.test.tslocally against the real endpoints with.env.relaunchsourced:glm-codingflavors — observed401 {"error":{"code":"401","message":"token expired or incorrect"}}on all four (z.ai coding ×2 models, z.ai paas, open.bigmodel.cn). TheGLM_API_KEYin.env.relaunchis 11 characters — not a valid key shape. Blocker (owner): provide a valid GLM key (or activate the coding-plan resource per §8.1 — error 1113 was never reached because auth fails first). The multi-endpoint fallback order behaved exactly as ported.ASR_PROVIDERin.env.relaunchis notgroqandASR_API_KEYreturns 401 fromapi.groq.com— no Groq LLM key available. Blocker (owner): provideGROQ_API_KEYfor the fallback live test.skipIf(!GLM_API_KEY || CI)) and will produce model id/latency/usage numbers with valid keys; rerun command is in its header. No fabricated numbers are recorded here.Decisions logged (docs/relaunch/DECISIONS.md)
Groq fallback models chosen from Groq's current model list:
openai/gpt-oss-120b(long) /openai/gpt-oss-20b(fast) — llama-3.3/3.1 chat models are now Enterprise-only per console.groq.com/docs/models (2026-10-27); gpt-oss models are self-serve with verified prices ($0.15/$0.60 and $0.075/$0.30 per M) and 131k context. Also: vendor-verified price table only (GLM rows absent → 0);requireOwnerdeny-by-default, keyless previews never own AI settings; wholesale selection precedence; fallback content not padded to AI minima; quote resolution before validation with candidate backfill; subpath-aware URL allow list;units= total tokens for llm events; live test as CI-skipped vitest; OpenAI/OpenRouter placeholder defaults; WP-02 seams completed via merge.Blockers / notes for the integrator
GLM_API_KEY(current one is an 11-char placeholder) andGROQ_API_KEYneeded before live generation and the WP-10 eval gate.generateArtifactper artifact kind; persiststatus/provider/model/error_codeon the artifact row;truncatedis onGenerateResult(no §7.1 column — decide storage or ignore)./adminUI consumesGET/PUT /api/admin/ai+POST /api/admin/ai/testas-is; with Clerk +OWNER_EMAILSconfigured the gate opens with no further code.@types/nodepeer warnings from vitest (present since WP-00, untouched).