LLM-backed memory + cue reasoning (server-side, optional)

[wdwd720]

Stacked on PR #3 (cursor/memory-api-dataset-spine-c3fa) so this PR's diff is just the LLM layer.

What this adds

A real OpenAI-compatible LLM brain on the server, with strict structured output, a deterministic safety guard, and a mock provider for tests. The whole app still runs without a key — the LLM is purely additive.

.env (server-only)  →  OpenAiCompatibleProvider  →  /chat/completions (json_schema)
                              ↓
       LlmMemoryExtractor    LlmCueReasoner
                              ↓
                          safetyGuard      → Hearer Cue → HUD
                              ↓
                         deterministic fallback (always available)

What the LLM does (and doesn't)

• Memory extraction (POST /api/memory/extract) turns a sentence like "Usually after dinner I cook, so if you hear beeping remind me to check the stove." into a structured routine (trigger description, action label, priority). Falls back to the rule-based extractor on disabled / error / schema failure.
• Cue reasoning (POST /api/decide) takes a stable detection + memory + live context and proposes the HUD cue. The safety guard then rejects anything that claims certainty about hazards, mentions direction without evidence, exceeds 2 lines, repeats a transcript, etc.
• Never on every frame. Per-label cooldowns and a confidence threshold keep this cheap.
• Audio is never sent. Only short text and structured detection metadata.

Server layer (server/src/llm/)

• llmConfig.ts loads .env once. Read-only LlmConfig with enabled / apiKey / baseUrl / model / timeout / maxInput / storeRawTranscripts. maskApiKey() ensures the full key never appears in logs.
• llmTypes.ts, llmErrors.ts, llmSchemas.ts, llmPrompts.ts are the contract surface.
• openAiCompatibleProvider.ts issues a single fetch() to /chat/completions with response_format: { type: "json_schema", strict: true }, then re-validates the parsed JSON server-side.
• mockLlmProvider.ts lets tests exercise the same code path without a network. It reuses the rule-based extractor and applies the cue reasoning rules deterministically.
• llmProvider.ts factory: real provider when HEARER_LLM_ENABLED=true and a key is present; otherwise a DisabledLlmProvider whose status reports the reason honestly.

Server services

• services/llmMemoryExtractor.ts — wraps the provider, validates the response, persists items the model marked save=true (routine / commitment / important_item / known_person / location_rule). Returns mode/extracted/persisted/summary/confidence/fallbackUsed/reason.
• services/llmCueReasoner.ts — builds CueReasoningInput from memory summary + live context + recent feedback, asks the provider for a cue, runs the safety guard, returns either a Hearer Cue or a structured rejection.
• services/safetyGuard.ts — validateAndRejectUnsafeCue + sanitizeTranscriptForStorage + shouldStoreRawTranscript. Hard-deny patterns: stove is on, emergency, danger, you forgot, diagnoses, location-bias. Direction-only-when-known check. Max 2 lines / 80 chars. Transcript-leakage guard.

Endpoints

• GET /api/llm/status — { enabled, configured, provider, model, reason }.
• POST /api/memory/extract — returns mode: "llm" | "rule_based", fallbackUsed, persisted records, summary.
• POST /api/decide — stable detection + context override → cue. Persists the AudioDetection and CueRecord. Returns mode: "llm" | "llm_rejected_fallback" | "rule_based".
• buildApp({ storage, llmProvider }) factory in server/src/app.ts. Tests inject a mock provider; production server.ts is now a tiny entry-point.

Frontend wiring

• src/api/hearerApi.ts adds llmStatus() and decide() (15 s timeout).
• ApiMemoryClient exposes llmStatus(). MemoryClient.extract() surfaces the server's mode. LocalMemoryClient returns mode: "rule_based".
• AudioController.setServerBrain() is an optional hook that REPLACES the local decision when /api/decide returns a cue. The local decision is still emitted first so the dev event log stays responsive; the server result re-emits when it lands.
• App.tsx probes /api/llm/status on mount, installs the server-brain hook when the Memory API is connected, and exposes a Server brain toggle.
• Header chip reads LLM: connected · gpt-4.1-mini / LLM: mock / LLM: key missing / LLM: disabled fallback — the user always sees what's running.
• Teach Hearer shows "Saved 1 item via LLM extractor." / "…via rule-based extractor." so the demo is honest about which path ran.

Tests

npm test → 65 / 65 passing across 18 suites (was 46 / 13). New ones:

• server/src/__tests__/llmStatus.test.ts — disabled status with no key; /api/memory/extract uses rule_based fallback.
• server/src/__tests__/llmExtract.test.ts — mock LLM extracts the cooking routine (killer demo) and the Jason commitment.
• server/src/__tests__/decide.test.ts — deterministic fallback when LLM disabled; LLM cue "Kitchen timer beeping. / Check stove." when enabled with cooking routine; safety guard rejects "Stove is on."; rejects a direction cue when no direction supplied.
• server/src/__tests__/safetyGuard.test.ts — accepts safe cue; rejects stove is on, emergency, diagnoses, missing-direction cues, too long, too many lines.
• server/src/__tests__/llmSchemas.test.ts — accepts well-formed payloads; rejects unknown kind / priority.
• All 46 prior tests still pass.

npm run build succeeds (~252 kB JS gzipped to ~76 kB).

Privacy / safety

• API key only on the server.
• Audio is never sent to the LLM. Only short text + structured detection labels.
• Default HEARER_STORE_RAW_TRANSCRIPTS=false. Stored text is sanitised for phone / email / credit numbers.
• 12 s timeout and 4000-char input cap, both configurable.
• Per-label cooldown on /api/decide so the LLM is not called per frame.
• Hearer never claims sensor-grade certainty. "Check stove." never "Stove is on.". Direction only mentioned when supplied. No medical / emergency / location-bias language.

How to enable

cp .env.example .env
# edit:
OPENAI_API_KEY=sk-...
HEARER_LLM_ENABLED=true
OPENAI_MODEL=gpt-4.1-mini

npm run server  # http://localhost:8788
npm run dev     # http://localhost:5173

curl http://localhost:8788/api/llm/status should now show provider: "openai_compatible". Save the cooking routine in Teach Hearer, then click Beep ×5 in the audio fallback panel — with Server brain on, the HUD shows "Kitchen timer beeping. / Check stove." (URGENT · PHYSICAL).