diff --git a/ROADMAP.md b/ROADMAP.md index bd1f8da..23ea77f 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -21,7 +21,7 @@ hosted voice stays. |---|---|---|---|---|---| | 0 | Foundations | Land the work already written, and stop losing the listener's first line | A clean `main` and an input path that never drops a typed line | **P0** | §0 below (the reconciliation landed; the dropped line has not) | | 1 | Sound like a DJ | Talk and music actually interleave, instead of alternating at boundaries | A track gets a lead-in, not a label; the host can interject mid-track, not only at its edges | **P1** | the lead-in and the coda landed (#199, #200) and both already speak over the ducked track; what is missing is the autonomous mid-track beat, [#163](https://github.com/wine-fall/murmur/issues/163) | -| 2 | Say real things | The host gets real material — news, new releases, what is happening near the listener | An off-loop topic pool, weighted by the listener's language and timezone | **P1** | built, green and unmerged: [#203](https://github.com/wine-fall/murmur/pull/203) + [#201](https://github.com/wine-fall/murmur/pull/201) (absorbs [#44](https://github.com/wine-fall/murmur/issues/44)) | +| 2 | Say real things | The host gets real material — news, new releases, what is happening near the listener | An off-loop topic pool, weighted by the listener's language and timezone | **P1** | [spec 13](specs/spec13/13-real-world-topics.md), landed via [#201](https://github.com/wine-fall/murmur/pull/201) + [#203](https://github.com/wine-fall/murmur/pull/203); by-ear [#202](https://github.com/wine-fall/murmur/issues/202), absorbs [#44](https://github.com/wine-fall/murmur/issues/44) | | 5 | Log in to the catalogue you already have | The catalogues murmur cannot reach are the ones behind a login | An opt-in, guided login for one auth-gated source — NetEase first | **P1** | new; §5 below | | 3 | Pick well, play reliably | Candidates come from sources worth trusting, not from keyword soup | Dead stream probes down; picks back under the spec-04 budget | **P2** | [#164](https://github.com/wine-fall/murmur/issues/164), [#149](https://github.com/wine-fall/murmur/issues/149) + new | | 4 | Others can run it, and it does not rot | A second brain backend, and an eval track under the stochastic behavior | murmur runs without a Claude Code login; prompt regressions get caught by a test | **P3** (its eval half, [#98](https://github.com/wine-fall/murmur/issues/98), is P2 once lines 1/2 land) | [#89](https://github.com/wine-fall/murmur/issues/89), [#98](https://github.com/wine-fall/murmur/issues/98), [#80](https://github.com/wine-fall/murmur/issues/80), [#153](https://github.com/wine-fall/murmur/issues/153), [#102](https://github.com/wine-fall/murmur/issues/102) | diff --git a/specs/STATUS.md b/specs/STATUS.md index aadbac6..b92bf48 100644 --- a/specs/STATUS.md +++ b/specs/STATUS.md @@ -10,7 +10,7 @@ _This file is a **card, not a ledger**: an entry that is done and no longer guides the work gets **deleted**, not archived. History lives in git and PR bodies; measured facts live in the spec they verify._ -_Last updated: 2026-09-03 (memory v1.5 built — spec 05-01 recall & forgetting)_ +_Last updated: 2026-09-03 (real-world topics built — spec 13)_ ## Where we are @@ -18,7 +18,7 @@ _Last updated: 2026-09-03 (memory v1.5 built — spec 05-01 recall & forgetting) built.** L0 = `01-core-loop` + `02-voice-provider` (hosted voice); L1 adds `03-01-brain-harness` + `03-02-ducking` + `03-03` guided install + the `03-04` bed + spec 05 memory (now at v1.5 — `05-01` recall & forgetting), with 04, 06, -07, 10, 11 and 12 built on top. Unit gate +07, 10, 11, 12 and 13 (real-world topics) built on top. Unit gate green (vitest); real-SDK smokes passed per phase. **Each spec's own status header records what its build realized and the PR that landed it** — read the spec for what it does, its PR for how it got there. Everything left is under @@ -32,7 +32,7 @@ it closes. Add and remove entries with the `murmur-issue` skill, never by hand: CI fails if this section points at an issue that is already closed. - **#89** (eng) Second brain backend: Codex SDK — recorded direction, not scheduled. -- **#44** (eng) Cold-start talk repeats the same cozy imagery — a model-attractor problem, not hardcoded text. +- **#44** (eng) Cold-start talk repeats the same cozy imagery — absorbed by spec 13; closes on #202's first box. - **#79** (by-ear) The art-direction session for the TUI and the pet — spec 10 §6.1. - **#80** (by-ear) First-run onboarding in a real terminal — spec 06 criterion 12. - **#81** (by-ear) A real day of pacing — spec 07 §5.16. @@ -45,6 +45,7 @@ hand: CI fails if this section points at an issue that is already closed. - **#149** (by-ear) Does the music pick actually stop repeating — spec 03-01 §2.3. - **#197** (by-ear) Memory v1.5 by feel: fading, fold cadence, forgetting, how a recalled memory sounds — spec 05-01 §6. - **#198** (by-ear) The talk<->music transitions: announce hand-over and the slow lift — spec 03-02 §6.1. +- **#202** (by-ear) Real-world topics as a friend would mention them, and the clock as bearings — spec 13 §5, spec 04 §3.4. ## Pinned — do not relitigate diff --git a/specs/spec13/13-real-world-topics.md b/specs/spec13/13-real-world-topics.md new file mode 100644 index 0000000..5b80835 --- /dev/null +++ b/specs/spec13/13-real-world-topics.md @@ -0,0 +1,325 @@ +# spec/13 · real-world-topics — an off-loop pool of things that actually happened + +> **Status**: **Built 2026-09-03** (this PR). Pool, roll, fetch task, prompt +> rendering, the settings knob and the steer field all land; the unit suite is +> green; a real `fetchTopics` was smoked through the SDK and a real `rwt.offer` +> was read out of `.dev/dev.log` against the beat it produced. **§5's by-ear +> criteria are open** — user-run, tracked as one issue. +> **Part**: The "say real things" line of [`../../ROADMAP.md`](../../ROADMAP.md) +> §2. Gives the self-initiated talk task material from outside its own head: +> news, releases, what is happening where the listener is. Absorbs +> [#44](https://github.com/wine-fall/murmur/issues/44) (the cozy-imagery +> attractor) as the durable fix — a cold boot stops being identical when the +> host has something real in front of it. +> **Milestone**: companion character, after specs 04/05/07/11/12. Depends on +> the harness (spec 03-01), the talk look-ahead (spec 04), the settings layer +> (spec 12) and the steer task (spec 11). +> **Network posture (master §3.1, amended)**: this is the **fourth network +> call** — the host's brain, the voice, the music pull, and now a bounded +> WebSearch task. The "three network calls" wording in `DESIGN.md` is already +> stale ([#104](https://github.com/wine-fall/murmur/issues/104)) and is not +> edited here. Nothing about the listener leaves the machine except what the +> task is told: a language name, a timezone name, and the titles already in +> the pool. +> **Conventions**: English; written for a coding agent. Mechanism and +> contracts, not final code. Prompt text centralized in `src/prompts.ts`; no +> CJK in source (master §0). + +--- + +## 1. Goal & scope + +### Delivers + +1. **A topic pool** (`RwtPool`): a small file of real-world items — title, + a two-to-three-sentence gist in the listener's spoken language, a + category — fetched **off the live loop** and read from at talk-generation + time. Entries expire; the pool refreshes itself in the background. +2. **A fetch task** (`Brain.fetchTopics`): one bounded agentic run over the + SDK's built-in `WebSearch` plus one murmur terminal tool, `submit_topics`. + Neutral system framing — a researcher gathering material for a host, never + the persona speaking. +3. **A probability roll** (`RwtRoll`): whether a given talk batch is offered a + topic at all. Not every batch: an item on every batch is a news ticker, and + the listener said so. +4. **The prompt seam**: `ContextPack.rwt` and `rwtLine()` — one item on the + desk, brought in *as a host does*: named, said in a sentence or two, then + carried past. The anchor beats and the coda never carry it. +5. **The knob**: `rwtEnabled` in settings (default on), `--no-rwt`, and a + `rwt` field on `change_settings` so "stop with the news" typed to the radio + turns it off. +6. **The taste file**: `$MURMUR_HOME/rwt-policy.md`, the music-policy shape + (spec 03-01 §2.3) — code owns the contract (shape, freshness, dedupe, + privacy), the listener owns what to look for. + +### Out of scope (explicit non-goals) + +- **A live lookup on the talk path.** Picks already run 80–190 s in a bad + session; anything network-bound at `generateTalks` would break the spec-04 + look-ahead. Talk reads the pool and nothing else. +- **Storing region.** Region is read from the system timezone at fetch time + and written into the fetch prompt; it is never persisted and never asked + for. Language is not region (ROADMAP §2). +- **A new onboarding question, a pane row, or a TUI surface.** The knob is + reachable by flag, file, and the conversation. The pane stays at spec 12's + eight items. +- **Fact-checking, citations, or source attribution on air.** The host may + mention what it read; it does not read out URLs or outlets. +- **Personal or private material.** The fetch prompt forbids it outright + (§3.3); nothing in the pool is about the listener. +- **The stub brain.** `StubBrain.fetchTopics` returns nothing; an empty pool + never offers, so a stub run is exactly its pre-spec-13 self. + +--- + +## 2. Contracts / seams + +### 2.1 The pool entry and the file + +```ts +// src/rwt.ts +export type RwtTopic = { + readonly id: string // opaque, unique within the file + readonly title: string // one line, the thing itself + readonly gist: string // 2–3 spoken sentences, in the listener's language + readonly category: string // free text from the policy's list + readonly fetchedAt: number // epoch seconds + readonly used: boolean // offered once; never offered again +} +``` + +- **Location**: `cacheRoot()/rwt.json` (`src/paths.ts` → `rwtPoolPath`). It is + rebuildable — deleting it costs one fetch. +- **Shape on disk**: `{ refreshedAt?: number, entries: RwtTopic[] }`, parsed + with zod at the boundary. A missing, unreadable, or malformed file is an + empty pool, never a boot failure. +- **Expiry**: an entry older than `ttlHours` (default 48) is dropped on load + and on merge. +- **Refresh due**: `refreshedAt` absent, or older than `staleHours` + (default 6). Checked once at boot and then at every segment boundary. +- **Take**: `take()` returns the oldest fresh, unused entry, marks it used, + persists, and returns it — or `null`. Marking happens **at take time**, not + at air time: a beat that is generated and then discarded (a steer, a quit) + still burns its topic. Accepted: a burned topic costs nothing, a repeated + one costs the illusion. + +### 2.2 The fetch task — `Brain.fetchTopics` + +```ts +export type FetchTopicsRequest = { + readonly language: string // the gist's language, a name ("Japanese") + readonly timezone: string // IANA, from Intl — the only region signal + readonly today: string // YYYY-MM-DD, local + readonly avoid: readonly string[] // titles already in the pool + readonly policy: string // the taste half (§2.5) +} + +interface Brain { + // Bounded WebSearch run; [] on a stub, on a turn budget exhausted, or on + // any failure the caller treats as "no refresh this round". + fetchTopics(req: FetchTopicsRequest): Promise[]> +} +``` + +- **Harness change**: `Task` gains an optional `builtins?: readonly + string[]`. `agenticOptions` puts them on the SDK's `tools` **and** + `allowedTools` — bounded and pre-approved, so the run neither prompts nor + reaches anything else. The default stays `[]`: the music pick and the steer + task are exactly what they were. +- **The fetch task** passes `builtins: ['WebSearch']`, `maxTurns` 12, the + cheap tier (`rwtModel`, default Haiku), and one terminal tool + `submit_topics({ topics: [{ title, gist, category }] })` (zod; 1–8 items; + title ≤ 120 chars, gist ≤ 600, category ≤ 40). +- **System prompt**: `RWT_FETCH_SYSTEM_PROMPT` — "You gather real-world + material for a radio host" — never the persona. +- **Language**: the request's `language` is the effective spoken language, + read where the host reads its own: `settings.language` if set, else the + language the persona says it speaks (`personaLanguage`, the "speak in X" + clause the bundled seed carries), else the persona's own first line of + prose held up as the example ("the language this is written in: …") — a + generated persona is written in the listener's language and never names it + in English (spec 06 §2.2). Never the machine locale: it is not a record of + anything once the install is past its first run. + +### 2.3 The roll — `RwtRoll` + +`RandomCadence`'s shape (spec 03-02 §2.3), one rung down: `{ p, minGap, +maxGap, random }` over a counter of **talk batches since the last offer**. +Never before `minGap` batches; always by `maxGap`; `p` in between. Defaults +`p 0.35, minGap 1, maxGap 4`. Injected `random` for determinism. + +### 2.4 The Director seam + +```ts +// DirectorDeps +rwt?: { + offer(): RwtTopic | null // roll, then take; null = nothing this batch + maybeRefresh(): boolean // single-flight background refresh if due +} +``` + +- `maybeRefresh()` is poked at **every segment boundary**, beside + `compactor.maybeSchedule()`. It never blocks; a failure costs one debug + line and the next boundary retries once the pool is still stale. +- `offer()` is called **once per `generateTalks`** whose cue is neither + `anchor:*` nor `coda`, and only while `settings().rwtEnabled`. A hit lands + as `ContextPack.rwt = { title, gist }` on that batch's pack. +- Absent (stub runs, tests): no roll, no refresh, no field. + +### 2.5 The prompt seam + +- `ContextPack.rwt?: { readonly title: string; readonly gist: string }`. +- `rwtLine(ctx)` renders the item **on the desk for this stretch**, then the + usage: name the thing — the title, who, where, when — say what happened in + a sentence or two and what you make of it, then carry on; one item, in the + host's own voice; not a newsreader's rundown, not a "here is the news" + frame, not a list. The line is drawn on **register, never on content**: an + earlier draft forbade "a headline read out" and let the host "leave it", + and the measured result was the item scrubbed to mood (a Netflix release + became "a friend wanted to watch a show") — the #44 attractor with a fig + leaf. Absent → renders nothing. +- `DEFAULT_RWT_POLICY` / `RWT_POLICY_HEADER` / `buildFetchTopicsPrompt(req)` + live in `src/prompts.ts`. The listener's `rwt-policy.md` replaces the policy + wholesale (HTML comments stripped, the music-policy discipline). +- `STEER_SETTINGS_RULE` names the new knob so the reply turn knows "stop with + the news" is a settings ask. + +### 2.6 The knob (spec 12's shape) + +- `Config.rwtEnabled` (default `true`), `--no-rwt`, `settings.json` + `rwtEnabled`; layered file < flag exactly like `anchorsEnabled`. +- `Settings.rwtEnabled` in `SettingsValuesSchema` / `SettingsPatchSchema`. +- `change_settings` gains `rwt?: boolean` → `rwtEnabled`. +- Numeric knobs are env-only (`MURMUR_RWT_P`, `MURMUR_RWT_MIN_GAP`, + `MURMUR_RWT_MAX_GAP`, `MURMUR_RWT_STALE_HOURS`, `MURMUR_RWT_TTL_HOURS`), + parsed with the same warn-and-default posture as `MURMUR_TTS_*`. + +### 2.7 The verifiable seam — `host.debug` + +| line | when | +|---|---| +| `rwt.refresh n= ms=` | a background refresh merged `n` new entries | +| `rwt.refresh failed ()` | the fetch threw or returned nothing | +| `rwt.offer ` | a topic was taken for a batch | +| `rwt.pool fresh= used=` | after every load/merge/take | + +Read these in `.dev/dev.log` and compare against the beat text before +believing the model did anything. The model's own narration is not evidence. + +--- + +## 3. Design + +### 3.1 Off the loop, by construction + +The pool is the only thing the talk path touches: `offer()` is a synchronous +file-backed read. The fetch runs where the Compactor runs — launched from a +boundary poke, single-flight, unawaited, total (never rejects). A slow or hung +search delays nothing on air; a listener with no network gets the pre-spec-13 +radio plus one debug line per stale boundary. + +### 3.2 Not every batch + +The roll exists because of one by-ear rule: a topic on every batch reads as a +segment, and a segment is what murmur is not. `maxGap` still bounds the +silence so a fresh pool is not ignored forever. The counter is per process; +it does not persist — a restart may offer on the first batch, which is fine. + +### 3.3 The fetch prompt (contract half, code-owned) + +`buildFetchTopicsPrompt` states, in this order: the language the gists must +be written in; the timezone with "weight what matters there, international as +the fallback"; today's date and the freshness rule (**today or yesterday** +only); the titles to avoid (already in the pool); the privacy line (nothing +about private individuals, nothing that identifies a person who is not a +public figure); the output contract (call `submit_topics` once, 3–8 items, +each gist two to three spoken sentences a friend could say without reading +from a screen, no URLs, no outlet names). Then `RWT_POLICY_HEADER` and the +policy. + +### 3.4 The taste half (listener-owned) + +`DEFAULT_RWT_POLICY`: the categories — news, tech, entertainment, sports — +and the weighting: mostly what is happening where the listener is, some of +what the whole world is talking about, nothing that needs a screen to make +sense of, nothing that is only a number, prefer the human-scale angle of a +big story over the headline, and **keep the hard nouns** — a title, a name, +a place, a date — because they are what make the thing real said aloud. Seeded to `rwt-policy.md` on first use so it is +discoverable; read fresh on every fetch. + +### 3.5 Language and region without a store + +Language: the listener's override, else the persona's own word. The persona +is the record once the install is past its first run — the machine locale +that seeded it may have changed since (measured: a persona reading "Always +speak in Chinese (Mandarin)" on a machine whose `LANG` is now `en_US`, where a +locale read would have produced English gists). Region: `Intl.DateTimeFormat().resolvedOptions().timeZone`, read at +fetch time, in the prompt only. A listener in Tokyo with a French persona gets +French gists about what matters in Japan, which is the intended reading of +"language is not region". + +### 3.6 Failure posture + +| failure | effect | +|---|---| +| fetch throws / times out / no tool call | one debug line; pool unchanged; retried at a later stale boundary | +| malformed pool file | empty pool; overwritten by the next successful refresh | +| pool write fails | the take still stands in memory; one debug line (`rwt.pool not persisted`), never a throw on the talk path | +| no network at all | never offers; radio unchanged | +| `rwtEnabled` false | no roll, no offer; the refresh still keeps the pool warm so turning it back on is instant | + +--- + +## 4. Dependencies + +- Spec 03-01 (`runTask`, the in-process MCP tool seam) — extended with + `builtins`. +- Spec 04 (`generateTalks` serves both the cold path and the refill) — the + one call site the roll hangs on. +- Spec 11 (`change_settings`) and spec 12 (the settings store) — the knob. +- Spec 05 §3.5 for where `ContextPack` fields are assembled. + +--- + +## 5. Acceptance criteria + +### Unit (deterministic; all green in this PR) + +1. Pool: expired entries drop on load; `take()` marks and persists; a second + `take()` never returns the same id; `refreshDue` obeys `staleHours`. +2. Roll: with injected random, `minGap` holds, `maxGap` forces, `p` decides + in between. +3. Prompt: `rwtLine` renders the title and gist with the usage lines when + present, nothing when absent; an `anchor:*` or `coda` cue never carries it. +4. Harness: `agenticOptions` with `builtins: ['WebSearch']` lists it in both + `tools` and `allowedTools`; the default lists neither. +5. Knob: `--no-rwt` → `rwtEnabled: false`; the store seeds it from Config; + `change_settings({ rwt: false })` lands `rwtEnabled: false`. +6. Director: a stub/fake `rwt` dep is offered once per non-anchor, non-coda + batch and never on those two. + +### Real seam (done once in this PR, evidence in the PR body) + +7. A real `fetchTopics` through the SDK writes ≥ 3 entries into + `cache/rwt.json`, gists in the requested language, dated today/yesterday. +8. A `make dev` run shows `rwt.offer ` in `.dev/dev.log`, and the beat + generated from that batch is read against the entry's gist. + +### By-ear (open — one issue) + +9. A mentioned topic is named — the title, who, where — and said the way a + host says it, not a newsreader: one item, no rundown, no list. +10. The proportion feels right — present but not every stretch. +11. The gist language matches the persona's spoken language. +12. Turning it off by typing works and the host does not keep mentioning news. + +--- + +## 6. Open questions + +- Whether the refresh should also fire when the pool is *empty of unused + entries* rather than only when stale. Left as the stale-only rule until + by-ear says the pool runs dry. +- Whether `maxGap` should count aired beats rather than batches. Batches are + what `generateTalks` sees; aired beats would need the roll to move into the + buffer. diff --git a/src/app.ts b/src/app.ts index d4a69c4..27cc2e8 100644 --- a/src/app.ts +++ b/src/app.ts @@ -24,7 +24,7 @@ import { ClaudeBrain, StubBrain } from './brain.ts' import { LiveCadence, PacingCadence } from './cadence.ts' import { Compactor } from './compaction.ts' import { packageVersion, type Config, ttsFromFile } from './config.ts' -import type { Harness, MemoryStore, VoiceProvider } from './contracts.ts' +import type { Brain, Harness, MemoryStore, VoiceProvider } from './contracts.ts' import { canOpenBrowser, copyToClipboard, @@ -46,12 +46,13 @@ import { HostedListening } from './listening-data.ts' import { InProcessMemoryStore, PersistentMemoryStore } from './memory.ts' import { sentinelRoot } from './paths.ts' import { readMusicPolicy, seedMusicPolicy } from './music-policy.ts' +import { readRwtPolicy, RealWorldTopics, RwtPool, RwtRoll, seedRwtPolicy } from './rwt.ts' import { MusicProgrammer } from './music-programmer.ts' import { startReport, type ReportDeps, type ReportSession } from './report.ts' import { SteerResponder } from './steer-responder.ts' import { YtDlpMusicProvider } from './music.ts' import { detectLanguage } from './locale.ts' -import { loadPersona, personaLine } from './persona.ts' +import { loadPersona, personaLanguage, personaLine } from './persona.ts' import { lineReader, quitLatch, runSetup, setupComplete, type SetupTargets } from './guide.ts' import { buildFindMusicInstruction } from './prompts.ts' import { LedgerScheduler } from './scheduler.ts' @@ -218,6 +219,7 @@ export function buildSettingsStore( recentWindow: resolved.recentWindow, muted: resolved.muted, tuiPet: resolved.tuiPet, + rwtEnabled: resolved.rwtEnabled, }, touched: stored, log, @@ -280,6 +282,54 @@ function buildMusic( return { source, cadence, engine } } +// The language the gists are written in (spec 13 §3.5), read where the host +// reads its own: the listener's override, else what the persona says it +// speaks, else the persona's own prose held up as the example — a generated +// persona is written in the listener's language and never names it (spec 06 +// §2.2), and the machine locale is not a record of anything. +const PERSONA_SAMPLE_CHARS = 120 + +export function rwtLanguage(override: string | undefined, persona: string): string { + const named = override ?? personaLanguage(persona) + if (named !== undefined) return named + const prose = persona + .split('\n') + .map((line) => line.trim()) + .find((line) => line !== '' && !line.startsWith('#')) + return `the language this is written in: "${(prose ?? persona).slice(0, PERSONA_SAMPLE_CHARS)}"` +} + +// Real-world topics (spec 13): the pool under cache/, the roll from the env +// knobs, the fetch on the cheap tier. `language` is read at fetch time from +// where the host reads it, so an override lands on the next refresh; region +// is the system timezone, in the prompt only, never stored. +export function buildRwt( + config: Config, + brain: Pick, + language: () => string, + host: Host, +): RealWorldTopics { + if (seedRwtPolicy(config.rwtPolicyPath)) host.debug?.(`rwt.policy seeded ${config.rwtPolicyPath}`) + return new RealWorldTopics({ + pool: new RwtPool({ + path: config.rwtPoolPath, + ttlHours: config.rwtTtlHours, + staleHours: config.rwtStaleHours, + ...(host.debug !== undefined && { log: host.debug.bind(host) }), + }), + roll: new RwtRoll({ p: config.rwtP, minGap: config.rwtMinGap, maxGap: config.rwtMaxGap }), + brain, + request: () => ({ + language: language(), + timezone: Intl.DateTimeFormat().resolvedOptions().timeZone, + // en-CA is the one locale whose short date is ISO YYYY-MM-DD, local. + today: new Date().toLocaleDateString('en-CA'), + policy: readRwtPolicy(config.rwtPolicyPath), + }), + ...(host.debug !== undefined && { log: host.debug.bind(host) }), + }) +} + // Presence wiring (spec 07). The sensor is what puts the activity cue in the // pack and stretches the away gap, so it rides along whenever EITHER feature // is on — but with both off, the block is dropped entirely and the Director is @@ -786,6 +836,18 @@ export async function runApp(config: Config, maxSegments?: number): Promise rwtLanguage(settings.current().language, persona), + host, + ) + const director = new Director({ persona, brain, @@ -804,6 +866,7 @@ export async function runApp(config: Config, maxSegments?: number): Promise { + // Offline: an empty pool never offers, so a stub run is its pre-spec-13 self. + return [] + } } // Full isolation from the user's local Claude Code environment: no CLAUDE.md / @@ -110,20 +118,24 @@ export function isolatedOptions(systemPrompt: string, model: string): Options { // Options for an agentic task over murmur's OWN in-process MCP tools (spec // 03-01 §2.1): same isolation, but the allowlist is exactly murmur's tools. +// `builtins` (spec 13 §2.2) are the SDK's own tools a task may use beside +// murmur's: on `tools` so they are the whole built-in surface, and on +// `allowedTools` so the run never stops to ask. Default none. export function agenticOptions( systemPrompt: string, model: string, server: McpSdkServerConfigWithInstance, toolNames: string[], maxTurns: number, + builtins: readonly string[] = [], ): Options { return { systemPrompt, model, settingSources: [], strictMcpConfig: true, - tools: [], - allowedTools: toolNames, + tools: [...builtins], + allowedTools: [...toolNames, ...builtins], mcpServers: { murmur: server }, skills: [], maxTurns, @@ -360,7 +372,7 @@ export class ClaudeBrain implements Brain, Harness, GuideCapable { const allowed = tools.map((t) => `mcp__murmur__${t.name}`) const q = query({ prompt: task.prompt, - options: agenticOptions(task.systemPrompt, task.model, server, allowed, task.maxTurns), + options: agenticOptions(task.systemPrompt, task.model, server, allowed, task.maxTurns, task.builtins), }) for await (const _message of q) { if (captured !== null) break @@ -406,6 +418,13 @@ export class ClaudeBrain implements Brain, Harness, GuideCapable { return this.generate(SEED_PERSONA_SYSTEM_PROMPT, buildSeedPersonaPrompt(answers, language)) } + // The real-world fetch (spec 13 §2.2): the same bounded loop, with WebSearch + // let in and the cheap tier driving. null (budget out, no terminal call) + // reads as "nothing this round". + async fetchTopics(req: FetchTopicsRequest): Promise { + return (await this.runTask(fetchTopicsTask(req, this.model))) ?? [] + } + private async generate(persona: string, prompt: string): Promise { const parts: string[] = [] for await (const message of query({ prompt, options: isolatedOptions(persona, this.model) })) { diff --git a/src/config.ts b/src/config.ts index fc33d25..4966856 100644 --- a/src/config.ts +++ b/src/config.ts @@ -14,7 +14,16 @@ import { parseArgs } from 'node:util' import { z } from 'zod' import { LogEvidenceSchema, resolveLogSource, type LogEvidence } from './dev-log.ts' -import { dataRoot, homeRoot, musicPolicyPath, settingsPath, tuiSocketPath, voiceConfigPath } from './paths.ts' +import { + dataRoot, + homeRoot, + musicPolicyPath, + rwtPolicyPath, + rwtPoolPath, + settingsPath, + tuiSocketPath, + voiceConfigPath, +} from './paths.ts' import { DEFAULT_PERSONA_PATH } from './prompts.ts' import { readSettingsFile } from './settings.ts' import { MAX_SPEED, MIN_SPEED, readVoiceConfig, type VoiceConfig } from './voice-config.ts' @@ -100,6 +109,19 @@ export const ConfigSchema = z.object({ // degrades to talk-with-silence. bedEnabled: z.boolean().default(true), + // --- real-world topics (spec 13 §2.6) ----------------------------------- // + // On/off is a settings-layer knob (file < flag, anchorsEnabled's shape); the + // numbers are env-only by-ear knobs. The fetch rides the cheap tier. + rwtEnabled: z.boolean().default(true), + rwtPoolPath: z.string().default(() => rwtPoolPath()), + rwtPolicyPath: z.string().default(() => rwtPolicyPath()), + rwtModel: z.string().default('claude-haiku-4-5-20251001'), + rwtP: z.coerce.number().min(0).max(1).default(0.35), + rwtMinGap: z.coerce.number().int().nonnegative().default(1), + rwtMaxGap: z.coerce.number().int().nonnegative().default(4), + rwtStaleHours: z.coerce.number().positive().default(6), + rwtTtlHours: z.coerce.number().positive().default(48), + // --- proactive & pacing (spec 07 §3.7) ---------------------------------- // // On/off as config; the behavioral shape (thresholds, windows) stays as // module constants. Both off = pre-spec-07 behavior. @@ -232,6 +254,23 @@ function ttsFromEnv(env: NodeJS.ProcessEnv): Partial { } } +// The MURMUR_RWT_* numbers (spec 13 §2.6): the same warn-and-default posture, +// omitted when unset so the schema default stands. +function rwtFromEnv(env: NodeJS.ProcessEnv): Partial { + const p = envNumber(env, 'MURMUR_RWT_P', z.coerce.number().min(0).max(1)) + const minGap = envNumber(env, 'MURMUR_RWT_MIN_GAP', z.coerce.number().int().nonnegative()) + const maxGap = envNumber(env, 'MURMUR_RWT_MAX_GAP', z.coerce.number().int().nonnegative()) + const staleHours = envNumber(env, 'MURMUR_RWT_STALE_HOURS', z.coerce.number().positive()) + const ttlHours = envNumber(env, 'MURMUR_RWT_TTL_HOURS', z.coerce.number().positive()) + return { + ...(p !== undefined && { rwtP: p }), + ...(minGap !== undefined && { rwtMinGap: minGap }), + ...(maxGap !== undefined && { rwtMaxGap: maxGap }), + ...(staleHours !== undefined && { rwtStaleHours: staleHours }), + ...(ttlHours !== undefined && { rwtTtlHours: ttlHours }), + } +} + // The guide-written endpoint (spec 03-03 §7.2). The lowest layer of the three: // a damaged or absent file is simply no endpoint, never a boot failure. // @@ -275,6 +314,7 @@ export function parseCli(argv: string[], env: NodeJS.ProcessEnv = process.env): 'no-bed': { type: 'boolean' }, 'no-anchors': { type: 'boolean' }, 'no-gating': { type: 'boolean' }, + 'no-rwt': { type: 'boolean' }, tui: { type: 'boolean' }, plain: { type: 'boolean' }, setup: { type: 'boolean' }, @@ -305,6 +345,9 @@ export function parseCli(argv: string[], env: NodeJS.ProcessEnv = process.env): home: homeRoot(env), memoryDir: join(dataRoot(env), 'memory'), musicPolicyPath: musicPolicyPath(env), + rwtPoolPath: rwtPoolPath(env), + rwtPolicyPath: rwtPolicyPath(env), + ...rwtFromEnv(env), listeningApiKey: env.MURMUR_LISTENING_API_KEY?.trim() ?? '', listeningUrl: env.MURMUR_LISTENING_URL?.trim() ?? '', tuiSocket: tuiSocketPath(env), @@ -324,6 +367,7 @@ export function parseCli(argv: string[], env: NodeJS.ProcessEnv = process.env): ...(values['no-bed'] === true && { bedEnabled: false }), ...(values['no-anchors'] === true && { anchorsEnabled: false }), ...(values['no-gating'] === true && { gatingEnabled: false }), + ...(values['no-rwt'] === true && { rwtEnabled: false }), ...(values.tui === true && { frontEnd: 'tui' }), // Last, so an explicit opt-out always wins over a redundant opt-in. ...(values.plain === true && { frontEnd: 'plain' }), diff --git a/src/contracts.ts b/src/contracts.ts index 95aed8e..1d29d94 100644 --- a/src/contracts.ts +++ b/src/contracts.ts @@ -67,6 +67,38 @@ export type ContextPack = { readonly coveredTopics?: readonly string[] readonly activity?: Activity readonly cue?: string + // One real-world item offered to this batch (spec 13 §2.5): material, not + // an assignment. Absent on most batches — the roll decides — and always + // absent on an anchor or coda beat. + readonly rwt?: { readonly title: string; readonly gist: string } +} + +// --- real-world topics (spec 13) ------------------------------------------ // + +// What the fetch task hands back: the item itself. The pool adds identity, +// age and the used mark. +export type FetchedTopic = { + readonly title: string + readonly gist: string + readonly category: string +} + +export type RwtTopic = FetchedTopic & { + readonly id: string + readonly fetchedAt: number // epoch seconds + readonly used: boolean +} + +// Everything the fetch is told about the listener (spec 13 §2.2): a language +// name for the gists, the timezone as the only region signal, today's date, +// the titles already held, and the listener's taste half. Nothing else leaves +// the machine. +export type FetchTopicsRequest = { + readonly language: string + readonly timezone: string + readonly today: string + readonly avoid: readonly string[] + readonly policy: string } export interface VoiceProvider { @@ -244,6 +276,10 @@ export type Task = { readonly model: string // tier per task (music search -> Haiku) readonly maxTurns: number // hard bound on the tool-use loop readonly tools: (finish: (value: T) => void) => TaskTool[] + // Built-in SDK tools the task may use beside murmur's own (spec 13 §2.2). + // Bounded AND pre-approved by the harness. Default none: the pick and the + // steer task stay tool-less underneath. + readonly builtins?: readonly string[] } // The agentic capability, separate from the tool-less Brain so talk-only brains @@ -391,4 +427,8 @@ export interface Brain { // the answers do not settle the question. Tool-less text generation, same // posture as compactProfile. seedPersona(answers: readonly SeedAnswer[], language: string): Promise + // One bounded WebSearch run for real-world material (spec 13 §2.2), under a + // neutral framing — a researcher, never the persona. [] on the stub, on an + // exhausted turn budget, or when the model never made the terminal call. + fetchTopics(req: FetchTopicsRequest): Promise } diff --git a/src/director.ts b/src/director.ts index 390b8bf..9cf44d4 100644 --- a/src/director.ts +++ b/src/director.ts @@ -37,6 +37,7 @@ import type { TrackSource, Turn, VoiceProvider, + RwtTopic, } from './contracts.ts' import type { Host } from './host.ts' import { COMMANDS, type ProgramState } from './ipc.ts' @@ -205,6 +206,9 @@ export type DirectorSettings = { recentWindow: number anchorsEnabled: boolean musicEnabled: boolean + // Whether an ordinary talk batch may be offered real-world material (spec + // 13 §2.6); read live at each batch, so "stop with the news" lands at once. + rwtEnabled: boolean // Absent = the persona decides (spec 12 §3.9). Read live like every other // knob here, so a change lands on the next beat with no restart. language?: string | undefined @@ -234,6 +238,10 @@ export type DirectorDeps = { // boundary. Absent = disabled (stub runs, tests). The Director only pokes; // scheduling, single-flight, and failure posture live in the Compactor. compactor?: { maybeSchedule(): boolean } + // Real-world material (spec 13 §2.4): one synchronous offer per ordinary + // talk batch, and a refresh poked at every boundary that runs off the loop + // like the compactor's fold. Absent = never offered (stub runs, tests). + rwt?: { offer(): RwtTopic | null; maybeRefresh(): boolean } // The mid-broadcast recall (spec 10 §3.4): a typed /setup parks the talk // loop inside this call — the engine keeps playing — and the loop resumes // when it returns. Absent (stub runs): /setup answers with the shell pointer. @@ -439,6 +447,7 @@ export class Director { private beginBoundary(): void { this.now = new Date() this.readActivity() + this.deps.rwt?.maybeRefresh() // background, single-flight } // The sensor read, honoring MURMUR_ACTIVITY (by-ear). Also taken right after @@ -656,7 +665,11 @@ export class Director { // them instead of regenerating the same beat — the buffered text lives here // in the Director, so the stateless Brain is told what is already scheduled, // not only what has aired and been recorded. - private context(queued: readonly string[] = [], cue?: string): ContextPack { + private context( + queued: readonly string[] = [], + cue?: string, + rwt?: ContextPack['rwt'], + ): ContextPack { const window = this.deps.settings().recentWindow const recent = this.deps.memory.recent(window) const turns: Turn[] = queued.map((text) => ({ role: 'radio', text })) @@ -675,6 +688,7 @@ export class Director { coveredTopics: this.deps.memory.recentTopics(window), ...(this.activity !== undefined && { activity: this.activity }), ...(cue !== undefined && { cue }), + ...(rwt !== undefined && { rwt }), } } @@ -1106,9 +1120,14 @@ export class Director { queued: readonly string[] = [], cue?: string, ): Promise { + // One roll per batch, before the retry loop: a retried batch is the same + // batch, not a second chance at a topic. Anchors and the coda have a job + // of their own and are never offered one (spec 13 §2.4). + const offered = cue === undefined && this.deps.settings().rwtEnabled ? this.deps.rwt?.offer() : undefined + const rwt = offered == null ? undefined : { title: offered.title, gist: offered.gist } for (let attempt = 1; attempt <= ATTEMPTS; attempt++) { try { - return await this.deps.brain.nextTalks(this.context(queued, cue), count) + return await this.deps.brain.nextTalks(this.context(queued, cue, rwt), count) } catch (err) { if (attempt < ATTEMPTS) { this.deps.host.debug?.(`talk.next_talks failed (attempt ${attempt}/${ATTEMPTS}); retrying`) diff --git a/src/ipc.ts b/src/ipc.ts index 7fdefc1..08cab96 100644 --- a/src/ipc.ts +++ b/src/ipc.ts @@ -83,6 +83,8 @@ export const SettingsValuesSchema = z.object({ // swap (`--voice stub` remains the dev-surface knob for not synthesizing). muted: z.boolean(), tuiPet: z.boolean(), + // Whether the host is offered real-world material at all (spec 13 §2.6). + rwtEnabled: z.boolean(), // The one OPTIONAL knob (spec 12 §3.9). Absent means the listener never said, // and the persona decides; set is an override applied as a directive on top // of the persona, never an edit to persona.md. Free text — a language name as @@ -102,6 +104,7 @@ export const SettingsPatchSchema = z.object({ recentWindow: z.number().int().positive().optional(), muted: z.boolean().optional(), tuiPet: z.boolean().optional(), + rwtEnabled: z.boolean().optional(), // Empty string is legal HERE and only here: it is how the listener clears the // override and hands the language back to the persona (spec 12 §3.9). language: z.union([LanguageSchema, z.literal('')]).optional(), diff --git a/src/music-policy.ts b/src/music-policy.ts index e059545..199868e 100644 --- a/src/music-policy.ts +++ b/src/music-policy.ts @@ -53,7 +53,9 @@ export function parseMusicPolicy(text: string): string | undefined { return body === '' ? undefined : body } -export function readMusicPolicy(path: string): string | undefined { +// The policy-file discipline, shared with the real-world topic policy (spec +// 13 §2.5): comments stripped, absent/empty/unreadable = no policy. +export function readPolicyFile(path: string): string | undefined { try { return parseMusicPolicy(readFileSync(path, 'utf-8')) } catch { @@ -64,12 +66,20 @@ export function readMusicPolicy(path: string): string | undefined { // Seeded once at boot, because a policy the listener never sees is one they // can never edit. `wx` makes "already there" a normal outcome, so their own // text is never overwritten. Returns whether this call wrote the file. -export function seedMusicPolicy(path: string): boolean { +export function seedPolicyFile(path: string, template: string): boolean { try { mkdirSync(dirname(path), { recursive: true }) - writeFileSync(path, TEMPLATE, { encoding: 'utf-8', flag: 'wx' }) + writeFileSync(path, template, { encoding: 'utf-8', flag: 'wx' }) return true } catch { return false } } + +export function readMusicPolicy(path: string): string | undefined { + return readPolicyFile(path) +} + +export function seedMusicPolicy(path: string): boolean { + return seedPolicyFile(path, TEMPLATE) +} diff --git a/src/paths.ts b/src/paths.ts index 7155f3f..b700da4 100644 --- a/src/paths.ts +++ b/src/paths.ts @@ -86,3 +86,14 @@ export function settingsPath(env: NodeJS.ProcessEnv = process.env): string { export function musicPolicyPath(env: NodeJS.ProcessEnv = process.env): string { return join(homeRoot(env), 'music-policy.md') } + +// The real-world topic pool (spec 13 §2.1): rebuildable, so it lives under +// cache/ — deleting it costs one fetch. +export function rwtPoolPath(env: NodeJS.ProcessEnv = process.env): string { + return join(cacheRoot(env), 'rwt.json') +} + +// The listener's topic policy (spec 13 §2.5): the music-policy shape, beside it. +export function rwtPolicyPath(env: NodeJS.ProcessEnv = process.env): string { + return join(homeRoot(env), 'rwt-policy.md') +} diff --git a/src/persona.ts b/src/persona.ts index aa2da31..2677035 100644 --- a/src/persona.ts +++ b/src/persona.ts @@ -44,3 +44,18 @@ export function loadPersona(path: string, language: string): string { if (!text) throw new Error(`persona seed file is empty: ${path}`) return renderPersona(text, language) } + +// The language the persona says it speaks (spec 13 §3.5). Once past the first +// run the persona is the record — the machine locale that seeded it may have +// changed since — and it states its language in a sentence, not a field: +// "Always speak in Chinese (Mandarin)." / "Speak in Japanese, softly." The +// first such clause, up to the sentence's end. A language is a proper noun, +// so the capture must open with a capital: "speak in a warm tone" is manner, +// not language. Undefined when the persona never says — a generated persona +// is written in the listener's language and does not name it in English. +const SPEAKS_IN = /\b[Ss]peak(?:s|ing)? in ([A-Z][^.,;\n*]*)/ + +export function personaLanguage(persona: string): string | undefined { + const name = SPEAKS_IN.exec(persona)?.[1]?.trim() + return name ? name : undefined +} diff --git a/src/prompts.ts b/src/prompts.ts index 412b6d5..91c8697 100644 --- a/src/prompts.ts +++ b/src/prompts.ts @@ -8,7 +8,7 @@ import { fileURLToPath } from 'node:url' -import type { ContextPack, RecallHit, SeedAnswer, Turn } from './contracts.ts' +import type { ContextPack, FetchTopicsRequest, RecallHit, SeedAnswer, Turn } from './contracts.ts' // The bundled static persona seed (L0). Spec 06 will generate/evolve personas // at runtime; this is only the default. @@ -176,6 +176,27 @@ function coveredLine(ctx: ContextPack): string { return `\n(Recently covered — don't repeat these: ${ctx.coveredTopics.join(', ')})` } +// One real-world item on the desk for this stretch (spec 13 §2.5). A host +// names the thing — the title, who, where, when — says what happened and what +// they make of it, and carries on; what a host does NOT do is switch into a +// newsreader's rundown. The line draws on register, never on content: an item +// with its names scrubbed out is the cozy-imagery attractor (#44) wearing a +// fig leaf. The anchor beats and the coda have a job of their own and never +// carry one, even if a pack arrives with it. Absent -> nothing. +function rwtLine(ctx: ContextPack): string { + const rwt = ctx.rwt + if (rwt === undefined) return '' + const cue = ctx.cue ?? '' + if (cue === CODA_CUE || cue.startsWith('anchor:')) return '' + return ( + `\n(On the desk for this stretch, from today: ${rwt.title} — ${rwt.gist})\n` + + 'Bring it in the way a host does: name the thing — the title, who, where, ' + + 'when — say what happened in a sentence or two and what you make of it, ' + + 'then carry on. One item, in your own voice. Not a newsreader\'s rundown, ' + + 'not a "here is the news" frame, not a list.' + ) +} + // Render recent turns as a transcript. The host's own prior lines are "You"; // the listener's lines are "Listener". function renderTranscript(ctx: ContextPack, dropTrailingUser?: string): string { @@ -196,7 +217,7 @@ export function buildNextTalkPrompt(ctx: ContextPack): string { const head = transcript ? `(The program so far)\n${transcript}\n\nNow continue — say your next beat.` : 'The program is just starting. Open naturally with your first beat.' - return `${profileBlock(ctx)}${head}${coveredLine(ctx)}${sceneLine(ctx)}${musicLine(ctx)}${pacingLines(ctx)}\n${groundingRules(ctx)}\n${OUTPUT_RULES}` + return `${profileBlock(ctx)}${head}${coveredLine(ctx)}${sceneLine(ctx)}${musicLine(ctx)}${pacingLines(ctx)}${rwtLine(ctx)}\n${groundingRules(ctx)}\n${OUTPUT_RULES}` } // Prompt for the next `count` self-initiated beats in one call. The beats come @@ -208,7 +229,7 @@ export function buildNextTalksPrompt(ctx: ContextPack, count: number): string { ? `(The program so far)\n${transcript}\n\nNow continue — say your next ${count} beats.` : `The program is just starting. Open naturally with your first ${count} beats.` return ( - `${profileBlock(ctx)}${head}${coveredLine(ctx)}${sceneLine(ctx)}${musicLine(ctx)}${pacingLines(ctx)}\n` + + `${profileBlock(ctx)}${head}${coveredLine(ctx)}${sceneLine(ctx)}${musicLine(ctx)}${pacingLines(ctx)}${rwtLine(ctx)}\n` + `${groundingRules(ctx)}\n` + 'Each beat is one small stretch of radio (a few sentences, spoken aloud — ' + 'no markup, labels, or stage directions). Return ' + @@ -322,6 +343,62 @@ export function buildMusicSituation(recent: readonly Turn[], avoid: readonly str ) } +// --- real-world topics (spec 13 §3.3/§3.4) -------------------------------- // + +// A researcher, never the persona: the gists are handed to the host later as +// material, so nothing here may speak in the host's voice. +export const RWT_FETCH_SYSTEM_PROMPT = + 'You gather real-world material for a radio host: a few things that ' + + 'actually happened, each told briefly enough that a friend could mention ' + + 'it in passing without reading from a screen.' + +export const RWT_POLICY_HEADER = 'What to look for:' + +// The TASTE half — replaceable wholesale by $MURMUR_HOME/rwt-policy.md. +export const DEFAULT_RWT_POLICY = `1. Four kinds of thing: news, tech, entertainment, sports. Mix them; do not + let one kind take the whole batch. + +2. Mostly what is happening where the listener is, some of what the whole + world is talking about. Local first, international as the fallback, never + the other way round. + +3. Prefer the human-scale angle of a big story over the headline: what it is + like for the people in it, not the number in the title. + +4. Nothing that needs a screen to make sense of — no charts, no tables, no + "as shown below". Nothing that is only a figure. + +5. Something a host would actually bring up on air: a release, a match, a + small strange thing that happened, a thing people are arguing about. Skip + what is merely important. + +6. Keep the hard nouns. A title, a name, a place, a date, a number that + matters — those are what make a thing real when it is said aloud. A gist + with them scrubbed out is mood, not material.` + +// The CONTRACT half — code-owned: language, region, freshness, dedupe, +// privacy, and how the task ends. A listener policy cannot loosen these. +export function buildFetchTopicsPrompt(req: FetchTopicsRequest): string { + const avoid = + req.avoid.length === 0 + ? '' + : `\nAlready in the pool — find something else:\n${req.avoid.map((t) => `- ${t}`).join('\n')}\n` + return ( + `Write every title and every gist in ${req.language}. The listener is in the ${req.timezone} ` + + 'timezone; weight what matters there, international as the fallback.\n' + + `Today is ${req.today}. Only things from today or yesterday — nothing older, ` + + 'nothing undated.\n' + + 'Nothing about private individuals, and nothing that identifies a person ' + + 'who is not a public figure.\n' + + `${avoid}\n` + + 'Use WebSearch to find candidates, then call submit_topics ONCE with three ' + + 'to eight items. Each item: a one-line title, a gist of two to three spoken ' + + 'sentences a friend could say from memory (no URLs, no outlet names, no ' + + 'quotes), and its kind. Calling submit_topics ends the task.\n\n' + + `${RWT_POLICY_HEADER}\n${req.policy.trim()}` + ) +} + // --- cadence (spec 03-02 §2.3, brain mode only) --------------------------- // export const CADENCE_INSTRUCTION = `You are pacing a personal radio program. Decide what the NEXT segment should @@ -835,7 +912,8 @@ const STEER_END_RULE = const STEER_SETTINGS_RULE = '- An explicit ask to change how the radio behaves — music on/off, more ' + 'music or more talk, breathing room, sound/mute, the morning and night ' + - 'moments, the pixel pet, memory span, or the language it speaks -> call ' + + 'moments, the pixel pet, memory span, the language it speaks, or whether it ' + + 'brings up real-world news and happenings at all -> call ' + 'change_settings with only the fields they asked about, then say what ' + 'changed. A mood remark is not a request ("this song is too loud" is not ' + '"mute"). For the language, pass the language name; pass an empty string to ' + diff --git a/src/rwt.ts b/src/rwt.ts new file mode 100644 index 0000000..f4c64c3 --- /dev/null +++ b/src/rwt.ts @@ -0,0 +1,343 @@ +// Real-world topics (spec 13): a small file-backed pool of things that actually +// happened, fetched OFF the live loop and read from at talk-generation time. +// +// Three pieces, each the shape of something already here: the pool is a +// cache file with expiry; the roll is RandomCadence one rung down (a +// probability with guardrails over a counter); the feed launches the fetch the +// way the Compactor launches a fold — single-flight, unawaited, total. The +// talk path only ever touches `offer()`, a synchronous read. + +import { randomUUID } from 'node:crypto' +import { mkdirSync, readFileSync, writeFileSync } from 'node:fs' +import { dirname } from 'node:path' + +import { tool } from '@anthropic-ai/claude-agent-sdk' +import { z } from 'zod' + +import type { Brain, FetchedTopic, FetchTopicsRequest, RwtTopic, Task } from './contracts.ts' +import { readPolicyFile, seedPolicyFile } from './music-policy.ts' +import { buildFetchTopicsPrompt, DEFAULT_RWT_POLICY, RWT_FETCH_SYSTEM_PROMPT } from './prompts.ts' + +// --- the pool (§2.1) ------------------------------------------------------ // + +const TopicSchema = z.object({ + id: z.string(), + title: z.string(), + gist: z.string(), + category: z.string(), + fetchedAt: z.number(), + used: z.boolean(), +}) + +const PoolFileSchema = z.object({ + refreshedAt: z.number().optional(), + entries: z.array(TopicSchema), +}) + +type PoolFile = z.infer + +export type RwtPoolOptions = { + path: string + ttlHours?: number + staleHours?: number + // Epoch seconds; injected so tests own the clock. + now?: () => number + log?: (message: string) => void +} + +const DEFAULT_TTL_HOURS = 48 +const DEFAULT_STALE_HOURS = 6 + +export class RwtPool { + private path: string + private ttlS: number + private staleS: number + private now: () => number + private log: ((message: string) => void) | undefined + private file: PoolFile + + constructor({ path, ttlHours = DEFAULT_TTL_HOURS, staleHours = DEFAULT_STALE_HOURS, now, log }: RwtPoolOptions) { + this.path = path + this.ttlS = ttlHours * 3600 + this.staleS = staleHours * 3600 + this.now = now ?? (() => Date.now() / 1000) + this.log = log + this.file = this.load() + } + + // A missing, unreadable or malformed file is an empty pool — the cache is + // rebuildable, so nothing here may fail a boot. Expired entries drop here. + private load(): PoolFile { + let parsed: unknown + try { + parsed = JSON.parse(readFileSync(this.path, 'utf-8')) + } catch { + return { entries: [] } + } + const checked = PoolFileSchema.safeParse(parsed) + if (!checked.success) return { entries: [] } + return { ...checked.data, entries: checked.data.entries.filter((e) => this.fresh(e)) } + } + + private fresh(entry: RwtTopic): boolean { + return this.now() - entry.fetchedAt < this.ttlS + } + + // Best-effort: the in-memory state stands whether or not the disk took it. + // take() runs on the talk path, and an unwritable cache must cost a line in + // the log, never the radio. + private save(): void { + try { + mkdirSync(dirname(this.path), { recursive: true }) + writeFileSync(this.path, `${JSON.stringify(this.file, null, 2)}\n`, 'utf-8') + } catch (err) { + this.log?.(`rwt.pool not persisted (${String(err)})`) + } + } + + refreshDue(): boolean { + const at = this.file.refreshedAt + return at === undefined || this.now() - at >= this.staleS + } + + titles(): string[] { + return this.file.entries.filter((e) => this.fresh(e)).map((e) => e.title) + } + + counts(): { fresh: number; used: number } { + const live = this.file.entries.filter((e) => this.fresh(e)) + const used = live.filter((e) => e.used).length + return { fresh: live.length - used, used } + } + + // The oldest fresh unused entry, marked used at TAKE time: a beat generated + // and then discarded still burns its topic (a repeat costs more than a miss). + take(): RwtTopic | null { + const index = this.file.entries.findIndex((e) => !e.used && this.fresh(e)) + if (index === -1) return null + const taken = { ...this.file.entries[index]!, used: true } + this.file.entries[index] = taken + this.save() + return taken + } + + // Add what a fetch brought back, skipping titles already held, and stamp + // the refresh. Returns how many were new. + // Add what a fetch brought back, skipping titles still held — an expired + // entry is dropped first, so a recurring story can come back — and stamp + // the refresh. Returns how many were new. + merge(topics: readonly FetchedTopic[]): number { + const entries = this.file.entries.filter((e) => this.fresh(e)) + const held = new Set(entries.map((e) => e.title)) + const at = this.now() + let added = 0 + for (const topic of topics) { + if (held.has(topic.title)) continue + held.add(topic.title) + entries.push({ ...topic, id: randomUUID().slice(0, 8), fetchedAt: at, used: false }) + added++ + } + this.file = { refreshedAt: at, entries } + this.save() + return added + } +} + +// --- the roll (§2.3) ------------------------------------------------------ // + +export type RwtRollOptions = { + p?: number + minGap?: number + maxGap?: number + // Injected so tests are deterministic. + random?: () => number +} + +// Probability p per talk batch, guarded: never before minGap batches since the +// last offer, always by maxGap — an item on every batch is a ticker, and a +// fresh pool ignored forever is a wasted fetch. +export class RwtRoll { + private p: number + private minGap: number + private maxGap: number + private random: () => number + private since = 0 + + constructor({ p = 0.35, minGap = 1, maxGap = 4, random = Math.random }: RwtRollOptions = {}) { + this.p = p + this.minGap = Math.max(0, minGap) + this.maxGap = Math.max(this.minGap, maxGap) + this.random = random + } + + roll(): boolean { + this.since++ + if (this.since < this.minGap) return false + const hit = this.since >= this.maxGap || this.random() < this.p + if (hit) this.since = 0 + return hit + } +} + +// --- the feed (§2.4 / §3.1) ----------------------------------------------- // + +export type RealWorldTopicsDeps = { + pool: RwtPool + roll: RwtRoll + brain: Pick + // Resolved at fetch time, so a language change lands on the next refresh. + request: () => Omit + log?: (message: string) => void +} + +type Refresh = { promise: Promise; done: () => boolean } + +export class RealWorldTopics { + private deps: RealWorldTopicsDeps + private refresh: Refresh | null = null + + constructor(deps: RealWorldTopicsDeps) { + this.deps = deps + } + + // The talk path's one call: roll, then take. An empty pool does not consume + // the roll, so the first item after a refresh is not owed to an old miss. + offer(): RwtTopic | null { + if (this.deps.pool.counts().fresh === 0) return null + if (!this.deps.roll.roll()) return null + const taken = this.deps.pool.take() + if (taken === null) return null + this.log(`rwt.offer ${taken.id}`) + this.logPool() + return taken + } + + // Launch one background refresh if the pool is stale and none is in flight. + // Returns whether it launched one. + maybeRefresh(): boolean { + if (!this.deps.pool.refreshDue()) return false + if (this.refresh !== null && !this.refresh.done()) return false + let settled = false + this.refresh = { promise: this.run().finally(() => (settled = true)), done: () => settled } + return true + } + + // Await the in-flight refresh, if any (shutdown / tests). + async drain(): Promise { + await this.refresh?.promise + this.refresh = null + } + + // Total: runs unawaited, so an escape here would be an unhandled rejection + // taking the radio down instead of one quiet line in the log. + private async run(): Promise { + const started = Date.now() + try { + const topics = await this.deps.brain.fetchTopics({ + ...this.deps.request(), + avoid: this.deps.pool.titles(), + }) + if (topics.length === 0) { + this.log('rwt.refresh failed (no topics returned)') + return + } + const n = this.deps.pool.merge(topics) + this.log(`rwt.refresh n=${n} ms=${Date.now() - started}`) + this.logPool() + } catch (err) { + this.log(`rwt.refresh failed (${String(err)})`) + } + } + + private logPool(): void { + const { fresh, used } = this.deps.pool.counts() + this.log(`rwt.pool fresh=${fresh} used=${used}`) + } + + private log(message: string): void { + this.deps.log?.(message) + } +} + +// --- the fetch task (§2.2) ------------------------------------------------ // + +const FETCH_MAX_TURNS = 12 + +const topicShape = { + topics: z + .array( + z.object({ + title: z.string().max(120).describe('one line, the thing itself'), + gist: z.string().max(600).describe('two to three spoken sentences, in the requested language'), + category: z.string().max(40).describe('its kind, from the list you were given'), + }), + ) + .min(1) + .max(8), +} + +// Trim, drop anything without a title or gist. Runs on schema-validated input. +export function cleanTopics(raw: z.infer): FetchedTopic[] { + const out: FetchedTopic[] = [] + for (const t of raw) { + const title = t.title.trim() + const gist = t.gist.trim() + if (!title || !gist) continue + out.push({ title, gist, category: t.category.trim() }) + } + return out +} + +// One bounded run over WebSearch, ended by submit_topics (the spec 03-01 +// termination rule). Neutral framing: the researcher, never the host. +export function fetchTopicsTask(req: FetchTopicsRequest, model: string): Task { + return { + systemPrompt: RWT_FETCH_SYSTEM_PROMPT, + prompt: buildFetchTopicsPrompt(req), + model, + maxTurns: FETCH_MAX_TURNS, + builtins: ['WebSearch'], + tools: (finish) => [ + tool( + 'submit_topics', + 'Hand over the real-world items you found. Call it once; calling it ends the task.', + topicShape, + async (args) => { + const topics = cleanTopics(args.topics) + if (topics.length > 0) finish(topics) + return { content: [{ type: 'text', text: JSON.stringify({ ok: true, topics: topics.length }) }] } + }, + ), + ], + } +} + +// --- the taste file (§2.5) ------------------------------------------------ // + +const POLICY_TEMPLATE = ` + +${DEFAULT_RWT_POLICY} +` + +export function readRwtPolicy(path: string): string { + return readPolicyFile(path) ?? DEFAULT_RWT_POLICY +} + +export function seedRwtPolicy(path: string): boolean { + return seedPolicyFile(path, POLICY_TEMPLATE) +} diff --git a/src/steer-tools.ts b/src/steer-tools.ts index db2a779..a4fa830 100644 --- a/src/steer-tools.ts +++ b/src/steer-tools.ts @@ -30,6 +30,7 @@ type SettingsIntent = { pet?: boolean | undefined memorySpan?: number | undefined language?: string | undefined + rwt?: boolean | undefined } function settingsPatch(intent: SettingsIntent): SettingsPatch | null { @@ -45,6 +46,7 @@ function settingsPatch(intent: SettingsIntent): SettingsPatch | null { ...(intent.pet !== undefined && { tuiPet: intent.pet }), ...(intent.memorySpan !== undefined && { recentWindow: intent.memorySpan }), ...(intent.language !== undefined && { language: intent.language }), + ...(intent.rwt !== undefined && { rwtEnabled: intent.rwt }), } return Object.keys(patch).length === 0 ? null : patch } @@ -120,6 +122,10 @@ export function steerTools(actions: SteerActions, finish: (replyText: string) => 'the language to speak, as a name ("Japanese", "Traditional ' + 'Chinese"). Empty string returns it to its own default.', ), + rwt: z + .boolean() + .optional() + .describe('whether the host brings up real-world news and happenings at all'), }, async (args) => { // The pane greys the music items when this run has no pipeline; the diff --git a/test/app.test.ts b/test/app.test.ts index 29038c6..65fd81d 100644 --- a/test/app.test.ts +++ b/test/app.test.ts @@ -18,6 +18,8 @@ import { buildMemory, ensureTuiDeps, buildPacing, + buildRwt, + rwtLanguage, buildSettingsStore, buildVoice, escalatingSigint, @@ -37,9 +39,12 @@ import { HostedVoice } from '../src/hosted-voice.ts' import { IpcHost } from '../src/ipc-host.ts' import { InProcessMemoryStore, PersistentMemoryStore } from '../src/memory.ts' import { LedgerScheduler } from '../src/scheduler.ts' +import { DEFAULT_RWT_POLICY } from '../src/prompts.ts' import { readSettingsFile } from '../src/settings.ts' import { StubVoice } from '../src/voice.ts' +import { FakeBrain, FakeHost } from './fakes.ts' + // A murmur home with nothing in it — so a stray real ~/.murmur/voice.json on // the developer's machine can never decide what these tests see. Every config // built here starts from one unless the test names its own. @@ -406,6 +411,45 @@ describe('memory wiring', () => { // spec 12 §2.4: one store per run, seeded from the merged config (flags/env // respected), persisting around the file's user-touched keys. +// spec 13 §3.5: language from where the host reads it, region from the system +// clock only, the policy file seeded so the listener can find it. +describe('real-world topics wiring (spec 13)', () => { + it('seeds the policy file and resolves the request at fetch time', async () => { + const home = emptyHome() + const c = config([], { MURMUR_HOME: home }) + const brain = new FakeBrain() + const rwt = buildRwt(c, brain, () => 'Japanese', new FakeHost()) + expect(existsSync(join(home, 'rwt-policy.md'))).toBe(true) + rwt.maybeRefresh() + await rwt.drain() + const req = brain.fetchRequests[0]! + expect(req.language).toBe('Japanese') + expect(req.timezone).toBe(Intl.DateTimeFormat().resolvedOptions().timeZone) + expect(req.today).toMatch(/^\d{4}-\d{2}-\d{2}$/) + expect(req.policy).toBe(DEFAULT_RWT_POLICY) + }) + + it('the gist language is the override, else what the persona says it speaks, else the persona itself', () => { + expect(rwtLanguage('Japanese', 'Always speak in Chinese (Mandarin).')).toBe('Japanese') + expect(rwtLanguage(undefined, 'Always speak in Chinese (Mandarin).')).toBe('Chinese (Mandarin)') + // A generated persona is written in the listener's language and never + // names it in English (spec 06 §2.2): the text itself is the answer. + const french = '# Brume\n\nTu es la voix de la nuit, douce et lente. Tu parles sans te presser.' + expect(rwtLanguage(undefined, french)).toMatch(/the language this is written in: "Tu es la voix de la nuit/) + }) + + it('a listener policy replaces the default wholesale', async () => { + const home = emptyHome() + const c = config([], { MURMUR_HOME: home }) + writeFileSync(join(home, 'rwt-policy.md'), '\nOnly cats.\n') + const brain = new FakeBrain() + const rwt = buildRwt(c, brain, () => 'English', new FakeHost()) + rwt.maybeRefresh() + await rwt.drain() + expect(brain.fetchRequests[0]!.policy).toBe('Only cats.') + }) +}) + describe('settings store wiring (spec 12)', () => { it('starts from the merged config and persists around the touched keys', () => { const home = emptyHome() @@ -417,6 +461,13 @@ describe('settings store wiring (spec 12)', () => { expect(readSettingsFile(join(home, 'settings.json'))).toEqual({ gapSeconds: 5, tuiPet: false }) }) + it('seeds the real-world-topics knob from the flag, and the file wins a flag-less boot (spec 13 §2.6)', () => { + const home = emptyHome() + expect(buildSettingsStore(config(['--no-rwt'], { MURMUR_HOME: home })).current().rwtEnabled).toBe(false) + writeFileSync(join(home, 'settings.json'), JSON.stringify({ rwtEnabled: false })) + expect(buildSettingsStore(config([], { MURMUR_HOME: home })).current().rwtEnabled).toBe(false) + }) + it('a persisted mute seeds the store without touching the voice provider', () => { const home = emptyHome() writeFileSync(join(home, 'settings.json'), JSON.stringify({ muted: true })) diff --git a/test/brain.test.ts b/test/brain.test.ts index b7e5b74..b3fdead 100644 --- a/test/brain.test.ts +++ b/test/brain.test.ts @@ -23,6 +23,17 @@ describe('StubBrain', () => { expect(reply).toContain('hi') }) + it('fetchTopics returns nothing, so a stub run never offers a topic (spec 13)', async () => { + const topics = await new StubBrain().fetchTopics({ + language: 'English', + timezone: 'UTC', + today: '2026-09-03', + avoid: [], + policy: '', + }) + expect(topics).toEqual([]) + }) + it('compactProfile is a no-op (offline chatter never rewrites the profile)', async () => { const brain = new StubBrain() const updated = await brain.compactProfile('who you are', [{ role: 'radio', text: 'x' }]) @@ -69,6 +80,15 @@ describe('agenticOptions', () => { expect(o.maxTurns).toBe(2) expect(o.skills).toEqual([]) }) + + // spec 13 §2.2: a task may name built-ins; they are bounded (tools) AND + // pre-approved (allowedTools), and nothing else appears. + it('a task with builtins gets exactly those, bounded and pre-approved', () => { + const server = { type: 'sdk', name: 'murmur' } as never + const o = agenticOptions('sys', 'model-x', server, ['mcp__murmur__submit_topics'], 12, ['WebSearch']) + expect(o.tools).toEqual(['WebSearch']) + expect(o.allowedTools).toEqual(['mcp__murmur__submit_topics', 'WebSearch']) + }) }) // --- guide harness (spec 03-03) ------------------------------------------- // diff --git a/test/config.test.ts b/test/config.test.ts index 77314f4..e97a21d 100644 --- a/test/config.test.ts +++ b/test/config.test.ts @@ -186,6 +186,35 @@ describe('pacing flags', () => { }) }) +// spec 13 §2.6: the knob is anchorsEnabled's shape (file < flag); the numbers +// are env-only, with the MURMUR_TTS_* warn-and-default posture. +describe('real-world topics config', () => { + it('defaults on; --no-rwt turns it off', () => { + expect(parseCli([], NO_ENV).config.rwtEnabled).toBe(true) + expect(parseCli(['--no-rwt'], NO_ENV).config.rwtEnabled).toBe(false) + }) + + it('places the pool under cache/ and the policy at the home root', () => { + const { config } = parseCli([], { MURMUR_HOME: '/tmp/mh' }) + expect(config.rwtPoolPath).toBe('/tmp/mh/cache/rwt.json') + expect(config.rwtPolicyPath).toBe('/tmp/mh/rwt-policy.md') + }) + + it('reads the roll and freshness numbers from env, and ignores a bad one with a warning', () => { + const { config } = parseCli( + [], + isolated({ MURMUR_RWT_P: '0.5', MURMUR_RWT_MIN_GAP: '2', MURMUR_RWT_MAX_GAP: '6', MURMUR_RWT_STALE_HOURS: '3', MURMUR_RWT_TTL_HOURS: '24' }), + ) + expect([config.rwtP, config.rwtMinGap, config.rwtMaxGap, config.rwtStaleHours, config.rwtTtlHours]).toEqual([ + 0.5, 2, 6, 3, 24, + ]) + const warn = vi.spyOn(console, 'warn').mockImplementation(() => {}) + expect(parseCli([], isolated({ MURMUR_RWT_P: 'often' })).config.rwtP).toBe(0.35) + expect(warn).toHaveBeenCalledWith(expect.stringContaining('MURMUR_RWT_P')) + warn.mockRestore() + }) +}) + // spec 03-01 §2.3: the pick policy is a file the listener owns, at the home // root; the listening-data key is a secret, so it only ever comes from env. describe('music discovery config', () => { diff --git a/test/director-rwt.test.ts b/test/director-rwt.test.ts new file mode 100644 index 0000000..f1c73d2 --- /dev/null +++ b/test/director-rwt.test.ts @@ -0,0 +1,162 @@ +// The Director's side of spec 13 (§2.4): one offer per ordinary talk batch, +// never on an anchor or coda, gated live by the knob; the refresh poked at +// every boundary and never awaited. +import { describe, expect, it } from 'vitest' + +import type { ActivitySensor } from '../src/activity.ts' +import { EveryNCadence } from '../src/cadence.ts' +import type { RwtTopic } from '../src/contracts.ts' +import { Director, type DirectorDeps } from '../src/director.ts' +import { InProcessMemoryStore } from '../src/memory.ts' +import type { AnchorId, Scheduler } from '../src/scheduler.ts' +import { + directorSettings, + FakeBrain, + FakeHost, + FakeMixingPlayer, + FakeTrackSource, + FakeVoice, + pickOf, + until, +} from './fakes.ts' + +class FakeRwt { + offers = 0 + refreshes = 0 + topic: RwtTopic | null = { + id: 'ab12', + title: 'Typhoon season opens early', + gist: 'The first storm came in a month ahead of the usual.', + category: 'news', + fetchedAt: 0, + used: false, + } + + offer(): RwtTopic | null { + this.offers++ + return this.topic + } + + maybeRefresh(): boolean { + this.refreshes++ + return false + } +} + +// Presence, pinned: the pacing block needs a sensor to exist at all. +const presentSensor: ActivitySensor = { + state: () => 'present', + idleMs: () => 0, + noteInput: () => {}, +} + +class FakeScheduler implements Scheduler { + pending: AnchorId | null + constructor(pending: AnchorId | null) { + this.pending = pending + } + due(_now: Date): AnchorId | null { + return this.pending + } + markFired(_id: AnchorId, _now: Date): void { + this.pending = null + } +} + +function build(over: Partial & { rwtEnabled?: boolean; player?: FakeMixingPlayer } = {}) { + const { rwtEnabled = true, player = new FakeMixingPlayer(), ...rest } = over + const brain = new FakeBrain() + const host = new FakeHost() + const rwt = new FakeRwt() + const knobs = directorSettings({ gapSeconds: 0, rwtEnabled }) + const deps: DirectorDeps = { + persona: 'p', + brain, + voice: new FakeVoice(), + player, + memory: new InProcessMemoryStore(), + host, + settings: () => knobs, + openUrl: () => {}, + rwt, + ...rest, + } + return { brain, host, player, rwt, knobs, director: new Director(deps) } +} + +describe('real-world topics on the talk path (spec 13 §2.4)', () => { + it('offers once per ordinary batch and the pack carries the item', async () => { + const { brain, rwt, director } = build() + // Enough batches that no call fails: a retried batch is the same batch + // and rolls once, so offers and calls only match while nothing retries. + brain.batches = [['a', 'b'], ['c', 'd'], ['e', 'f'], ['g', 'h']] + await director.run(3) + await until(() => brain.nextTalksCalls >= 2, 'refill fired') + expect(rwt.offers).toBe(brain.nextTalksCalls) + for (const ctx of brain.talkContexts) { + expect(ctx.rwt).toEqual({ + title: 'Typhoon season opens early', + gist: 'The first storm came in a month ahead of the usual.', + }) + } + }) + + it('a null offer leaves the pack without the field', async () => { + const { brain, rwt, director } = build() + rwt.topic = null + brain.batches = [['a', 'b'], ['c', 'd']] + await director.run(1) + expect(rwt.offers).toBeGreaterThanOrEqual(1) + for (const ctx of brain.talkContexts) expect(ctx.rwt).toBeUndefined() + }) + + it('the knob off means no roll at all', async () => { + const { brain, rwt, director } = build({ rwtEnabled: false }) + brain.batches = [['a', 'b']] + await director.run(1) + expect(rwt.offers).toBe(0) + expect(brain.talkContexts[0]!.rwt).toBeUndefined() + }) + + it('an anchor beat is never offered one', async () => { + const { brain, rwt, director } = build({ + pacing: { sensor: presentSensor, scheduler: new FakeScheduler('morning'), gating: false }, + }) + brain.batches = [['good morning'], ['after', 'that']] + await director.run(1) + expect(brain.talkContexts[0]!.cue).toBe('anchor:morning') + expect(brain.talkContexts[0]!.rwt).toBeUndefined() + // The refill the anchor kicks off IS an ordinary batch; only that rolls. + expect(rwt.offers).toBe(brain.talkContexts.filter((c) => c.cue === undefined).length) + }) + + it('the coda is never offered one', async () => { + const source = new FakeTrackSource() + source.picks = [pickOf('https://stream/s1')] + const engine = new FakeMixingPlayer() + const { brain, player, rwt, director } = build({ + player: engine, + music: { source, cadence: new EveryNCadence(1), engine }, + }) + brain.batches = [['talk one', 'talk two'], ['three', 'four']] + brain.cueBeats = { coda: ['the coda'] } + const run = director.run(2) + await until(() => player.handles.length === 1, 'song on air') + await until(() => brain.talkContexts.some((c) => c.cue === 'coda'), 'coda requested') + const offersBeforeEnd = rwt.offers + player.handles[0]!.end() + await run + const coda = brain.talkContexts.find((c) => c.cue === 'coda')! + expect(coda.rwt).toBeUndefined() + // every offer came from an ordinary batch + expect(rwt.offers).toBe(brain.talkContexts.filter((c) => c.cue === undefined).length) + expect(offersBeforeEnd).toBeLessThanOrEqual(rwt.offers) + }) + + it('the refresh is poked at every boundary', async () => { + const { brain, rwt, director } = build() + brain.batches = [['a', 'b', 'c']] + await director.run(3) + expect(rwt.refreshes).toBeGreaterThanOrEqual(3) + }) +}) diff --git a/test/fakes.ts b/test/fakes.ts index c848940..d0b92ee 100644 --- a/test/fakes.ts +++ b/test/fakes.ts @@ -7,6 +7,8 @@ import type { AudioClip, Brain, ContextPack, + FetchedTopic, + FetchTopicsRequest, Harness, MixingPlayer, MusicContext, @@ -30,7 +32,7 @@ import { LineQueue } from '../src/host.ts' // The Director's live-settings thunk (spec 12 §3.2), test defaults. Mutate the // returned object to exercise hot application. export function directorSettings(over: Partial = {}): DirectorSettings { - return { gapSeconds: 0, recentWindow: 12, anchorsEnabled: true, musicEnabled: true, ...over } + return { gapSeconds: 0, recentWindow: 12, anchorsEnabled: true, musicEnabled: true, rwtEnabled: true, ...over } } // Stands in for the model driving an agentic task: `play` is handed the task's @@ -107,6 +109,15 @@ export class FakeBrain implements Brain { failRespond = false seedAnswers: (readonly SeedAnswer[])[] = [] + // Scripted real-world items (spec 13); empty by default, like the stub. + topics: FetchedTopic[] = [] + fetchRequests: FetchTopicsRequest[] = [] + + async fetchTopics(req: FetchTopicsRequest): Promise { + this.fetchRequests.push(req) + return this.topics + } + async nextTalks(ctx: ContextPack, _count: number): Promise { this.nextTalksCalls++ this.talkContexts.push(ctx) diff --git a/test/ipc-host.test.ts b/test/ipc-host.test.ts index 7a94be6..89e466b 100644 --- a/test/ipc-host.test.ts +++ b/test/ipc-host.test.ts @@ -638,6 +638,7 @@ describe('IpcHost (spec 10 §2.1/§2.3)', () => { recentWindow: 12, muted: false, tuiPet: true, + rwtEnabled: true, } function wire(applyOk = true): SettingsPatch[] { diff --git a/test/ipc.test.ts b/test/ipc.test.ts index 0182ce7..8c51b6f 100644 --- a/test/ipc.test.ts +++ b/test/ipc.test.ts @@ -43,6 +43,7 @@ const ENGINE_MESSAGES: EngineMessage[] = [ recentWindow: 12, muted: true, tuiPet: true, + rwtEnabled: true, }, home: '/home/someone/.murmur', voiceConfigured: true, @@ -60,6 +61,7 @@ const ENGINE_MESSAGES: EngineMessage[] = [ recentWindow: 4, muted: false, tuiPet: false, + rwtEnabled: true, }, home: '/tmp/m', voiceConfigured: false, diff --git a/test/persona.test.ts b/test/persona.test.ts index bc03904..42886b7 100644 --- a/test/persona.test.ts +++ b/test/persona.test.ts @@ -4,7 +4,7 @@ import { join } from 'node:path' import { describe, expect, it } from 'vitest' -import { loadPersona, personaLine, renderPersona } from '../src/persona.ts' +import { loadPersona, personaLanguage, personaLine, renderPersona } from '../src/persona.ts' import { DEFAULT_PERSONA_PATH } from '../src/prompts.ts' describe('loadPersona', () => { @@ -76,3 +76,20 @@ describe('personaLine', () => { expect(personaLine('#')).toBe('(empty)') }) }) + +// spec 13 §3.5: the spoken language lives in the persona's own words once the +// install is past its first run — the machine locale may have changed since. +describe('personaLanguage', () => { + it('reads the language the persona says it speaks', () => { + expect(personaLanguage('# x\n- **Always speak in Chinese (Mandarin).** Natural and spoken.')).toBe( + 'Chinese (Mandarin)', + ) + expect(personaLanguage('You are Ame. Speak in Japanese, softly.')).toBe('Japanese') + }) + + it('is undefined when the persona never names one, and a manner is not a language', () => { + expect(personaLanguage('You are the host. Keep it warm.')).toBeUndefined() + expect(personaLanguage('Speak in a warm tone, never rushed.')).toBeUndefined() + expect(personaLanguage(renderPersona('speak in {{language}}', 'English'))).toBe('English') + }) +}) diff --git a/test/prompts.test.ts b/test/prompts.test.ts index 81b0d44..79f3cd5 100644 --- a/test/prompts.test.ts +++ b/test/prompts.test.ts @@ -30,6 +30,10 @@ import { STATUS_MICROCOPY, statusMicrocopy, withLanguage, + buildFetchTopicsPrompt, + DEFAULT_RWT_POLICY, + RWT_FETCH_SYSTEM_PROMPT, + RWT_POLICY_HEADER, } from '../src/prompts.ts' const ctx = (recent: ContextPack['recent']): ContextPack => ({ persona: 'p', recent }) @@ -863,3 +867,92 @@ describe('the persona a setup conversation runs under', () => { } }) }) + +// --- real-world topics (spec 13) ------------------------------------------ // + +describe('rwt rendering (spec 13 §2.5)', () => { + const base = { persona: 'p', recent: [] } + const rwt = { title: 'Typhoon season opens early', gist: 'The first storm came in a month ahead of the usual.' } + + // The line is drawn on register, never on content: a host names the thing + // and carries on; a newsreader reads a rundown. Scrubbing the names out is + // the #44 attractor again, so the prompt must ask for them, not forbid them. + it('renders the item on the desk with the host-not-newsreader usage, on both talk builders', () => { + for (const p of [buildNextTalkPrompt({ ...base, rwt }), buildNextTalksPrompt({ ...base, rwt }, 2)]) { + expect(p).toContain('Typhoon season opens early') + expect(p).toContain('a month ahead of the usual') + expect(p).toMatch(/name the thing — the title, who, where, when/) + expect(p).toMatch(/not a newsreader's rundown/i) + expect(p).toMatch(/not a list/i) + expect(p).not.toMatch(/never a headline/i) + expect(p).not.toMatch(/leave it/i) + } + }) + + it('renders nothing without an item', () => { + expect(buildNextTalkPrompt(base)).not.toMatch(/on the desk for this stretch/i) + }) + + it('never rides an anchor or coda beat, even if the pack carries one', () => { + for (const cue of ['anchor:morning', CODA_CUE]) { + const p = buildNextTalkPrompt({ ...base, rwt, cue }) + expect(p).not.toContain('Typhoon season opens early') + } + }) + + it('the reply path does not carry it — a reply answers the listener', () => { + expect(buildRespondPrompt('hey', { ...base, rwt })).not.toContain('Typhoon season opens early') + }) +}) + +describe('the fetch prompt (spec 13 §3.3)', () => { + const base = { persona: 'p', recent: [] } + const req = { + language: 'Japanese', + timezone: 'Asia/Tokyo', + today: '2026-09-03', + avoid: ['Already held', 'Also held'], + policy: 'Only cats.', + } + + it('is a neutral researcher framing, not the persona', () => { + expect(RWT_FETCH_SYSTEM_PROMPT).toMatch(/material for a radio host/i) + expect(RWT_FETCH_SYSTEM_PROMPT).not.toMatch(/you are the host/i) + }) + + it('states language, timezone, freshness, the held titles, privacy, and the terminal call', () => { + const p = buildFetchTopicsPrompt(req) + // the title reaches the host beside the gist, so it is in the language too + expect(p).toMatch(/every title and every gist in Japanese/) + expect(p).toContain('Asia/Tokyo') + expect(p).toContain('2026-09-03') + expect(p).toMatch(/today or yesterday/i) + expect(p).toContain('- Already held') + expect(p).toContain('- Also held') + expect(p).toMatch(/private/i) + expect(p).toContain('submit_topics') + expect(p).toContain(`${RWT_POLICY_HEADER}\nOnly cats.`) + }) + + it('an empty avoid list renders no list', () => { + expect(buildFetchTopicsPrompt({ ...req, avoid: [] })).not.toMatch(/already in the pool/i) + }) + + it('the default policy names the four categories and the local weighting', () => { + for (const word of ['news', 'tech', 'entertainment', 'sports']) { + expect(DEFAULT_RWT_POLICY.toLowerCase()).toContain(word) + } + expect(DEFAULT_RWT_POLICY).toMatch(/hard nouns/i) + expect(buildFetchTopicsPrompt({ ...req, policy: DEFAULT_RWT_POLICY })).toContain(DEFAULT_RWT_POLICY) + }) + + it('the steer settings rule names the knob so "stop with the news" is a settings ask', () => { + const p = buildSteerPrompt('hey', base, { + musicWired: false, + shutdownArmed: false, + settingsWired: true, + memoryWired: false, + }) + expect(p).toMatch(/real-world|news/i) + }) +}) diff --git a/test/rwt.test.ts b/test/rwt.test.ts new file mode 100644 index 0000000..5c5c828 --- /dev/null +++ b/test/rwt.test.ts @@ -0,0 +1,244 @@ +// The real-world topic pool (spec 13 §2.1), the roll (§2.3) and the off-loop +// refresh (§3.1): file-backed, expiring, single-flight, and never on the talk +// path. + +import { mkdtempSync, readFileSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' + +import { describe, expect, it } from 'vitest' + +import type { FetchedTopic, FetchTopicsRequest } from '../src/contracts.ts' +import { RWT_FETCH_SYSTEM_PROMPT } from '../src/prompts.ts' +import { fetchTopicsTask, RealWorldTopics, RwtPool, RwtRoll } from '../src/rwt.ts' +import { callTool, FakeHarness, until } from './fakes.ts' + +const HOUR = 3600 +const topic = (title: string, over: Partial = {}): FetchedTopic => ({ + title, + gist: `${title} happened.`, + category: 'news', + ...over, +}) + +function poolAt(clock: { now: number }, over: { ttlHours?: number; staleHours?: number } = {}) { + const path = join(mkdtempSync(join(tmpdir(), 'murmur-rwt-')), 'rwt.json') + return { path, pool: new RwtPool({ path, now: () => clock.now, ...over }) } +} + +describe('RwtPool (spec 13 §2.1)', () => { + it('starts empty on a missing file and is due for a refresh', () => { + const { pool } = poolAt({ now: 1000 }) + expect(pool.counts()).toEqual({ fresh: 0, used: 0 }) + expect(pool.refreshDue()).toBe(true) + expect(pool.take()).toBeNull() + }) + + it('merge persists the entries and stamps the refresh; take marks used and persists', () => { + const clock = { now: 10 * HOUR } + const { path, pool } = poolAt(clock) + expect(pool.merge([topic('A'), topic('B')])).toBe(2) + expect(pool.refreshDue()).toBe(false) + expect(pool.counts()).toEqual({ fresh: 2, used: 0 }) + + const first = pool.take() + expect(first?.title).toBe('A') + const second = pool.take() + expect(second?.title).toBe('B') + expect(second?.id).not.toBe(first?.id) + expect(pool.take()).toBeNull() + expect(pool.counts()).toEqual({ fresh: 0, used: 2 }) + + // A second pool over the same file sees the same state — nothing is only + // in memory. + const again = new RwtPool({ path, now: () => clock.now }) + expect(again.counts()).toEqual({ fresh: 0, used: 2 }) + expect(again.take()).toBeNull() + expect(again.refreshDue()).toBe(false) + }) + + it('an entry older than ttl drops on load; the pool goes stale after staleHours', () => { + const clock = { now: 100 * HOUR } + const { path, pool } = poolAt(clock, { ttlHours: 48, staleHours: 6 }) + pool.merge([topic('old')]) + clock.now += 5 * HOUR + expect(pool.refreshDue()).toBe(false) + clock.now += 2 * HOUR + expect(pool.refreshDue()).toBe(true) + expect(pool.counts().fresh).toBe(1) // stale is not expired + clock.now += 48 * HOUR + const reloaded = new RwtPool({ path, now: () => clock.now, ttlHours: 48 }) + expect(reloaded.counts()).toEqual({ fresh: 0, used: 0 }) + }) + + it('merge skips titles already in the pool and lists them for the fetch to avoid', () => { + const { pool } = poolAt({ now: 1000 }) + pool.merge([topic('A')]) + expect(pool.merge([topic('A'), topic('B')])).toBe(1) + expect(pool.titles()).toEqual(['A', 'B']) + }) + + it('an unwritable cache never throws on the talk path: the take stands in memory', () => { + const { pool } = poolAt({ now: 1000 }) + pool.merge([topic('A'), topic('B')]) + const dir = mkdtempSync(join(tmpdir(), 'murmur-rwt-ro-')) + const stuck = new RwtPool({ path: dir, now: () => 1000 }) // a directory: every write fails + expect(stuck.merge([topic('A')])).toBe(1) + expect(stuck.take()?.title).toBe('A') + expect(stuck.take()).toBeNull() + }) + + it('an expired entry does not block a recurring title from coming back', () => { + const clock = { now: 100 * HOUR } + const { pool } = poolAt(clock, { ttlHours: 48 }) + pool.merge([topic('A')]) + clock.now += 49 * HOUR // the process outlived the entry; nothing reloaded + expect(pool.merge([topic('A')])).toBe(1) + expect(pool.counts()).toEqual({ fresh: 1, used: 0 }) + }) + + it('a malformed file is an empty pool, not a boot failure', () => { + const { path } = poolAt({ now: 1000 }) + writeFileSync(path, '{not json') + const pool = new RwtPool({ path, now: () => 1000 }) + expect(pool.counts()).toEqual({ fresh: 0, used: 0 }) + pool.merge([topic('A')]) + expect(JSON.parse(readFileSync(path, 'utf-8')).entries).toHaveLength(1) + }) +}) + +describe('RwtRoll (spec 13 §2.3)', () => { + it('holds off until minGap and forces an offer at maxGap', () => { + const roll = new RwtRoll({ p: 1, minGap: 2, maxGap: 4, random: () => 0 }) + expect(roll.roll()).toBe(false) // 1 batch since the last offer + expect(roll.roll()).toBe(true) // 2 — p wins + const never = new RwtRoll({ p: 0, minGap: 1, maxGap: 3, random: () => 0.99 }) + expect(never.roll()).toBe(false) + expect(never.roll()).toBe(false) + expect(never.roll()).toBe(true) // 3 — guardrail + expect(never.roll()).toBe(false) // the counter reset + }) + + it('uses the injected RNG against p in between', () => { + const rolls = [0.1, 0.9] + const roll = new RwtRoll({ p: 0.35, minGap: 1, maxGap: 9, random: () => rolls.shift()! }) + expect(roll.roll()).toBe(true) + expect(roll.roll()).toBe(false) + }) +}) + +function feed(over: { + fetch?: (req: FetchTopicsRequest) => Promise + roll?: RwtRoll + clock?: { now: number } +} = {}) { + const clock = over.clock ?? { now: 1000 } + const { pool } = poolAt(clock) + const requests: FetchTopicsRequest[] = [] + const lines: string[] = [] + const rwt = new RealWorldTopics({ + pool, + roll: over.roll ?? new RwtRoll({ p: 1, minGap: 0, maxGap: 1, random: () => 0 }), + brain: { + fetchTopics: async (req) => { + requests.push(req) + return over.fetch === undefined ? [topic('A'), topic('B')] : over.fetch(req) + }, + }, + request: () => ({ language: 'Japanese', timezone: 'Asia/Tokyo', today: '2026-09-03', policy: 'news' }), + log: (m) => lines.push(m), + }) + return { pool, rwt, requests, lines, clock } +} + +describe('RealWorldTopics (spec 13 §2.4 / §3.1)', () => { + it('refreshes once in the background when due, single-flight, and merges the result', async () => { + const { pool, rwt, requests, lines } = feed() + expect(rwt.maybeRefresh()).toBe(true) + expect(rwt.maybeRefresh()).toBe(false) // in flight + await rwt.drain() + expect(requests).toHaveLength(1) + expect(requests[0]?.avoid).toEqual([]) + expect(requests[0]?.language).toBe('Japanese') + expect(pool.counts()).toEqual({ fresh: 2, used: 0 }) + expect(rwt.maybeRefresh()).toBe(false) // no longer due + expect(lines.some((l) => /^rwt\.refresh n=2 ms=\d+$/.test(l))).toBe(true) + expect(lines.some((l) => l === 'rwt.pool fresh=2 used=0')).toBe(true) + }) + + it('offers only when the roll lands and the pool has an unused entry', async () => { + const rolls = [0.99, 0.0] + const { rwt, lines } = feed({ roll: new RwtRoll({ p: 0.5, minGap: 0, maxGap: 9, random: () => rolls.shift()! }) }) + expect(rwt.offer()).toBeNull() // pool empty: the roll is not even consumed + rwt.maybeRefresh() + await rwt.drain() + expect(rwt.offer()).toBeNull() // 0.99 > p + const offered = rwt.offer() + expect(offered?.title).toBe('A') + expect(lines).toContain(`rwt.offer ${offered?.id}`) + }) + + it('a failed fetch costs one log line and leaves the pool as it was', async () => { + const { pool, rwt, lines } = feed({ fetch: async () => Promise.reject(new Error('offline')) }) + rwt.maybeRefresh() + await rwt.drain() + expect(pool.counts()).toEqual({ fresh: 0, used: 0 }) + expect(lines).toContain('rwt.refresh failed (Error: offline)') + expect(rwt.maybeRefresh()).toBe(true) // still due, retried + }) + + it('the fetch is told what is already in the pool', async () => { + const { rwt, requests, clock } = feed() + rwt.maybeRefresh() + await rwt.drain() + clock.now += 7 * HOUR + rwt.maybeRefresh() + await until(() => requests.length === 2) + expect(requests[1]?.avoid).toEqual(['A', 'B']) + }) +}) + +// The fetch task itself (spec 13 §2.2): WebSearch bounded in, one terminal +// tool out, the researcher framing. Played through the fake harness so the +// termination rule and the schema are exercised with no network. +describe('fetchTopicsTask (spec 13 §2.2)', () => { + const req: FetchTopicsRequest = { + language: 'Japanese', + timezone: 'Asia/Tokyo', + today: '2026-09-03', + avoid: [], + policy: 'news', + } + + it('names WebSearch as its one built-in, bounds the turns, and frames neutrally', async () => { + const harness = new FakeHarness() + await harness.runTask(fetchTopicsTask(req, 'model-x')) + const task = harness.lastTask! + expect(task.builtins).toEqual(['WebSearch']) + expect(task.maxTurns).toBe(12) + expect(task.model).toBe('model-x') + expect(task.systemPrompt).toBe(RWT_FETCH_SYSTEM_PROMPT) + expect(task.prompt).toContain('Asia/Tokyo') + expect(task.tools(() => {}).map((t) => t.name)).toEqual(['submit_topics']) + }) + + it('submit_topics finishes the task with the cleaned items', async () => { + const harness = new FakeHarness(async (tools) => { + await callTool(tools, 'submit_topics', { + topics: [ + { title: ' A ', gist: ' a happened ', category: 'news' }, + { title: '', gist: 'nothing', category: 'news' }, + ], + }) + }) + const got = await harness.runTask(fetchTopicsTask(req, 'model-x')) + expect(got).toEqual([{ title: 'A', gist: 'a happened', category: 'news' }]) + }) + + it('a call with nothing usable does not finish the task', async () => { + const harness = new FakeHarness(async (tools) => { + await callTool(tools, 'submit_topics', { topics: [{ title: '', gist: '', category: '' }] }) + }) + expect(await harness.runTask(fetchTopicsTask(req, 'model-x'))).toBeNull() + }) +}) diff --git a/test/settings.test.ts b/test/settings.test.ts index b1a657e..171f279 100644 --- a/test/settings.test.ts +++ b/test/settings.test.ts @@ -19,6 +19,7 @@ const BASE: Settings = { recentWindow: 12, muted: false, tuiPet: true, + rwtEnabled: true, } const home = () => mkdtempSync(join(tmpdir(), 'murmur-settings-')) diff --git a/test/steer-tools.test.ts b/test/steer-tools.test.ts index 66cf0e7..519b444 100644 --- a/test/steer-tools.test.ts +++ b/test/steer-tools.test.ts @@ -23,6 +23,7 @@ const BASE: Settings = { recentWindow: 12, muted: false, tuiPet: true, + rwtEnabled: true, } function harness(initial: Partial = {}, wired: { music?: boolean } = {}) { @@ -68,6 +69,10 @@ describe('change_settings (spec 12 §2.6)', () => { expect((await call({ anchors: false, pet: false })).ok).toBe(true) expect(store.current().anchorsEnabled).toBe(false) expect(store.current().tuiPet).toBe(false) + + // "stop with the news" (spec 13 §2.6) + expect((await call({ rwt: false })).ok).toBe(true) + expect(store.current().rwtEnabled).toBe(false) }) it('translates the mix gear the way the pane does, never raw field names', async () => { diff --git a/test/tui-settings.test.ts b/test/tui-settings.test.ts index 062523f..49f2d0f 100644 --- a/test/tui-settings.test.ts +++ b/test/tui-settings.test.ts @@ -17,6 +17,7 @@ const VALUES: Settings = { recentWindow: 12, muted: false, tuiPet: true, + rwtEnabled: true, } const snap = (