Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,11 @@ The local database dashboard (Supabase Studio) is at http://127.0.0.1:57323. Bef

### Replayed real answers

`npm run dev:replay` shows the interface with the real model's answers, for free and without a token. It replays answers the measurement recorded on gpt-oss-120b ([`replays.json`](src/server/ai/replays.json)), each reply whole, after the time the real model took. The intro's button, the grocery delivery suggestion and the other measured requests replay; any other message gets the scripted fake's answer, at the same pace. Searches and saves still run on your machine, so the search lines show local passages, while the words come from the recording. The header says "Replaying recorded answers". To refresh the copy after a new measurement, run `npm run replays:copy -- --from docs/records/measurements/<name>` where the records are.
`npm run dev:replay` shows the interface with the real model's answers, for free and without a token. It replays answers the measurement recorded on gpt-oss-120b ([`replays.json`](src/server/ai/replays.json)), each reply whole, after the time the real model took. The intro's button, the grocery delivery suggestion and the other measured requests replay; any other message gets the scripted fake's answer, at the same pace. Searches and saves still run on your machine, so the search lines show local passages, while the words come from the recording. The header says "Replaying recorded answers".

A **scenario tour** in the corner of the page walks through every state the chat and the PRD can show, which recorded answers alone never reach: rich text and tables, folded and refused searches, a section with every kind of statement, a long wait to stop, each kind of failure and, last, the used-up daily allowance. Each step says what to watch for; "Play next" sends it. The scripted answers run through the same chat code, so their searches, saves and quote checks are real.

To refresh the copy after a new measurement, run `npm run replays:copy -- --from docs/records/measurements/<name>` where the records are.

### The real chat model on your machine

Expand Down
14 changes: 11 additions & 3 deletions docs/plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,12 @@

### Build lane
- **Where:** its own worktree per piece, on a branch from an up-to-date `main`. If the session isn't in a worktree yet (`git worktree list` shows its folder as the main checkout, not under `.claude/worktrees/`), it creates one before touching anything, and never checks out a branch in the main folder, where the measurement lane works. Claude Code's worktree option puts it under `.claude/worktrees/`; run `npm ci` there first. A worktree made with `git worktree add` gets no `.env.local`: run `npm run env:local` there, or the end-to-end tests find no interviews. Its browser check uses `app-build` in `.claude/launch.json` (port 3200), never `app` or `app-real`, which are the measurement lane's on port 3000. It never runs `measure`, `measure:fake` or `dev:real`, never edits the measurement lane's files above, and changes code in `src/server` only where its piece needs it, never what the model receives (listed above); its pull request says which `src/server` files it changed and why.
- **Doing:** slice 23, the README rewritten for reviewers, is done on `docs/readme-for-reviewers` (`.claude/worktrees/readme`), from `main` at 6e1dae5 (slice 22's merge), waiting for the owner's push and merge. It opens on the demo's story (the owner's wish, 2026-10-21: a story, not terms such as "citation check" before the reader knows what the app does), then what's worth a look, the status and how it works; running it, the commands and the database follow, and the deploy steps fold into two collapsed sections. The facts were checked against the code at 6e1dae5; "How it works" keeps its heading (`tests/unit/readme.test.ts`), and so does "The real chat model on your machine" (`src/lib/env-guard.ts`). Prettier passed on it; nothing else ran, since only documents changed.
- **Doing:** slice 24, the scenario tour, is done on `feat/scenario-tour` (`.claude/worktrees/slice-22-replay`), from `main` at e4bd98e (slice 23's merge), waiting for the owner's look, push and merge. The owner asked for it on 2026-10-21, after finding that the replay never shows a table, among others, and chose all four groups of states, played in one sequence from a menu in the replay. Local checks on 2026-10-21: `verify` (527 unit tests) and `test:e2e` (69, of which 1 new in `tests/e2e/replay.spec.ts`) passed in one full run each. All 17 steps were played from the panel with `npm run dev:replay` at `REPLAY_SPEED=4` from the worktree on port 3200, read through the page's text (the Browser pane was hidden, so no screenshots): the tables, the rich text, the folded and refused searches, the section of every kind (8 of 13 quotes confirmed, as scripted), the refused save, Stop during the long wait, each failure with "Try again", the busy message and the quota notice. "Breaking off partway" first lost its words, fixed before the commit. Not seen: Jump to latest during the long answer, and the wide table's sideways scroll, which a hidden pane doesn't measure. It adds `src/server/ai/scenarios.ts` and `src/lib/scenarios.ts` and changes `src/server/ai/replay-chat-model.ts`, all for `dev:replay` only; nothing the real model receives changes, so the measurement lane needn't wait for it, though its comparison will list those files as code.
- **Next:**
1. A short GIF near the README's top, from `npm run dev:replay`, once the owner has looked at the replay themselves (asked 2026-10-21). The screen recording for launch stays (`docs/decisions.md`, 2026-10-17).
2. Slice 8, the rate limit, once 9d is done and the owner has decided its legal basis and size (*Deferred*).
- **Unverified:** on staging, a reload keeping the PRD and the downloaded file (local Chromium only); a stop ending the answer on Cloudflare too, where the call in flight may still be finished and counted; the quota message against Cloudflare's real refusal (the tests replay its reply of 2026-10-20) and whether its per-minute 429 carries a retry header; a source name's pop-over, "Your decision, not from the interviews." under a claim of the visitor's, the table fix and the intro screen; Safari and Firefox (only Chromium ran; without anchor positioning the pop-over is a centred box, by design, and without `field-sizing`, before Safari 26.2 and Firefox 152, the message box keeps its three lines; the moving band, whose fallback is the still grey line; the PRD's highlight and rolling count; Jump to latest; the folded line's slide, which falls back to showing at once; the words' fade; the copy button, its label and the clipboard; Inter and its resized fallback, which needs Arial installed, so Linux and Android show the system's font until Inter arrives; a table's cells keeping their 144px and its box's sideways scroll; the Thinking line's band). On staging, a real reply's table, and the Thinking line through the real model's waits with the times it shows (the fake model's are about 0s). On staging, a reply that arrives whole fading in as one, and a section's copy over HTTPS pasted into another app. Expect the Security Advisor's lint 0028 on `search_passages` on staging; it's intended (`supabase/AGENTS.md`). Production waits for launch and its Cloudflare token.
- **Waiting on the owner:** pushing `docs/readme-for-reviewers`, its pull request and the merge; a look at `npm run dev:replay` before the GIF; whether `next dev` may keep adding its block to `AGENTS.md` (KI-8); the GitHub description and topics whenever they like.
- **Waiting on the owner:** a look at `npm run dev:replay` and its tour, then pushing `feat/scenario-tour`, its pull request and the merge; the GIF after that; whether `next dev` may keep adding its block to `AGENTS.md` (KI-8); the GitHub description and topics whenever they like.

### Shared rules
- One branch per piece, from an up-to-date `main`; the owner pushes it, opens the pull request and merges it (`AGENTS.md`, Branches). After the other lane's merge, rebase an unpushed branch onto `main`, and merge `main` into a pushed one; never rewrite a pushed branch.
Expand Down Expand Up @@ -121,4 +121,12 @@

### 22 · Replayed real answers, to look at the interface here · risk: normal · status: done (6e1dae5)

### 23 · The README, for reviewers · risk: normal · status: done (on `docs/readme-for-reviewers`; its id on `main` goes here after the merge)
### 23 · The README, for reviewers · risk: normal · status: done (e4bd98e)

### 24 · A tour of every state the chat and the PRD can show · risk: normal · status: done (on `feat/scenario-tour`; its id on `main` goes here after the merge)
- **Goal:** the owner and the build lane can see, step by step, every state the chat and the PRD can show, at the real model's pace, for free and without the token: the replay shows only what the model happened to do in its recorded runs, and never a table (the owner, 2026-10-21).
- **Acceptance:**
- With `npm run dev:replay`, a small "Scenario tour" panel lists the steps in order, with what to watch for in each, which were played, and "Play next"; any step can be played on its own. It never shows in a build for staging or production, nor with `npm run dev`.
- Each step sends its own words as the visitor's message, and its answer is scripted like a recording: each model call's parts, whole, after a set wait. The searches, the saves and the check run here for real, on the local database and the fake embedder, as with the replay.
- The steps, in order: two real recorded answers (the first section, the grocery delivery claim); rich text (bold, code, lists, and marks the chat strips); tables (wide, line breaks in cells, empty cells, a row longer than the header); searches folding into one line with text between runs and a long query cut short; a refused search and a step that doesn't exist; one section with every kind of statement (confirmed, many sources, an unverified quote, the interviewer's question quoted, the wrong passage cited, an unknown source, citing nothing, the visitor's decision); a save refused for its form, then saved; Open Questions; a long answer for Jump to latest; a long wait for Stop; failing before the first word and partway, each once so "Try again" works; an answer cut off at its output limit; an answer with no words; the busy message; and last, the used-up allocation, which lasts until a reload.
- Unit tests: the scripted quotes meant to pass match the interviews and those meant to fail don't, a failure fails once, and the panel's next step and what it sends; an end-to-end check that the tables step's words, sent to the replay's server, show a table (that server runs a build, which never carries the panel). Nothing the real model receives changes; the pull request names the `src/server` and `src/lib` files it adds.
4 changes: 4 additions & 0 deletions src/features/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,10 @@
- **Its label** (`.copy-tip` in `src/app/globals.css`) is CSS, `aria-hidden`: shown on `.copy-button:hover` after 0.4s, closed by `data-pressed` from the press until the pointer leaves, held open by `data-open` to say "Couldn't copy".
- **After a copy** the name is "Copied" or "Couldn't copy" for `COPY_OUTCOME_MS`, and an `aria-live="polite"` span says it once, since screen readers don't reliably announce a focused button's new name. Not `role="status"`: the chat's status line is the page's one. `navigator.clipboard.writeText` rejects when refused (MDN, read 2026-10-21 through Context7), and a page that isn't a secure context has no `navigator.clipboard`; both read "Couldn't copy". The desktop app's Browser pane refuses the clipboard while it's hidden, so a successful copy is seen only in the end-to-end tests there (found 2026-10-21).

## The scenario tour (`workspace/scenario-tour.tsx`, slice 24)
- **Only with `npm run dev:replay`:** `Workspace` renders it when `NEXT_PUBLIC_CHAT_REPLAY` was "1" at compile time, as the header's "Replaying recorded answers", so no build for staging or production has it, and neither does the end-to-end tests' replay server, which runs that build. It's a `fixed` `<aside>` over the page's corner, beside the intro or the workspace, never in their layout, so the layout tests measure nothing new.
- **What it sends:** a step's words, through `playScenario`, which leaves the intro as its button does and clears a pending check-in. A step counts as played once its words are among the visitor's messages (`playedScenarios` in `src/lib/scenarios.ts`), so the tour survives a reload and "Start over" begins it again; "Play next" offers the step after the furthest one played. Both are disabled while an answer is on its way.

## The intro (`workspace/intro.tsx`, `workspace/disclosure.ts`)
- **What shows when:** the intro until the visitor leaves it or the conversation holds messages (`conversationStarted`, which holds for a restored conversation too); the suggestion to try to break it once a section is saved, until the visitor has sent it (`breakItSuggestion`). Its button and the suggestion import their words from `src/lib/suggestions.ts`, never a copy: the measurement sends the same words, and changing them means a new baseline.
- **A failed first request:** `useChat`'s `onFinish` runs after every answer, once the error status is set, with `isError` and the messages from the chat's own state (ai 7.0.143, `makeRequestImpl` in `node_modules/ai/dist/index.js`, read 2026-10-19; the AI SDK's `useChat` reference through Context7). `afterAnswer` brings the intro back only when the intro's request failed before a section was saved, and clears the error, so the chat never opens on the old one. The server can send an error after a save (a reply cut off at its output limit, or the provider failing on the last call); then the section stays, with the chat's "Try again". Don't decide this in `onError`: it runs before the status is set, and the rendered `messages` can lag behind the chat's state. A ref marks the intro's request, so a later failure stays in the chat.
Expand Down
117 changes: 117 additions & 0 deletions src/features/workspace/scenario-tour.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
"use client";

import { useId, useState } from "react";
import { Button } from "@/components/ui/button";
import { nextScenario, playedScenarios, SCENARIOS } from "@/lib/scenarios";
import { cn } from "@/lib/utils";

/**
* The scenario tour's panel (docs/plan.md, slice 24), shown only with `npm
* run dev:replay`: the steps in order, what to watch for in the next one, and
* "Play next", which sends its words as the visitor's message. A step counts
* as played once its words are in the conversation, so the tour picks up
* where it was after a reload. Any step can be played on its own from the
* list. It sits over the page's corner, never in its layout.
*/
export function ScenarioTour({
sent,
working,
onPlay,
}: {
/** The visitor's messages so far. */
sent: readonly string[];
working: boolean;
onPlay: (words: string) => void;
}) {
const [open, setOpen] = useState(true);
const [listOpen, setListOpen] = useState(false);
const panelId = useId();
const listId = useId();
const played = playedScenarios(sent);
const next = nextScenario(played);
const step = next === undefined ? undefined : SCENARIOS[next];
const count = SCENARIOS.length;

return (
<aside
aria-label="Scenario tour"
className="fixed right-4 bottom-4 z-50 w-80 max-w-[calc(100vw-2rem)] rounded-(--radius-control) border border-border-strong bg-background text-sm shadow-lg"
>
<button
type="button"
aria-expanded={open}
aria-controls={panelId}
onClick={() => setOpen((value) => !value)}
className="flex w-full items-center justify-between gap-3 px-4 py-2.5 text-left font-medium"
>
<span>Scenario tour</span>
<span className="text-muted-foreground tabular-nums">
{played.size} of {count} played
</span>
</button>
<div id={panelId} hidden={!open} className="space-y-3 px-4 pb-4">
{step === undefined ? (
<p className="text-muted-foreground">
The tour is done. Reload the page, then “Start over” begins it
again.
</p>
) : (
<div className="space-y-1.5">
<p className="text-muted-foreground tabular-nums">
Step {(next ?? 0) + 1} of {count}
</p>
<p className="font-medium">{step.title}</p>
<p className="text-muted-foreground">{step.watch}</p>
</div>
)}
<div className="flex items-center gap-4">
{step !== undefined && (
<Button
disabled={working}
onClick={() => onPlay(step.words)}
className="h-8 px-3"
>
Play next
</Button>
)}
<Button
variant="quiet"
aria-expanded={listOpen}
aria-controls={listId}
onClick={() => setListOpen((value) => !value)}
className="h-auto px-0"
>
All steps
</Button>
</div>
<ol
id={listId}
hidden={!listOpen}
className="max-h-64 space-y-0.5 overflow-y-auto"
>
{SCENARIOS.map((scenario, index) => (
<li key={scenario.id}>
<button
type="button"
disabled={working}
onClick={() => onPlay(scenario.words)}
className={cn(
"flex w-full gap-2 rounded-(--radius-control) px-2 py-1 text-left hover:bg-foreground/10 disabled:opacity-50",
index === next && "bg-foreground/10",
)}
>
<span className="w-5 shrink-0 text-muted-foreground tabular-nums">
{index + 1}
</span>
<span className="flex-1">{scenario.title}</span>
{played.has(scenario.id) && (
<span className="text-muted-foreground">played</span>
)}
</button>
</li>
))}
</ol>
</div>
</aside>
);
}
Loading
Loading