diff --git a/CLAUDE.md b/CLAUDE.md index 0f6954c5..c0a290fe 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -188,10 +188,15 @@ Re-anchoring after an edit is the reprocess action, superseding via reconcile. The plan-named golden cases gate it and the eval fixtures were refreshed Mistral-routed: [`docs/features/anchoring.md`](docs/features/anchoring.md). -Work proceeds through the V2 plan in order, with one owner-approved insertion: -reasoning-model support Part B (the probed `reasoning` capability and the -maxTokens headroom) landed 2026-08-04; Parts A (the thinking channel) and C -(storage and UI) follow after V2.1 item 4.2. +Work proceeds through the V2 plan in order, with one owner-approved insertion, +now complete: **reasoning-model support** (Parts A, B and C, 2026-08-04). +Thinking is a CHANNEL, not content: `completeStream` yields channel-tagged +deltas, the budget charges thinking, redaction strips it fail-closed, chat +stores it (`chat_message.thinking`, migration 0044) and shows it as a +collapsed live disclosure, the answer-redaction cascade erases it with its +answer, and the trust-artifact id gains a probed `--reasoning` marker at +emission time only. Never captured, cited, verified, or evaluated: +[`docs/features/reasoning.md`](docs/features/reasoning.md). ## Delivery loop diff --git a/docs/features/capabilities.md b/docs/features/capabilities.md index 035463e1..354f9204 100644 --- a/docs/features/capabilities.md +++ b/docs/features/capabilities.md @@ -65,10 +65,11 @@ The probe reads the reasoning field only as a yes/no. The thinking text is discarded in the adapter: it is never stored, verified, cited, displayed or evaluated, and it can never reach the JSON parser behind structured extraction. Displaying it as a channel is Parts A and C of the reasoning -design, deliberately not this. The configuration fingerprint does not yet -carry a reasoning marker for the same reason: whether a binding reasons is a -probed runtime fact, and the fingerprint is derived before any probe can run; -the marker lands with the channel in Part C. +design, deliberately not this. The configuration fingerprint carries the +reasoning marker at EMISSION time (Parts A and C): whether a binding reasons +is a probed runtime fact the static resolver cannot know, so the trust-artifact +id gains `--reasoning` from the same probe when a measurement is emitted, and +never anywhere else. Full narrative: [`reasoning.md`](reasoning.md). ## Thresholds diff --git a/docs/features/reasoning.md b/docs/features/reasoning.md new file mode 100644 index 00000000..23935fcf --- /dev/null +++ b/docs/features/reasoning.md @@ -0,0 +1,103 @@ +# Reasoning models: the thinking channel + +**Owner-approved design, delivered in three parts: B (2026-08-04, the probed +capability and the maxTokens headroom), then A and C together (the channel, +its storage, and the display). Migration 0044.** + +A reasoning model deliberates before it answers, and serves that deliberation +as a separate stream beside the answer. Cogeto supports this first-class +rather than working around it, under one governing principle: **thinking is a +CHANNEL, not content.** It is displayed, live and afterwards, because hiding +what the instance's own model said while deciding would be the opposite of +the product's posture; and it is never a source, never a citation, never a +measurement. + +## The three honesty rules + +1. **Thinking is never a source.** "Remember this" captures the user's own + words; capture reads user rows only, structurally. Thinking cannot be + captured, cited, verified, or contradicted, and the answer sanitizer and + citation machinery read `content` alone. +2. **Thinking is never evaluated.** The golden set and the trust artifact + measure answers. The eval cache records the text channel only, so a + fixture cannot freeze what the harness must not measure. +3. **A run with thinking on is a different measurement.** The trust-artifact + configuration id carries a `--reasoning` marker, appended at EMISSION time + from the same probe the capability panel uses, because whether a binding + reasons is a runtime fact the static resolver cannot know. A Mistral-routed + run probes off and emits the unchanged id, so every existing artifact, + gate, and cached fixture is untouched. + +## Part B: the probed capability (delivered first, separately) + +Whether a configuration reasons cannot be read off a model name: the same +weights are served both ways. So it is probed, like vision: a trivial prompt +at boot and per registry window, the answer surfaced on the capability panel +and boot banner. When on, every `maxTokens` is multiplied by +`COGETO_REASONING_HEADROOM` (default 4) for the bindings that reasoned, +because a cap sized for an answer is not sized for an answer plus its +deliberation; the exhausted-budget failure has its own named error instead of +masquerading as "returned no text". Details: +[`capabilities.md`](capabilities.md). + +## Part A: the channel + +`ModelGateway.completeStream` yields channel-tagged deltas +(`{channel: 'thinking' | 'text', text}`). The OpenAI-compatible adapter +surfaces `reasoning_content` (llama.cpp, DeepSeek), `reasoning` (OpenAI-style) +and `thinking` (Ollama) deltas on the thinking channel, and a thinking delta +also arms the Part B headroom, so live chat traffic teaches the adapter too. +Mistral yields text only; Anthropic maps `thinking_delta` blocks although +Cogeto never requests extended thinking. A non-reasoning model yields the +same bytes it always did, one field deeper. + +The four decorators keep their contracts, two by explicit ruling: + +- **The budget charges BOTH channels.** Thinking costs real tokens at the + provider, and on the reference reasoning model it is most of them; a + meter that ignored it would under-report spend several times over. +- **Redaction strips thinking, fail closed**, the vision posture. + Re-identification maps pseudonyms back into the text a user reads; a + reasoning model's deliberation interleaves pseudonym fragments the flush + logic cannot bound. Under redaction the thinking channel does not exist: + no delta, no empty disclosure. +- The egress audit counts both channels' characters (structural, never + content); the tier router dispatches untouched. + +## Part C: stored, streamed, shown + +`chat_message.thinking` (migration 0044, nullable text) stores the +deliberation beside the answer it produced. A `thinking` SSE event streams +deltas live, interleaved with `token` events. The chat UI renders a collapsed +**Thinking** disclosure above the answer (streaming while the model +deliberates, expandable, reopenable on a stored answer) and renders NOTHING +when there is no thinking: a non-reasoning model leaves no empty affordance. + +Erasure follows the answer. The answer-redaction cascade nulls `thinking` in +the same UPDATE that overwrites a citing answer's content, because reasoning +ABOUT an erased memory must not survive the citation that grounded it; row +deletion (message, conversation, source cascade) removes it implicitly, and +receipts are unchanged. + +## What is deliberately NOT here + +- No thinking in the answer prompt, the reply drafts, research synthesis, or + any prompt assembly: the channel ends at the disclosure and the column. +- No evaluation surface: the harness consumes answer text and never reads the + column or the channel. +- No configuration flag: display follows the model. A non-reasoning + configuration behaves byte-identically to the pre-channel system, held to + the reader-seam standard and tested as such. + +## Tests + +- `model-gateway/reasoning.spec.ts` (`reasoning_stream_channel`): labeled + channels in order, headroom armed from a stream, non-reasoning streams + unchanged. +- `model-gateway/redaction.spec.ts`: thinking stripped under redaction. +- `model-gateway/budgeted.gateway.spec.ts`: thinking charged. +- `chat/chat.integration.spec.ts` (`chat_thinking`): the SSE event, the + stored column, the DTO, and the answer never containing the deliberation. +- `chat/chat-answer-cascade.integration.spec.ts`: thinking nulled with the + redacted answer. +- `entrypoints/trust-scores.spec.ts`: the emission marker, on and off. diff --git a/project/shared/src/chat.ts b/project/shared/src/chat.ts index 746ab85e..5cdb610d 100644 --- a/project/shared/src/chat.ts +++ b/project/shared/src/chat.ts @@ -41,6 +41,10 @@ export interface ChatMessageDto { id: string; role: ChatRole; content: string; + /** The model's displayed deliberation (Part C of reasoning support), when a + * reasoning model produced one. A CHANNEL, never content: not capturable, + * not citable, not evaluated. Null for user rows and non-reasoning models. */ + thinking: string | null; createdAt: string; } @@ -105,6 +109,10 @@ export interface ChatResearchProposalRef { /** Server-sent events on POST /api/chat, in order: sources → token* → done. */ export type ChatStreamEvent = + /** A reasoning delta (Part C): displayed live in the collapsed Thinking + * disclosure. Interleaves with `token` events; absent entirely for a + * non-reasoning model. */ + | { type: 'thinking'; text: string } | { type: 'sources'; facts: ChatFactDto[] } | { type: 'token'; text: string } | { diff --git a/project/src/chat/chat-answer-cascade.integration.spec.ts b/project/src/chat/chat-answer-cascade.integration.spec.ts index 1450056c..58f7feda 100644 --- a/project/src/chat/chat-answer-cascade.integration.spec.ts +++ b/project/src/chat/chat-answer-cascade.integration.spec.ts @@ -41,7 +41,12 @@ describe(' chat-answer cascade (integration: real Postgres, real saga)', () => { // Messages need a container since — one per owner is enough here. const conversationIds = new Map(); - const insertMessage = async (ownerId: string, role: 'user' | 'assistant', content: string) => { + const insertMessage = async ( + ownerId: string, + role: 'user' | 'assistant', + content: string, + thinking: string | null = null, + ) => { let conversationId = conversationIds.get(ownerId); if (!conversationId) { const [conv] = await tdb.db.insert(conversation).values({ ownerId }).returning(); @@ -50,7 +55,7 @@ describe(' chat-answer cascade (integration: real Postgres, real saga)', () => { } const [row] = await tdb.db .insert(chatMessage) - .values({ ownerId, conversationId, role, content }) + .values({ ownerId, conversationId, role, content, thinking }) .returning({ id: chatMessage.id }); return row!.id; }; @@ -82,6 +87,8 @@ describe(' chat-answer cascade (integration: real Postgres, real saga)', () => { userA.userId, 'assistant', `The renewal is agreed {{cite:${m1.id}}}.`, + // Reasoning about the memory being erased (Part C): must go with it. + 'The Novira renewal fact seems most relevant here.', ); const citingOther = await insertMessage( userA.userId, @@ -105,6 +112,13 @@ describe(' chat-answer cascade (integration: real Postgres, real saga)', () => { // owner's AND the peer's (erasure is erasure); the timeline rows survive. expect(await contentOf(citing)).toBe(CHAT_ANSWER_REDACTED); expect(await contentOf(peerCiting)).toBe(CHAT_ANSWER_REDACTED); + // The thinking channel goes with the answer it deliberated (Part C): + // reasoning ABOUT an erased memory must not survive its citation. + const { rows: thinkingRows } = await tdb.pool.query<{ thinking: string | null }>( + `SELECT thinking FROM chat_message WHERE id = $1`, + [citing], + ); + expect(thinkingRows[0]!.thinking).toBeNull(); // An answer citing a different memory and the user's own words are untouched. expect(await contentOf(citingOther)).toContain(other.id); expect(await contentOf(userTurn)).toContain(m1.id); diff --git a/project/src/chat/chat-answer-cascade.ts b/project/src/chat/chat-answer-cascade.ts index fb83b6d9..11da5e10 100644 --- a/project/src/chat/chat-answer-cascade.ts +++ b/project/src/chat/chat-answer-cascade.ts @@ -46,7 +46,13 @@ export class ChatAnswerCascade implements DerivedCascade { ); const updated = await tx .update(chatMessage) - .set({ content: CHAT_ANSWER_REDACTED }) + .set({ + content: CHAT_ANSWER_REDACTED, + // Reasoning ABOUT an erased memory goes with the citation that + // grounded it (Part C): the thinking channel is content-bearing and + // must not outlive the answer it deliberated. + thinking: null, + }) .where(and(eq(chatMessage.role, 'assistant'), or(...citeMatches))) .returning({ id: chatMessage.id }); redacted += updated.length; diff --git a/project/src/chat/chat-capture.integration.spec.ts b/project/src/chat/chat-capture.integration.spec.ts index 49d77727..10435f58 100644 --- a/project/src/chat/chat-capture.integration.spec.ts +++ b/project/src/chat/chat-capture.integration.spec.ts @@ -10,6 +10,7 @@ import { createIngestionPipeline, createSuppressedFactLog } from '../ingestion/i import type { IngestionPipeline } from '../ingestion/index'; import { UserDirectory } from '../identity/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { StructuredExtractionRequest } from '../model-gateway/index'; import { RetrievalService } from '../retrieval/index'; import { ChatService } from './chat.service'; @@ -30,7 +31,7 @@ class ScriptedGateway extends ModelGateway { throw new Error('unused'); } // eslint-disable-next-line require-yield -- unused - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(texts: string[]): Promise { diff --git a/project/src/chat/chat-conversation.integration.spec.ts b/project/src/chat/chat-conversation.integration.spec.ts index 55f950ed..8570d3f7 100644 --- a/project/src/chat/chat-conversation.integration.spec.ts +++ b/project/src/chat/chat-conversation.integration.spec.ts @@ -5,6 +5,7 @@ import { startTestDatabase } from '../testing/index'; import type { TestDatabase } from '../testing/index'; import { UserDirectory } from '../identity/index'; import { ModelGateway } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { RetrievalService } from '../retrieval/index'; import type { ChatReplyResolverPort } from './chat-reply-resolver.port'; import type { ChatResearchProposal, ChatResearchResolverPort } from './chat-research-resolver.port'; @@ -38,9 +39,9 @@ class ScriptedGateway extends ModelGateway { complete(): never { throw new Error('no completion expected'); } - async *completeStream(request: { input: string }): AsyncIterable { + async *completeStream(request: { input: string }): AsyncIterable { this.streamCalls.push(request.input); - yield this.streamText; + yield { channel: 'text', text: this.streamText } as const; } async embed(texts: string[]): Promise { return texts.map(() => [0, 0, 0, 0]); diff --git a/project/src/chat/chat-conversations.integration.spec.ts b/project/src/chat/chat-conversations.integration.spec.ts index 5de2702e..891da392 100644 --- a/project/src/chat/chat-conversations.integration.spec.ts +++ b/project/src/chat/chat-conversations.integration.spec.ts @@ -14,6 +14,7 @@ import type { MemoryStore } from '../memory/index'; import { createIngestionPipeline, createSuppressedFactLog } from '../ingestion/index'; import { UserDirectory } from '../identity/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { StructuredExtractionRequest } from '../model-gateway/index'; import type { ZodType } from 'zod'; import { RetrievalService } from '../retrieval/index'; @@ -60,9 +61,9 @@ class ScriptedGateway extends ModelGateway { complete(): never { throw new Error('unused'); } - async *completeStream(request: { input: string }): AsyncIterable { + async *completeStream(request: { input: string }): AsyncIterable { this.streamCalls.push(request.input); - yield this.streamText; + yield { channel: 'text', text: this.streamText } as const; } async embed(texts: string[]): Promise { return texts.map((t) => fakeEmbedding(t, DIMS)); @@ -88,7 +89,7 @@ class CaptureGateway extends ModelGateway { throw new Error('unused'); } // eslint-disable-next-line require-yield -- unused - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(texts: string[]): Promise { diff --git a/project/src/chat/chat-reply-intent.integration.spec.ts b/project/src/chat/chat-reply-intent.integration.spec.ts index 8b59233d..e40d88f1 100644 --- a/project/src/chat/chat-reply-intent.integration.spec.ts +++ b/project/src/chat/chat-reply-intent.integration.spec.ts @@ -5,6 +5,7 @@ import type { TestDatabase, TestQdrant } from '../testing/index'; import { createMemoryStore } from '../memory/index'; import type { MemoryStore } from '../memory/index'; import { ModelGateway } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import { UserDirectory } from '../identity/index'; import { ChatService } from './chat.service'; import { RetrievalService } from '../retrieval/index'; @@ -42,7 +43,7 @@ class NoAnswerGateway extends ModelGateway { return 'test-embed'; } // eslint-disable-next-line require-yield -- must not be used - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { this.streamCalls += 1; throw new Error('reply intent must not stream an answer'); } diff --git a/project/src/chat/chat-research-intent.integration.spec.ts b/project/src/chat/chat-research-intent.integration.spec.ts index 06d362b0..3828719b 100644 --- a/project/src/chat/chat-research-intent.integration.spec.ts +++ b/project/src/chat/chat-research-intent.integration.spec.ts @@ -4,6 +4,7 @@ import { startTestDatabase } from '../testing/index'; import type { TestDatabase } from '../testing/index'; import { UserDirectory } from '../identity/index'; import { ModelGateway } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { RetrievalService } from '../retrieval/index'; import type { ChatResearchProposal, ChatResearchResolverPort } from './chat-research-resolver.port'; import { ChatService } from './chat.service'; @@ -49,7 +50,7 @@ class InertGateway extends ModelGateway { throw new Error('no completion expected'); } // eslint-disable-next-line require-yield -- ordinary-question path is stubbed to no facts - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('no stream expected'); } async embed(texts: string[]): Promise { diff --git a/project/src/chat/chat.integration.spec.ts b/project/src/chat/chat.integration.spec.ts index f94ecf55..2ef67746 100644 --- a/project/src/chat/chat.integration.spec.ts +++ b/project/src/chat/chat.integration.spec.ts @@ -5,6 +5,7 @@ import type { TestDatabase, TestQdrant } from '../testing/index'; import { createMemoryStore } from '../memory/index'; import type { MemoryStore, NewFact } from '../memory/index'; import { ModelGateway } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { CompletionRequest } from '../model-gateway/index'; import { UserDirectory } from '../identity/index'; import { ChatService } from './chat.service'; @@ -49,11 +50,16 @@ class ScriptedChatGateway extends ModelGateway { embeddingModelId(): string { return MODEL; } - async *completeStream(request: CompletionRequest): AsyncIterable { + /** Thinking deltas emitted before the answer (Part C tests). */ + thinkingDeltas: string[] = []; + + async *completeStream(request: CompletionRequest): AsyncIterable { this.streamRequests.push(request); - yield 'You owe Maja the draft contract before Thursday '; - yield '[F1]'; - yield '.'; + for (const thinking of this.thinkingDeltas) + yield { channel: 'thinking', text: thinking } as const; + yield { channel: 'text', text: 'You owe Maja the draft contract before Thursday ' } as const; + yield { channel: 'text', text: '[F1]' } as const; + yield { channel: 'text', text: '.' } as const; } } @@ -212,4 +218,42 @@ describe('chat (integration, real Postgres + real Qdrant, gateway mocked)', () = } expect(await counts()).toEqual(afterFirst); }); + + it('chat_thinking (Part C): streamed as its own event, stored beside the answer, never inside it', async () => { + gateway.thinkingDeltas = ['Considering the retrieved facts. ', 'Deciding on the citation. ']; + try { + const conversationId = (await chat.createConversation(userA)).id; + const events: import('@cogeto/shared').ChatStreamEvent[] = []; + for await (const event of chat.ask(userA, 'What do I owe Maja?', conversationId)) { + events.push(event); + } + + const thinkingEvents = events.filter((event) => event.type === 'thinking'); + expect(thinkingEvents.map((event) => (event as { text: string }).text)).toEqual([ + 'Considering the retrieved facts. ', + 'Deciding on the citation. ', + ]); + // The thinking never leaks into the answer text or its stored form. + const done = events.find((event) => event.type === 'done') as { content: string }; + expect(done.content).not.toContain('Considering the retrieved facts.'); + + const row = await tdb.pool.query<{ thinking: string | null; content: string }>( + `SELECT thinking, content FROM chat_message + WHERE conversation_id = $1 AND role = 'assistant' + ORDER BY created_at DESC LIMIT 1`, + [conversationId], + ); + expect(row.rows[0]!.thinking).toBe( + 'Considering the retrieved facts. Deciding on the citation. ', + ); + expect(row.rows[0]!.content).not.toContain('Considering'); + + // The message page DTO carries it for reopening the conversation. + const page = await chat.listMessages(userA, conversationId, { limit: 10 }); + const assistant = page.items.find((message) => message.role === 'assistant'); + expect(assistant?.thinking).toContain('Deciding on the citation.'); + } finally { + gateway.thinkingDeltas = []; + } + }); }); diff --git a/project/src/chat/chat.service.ts b/project/src/chat/chat.service.ts index 3e7ed7ec..ee72b849 100644 --- a/project/src/chat/chat.service.ts +++ b/project/src/chat/chat.service.ts @@ -144,8 +144,12 @@ export class ChatService { /** The handlers' narrow view of this service (see intent-plumbing). */ private sink(): ChatTurnSink { return { - storeAssistant: (principal: Principal, conversationId: string, content: string) => - this.storeAssistant(principal, conversationId, content), + storeAssistant: ( + principal: Principal, + conversationId: string, + content: string, + thinking?: string | null, + ) => this.storeAssistant(principal, conversationId, content, thinking ?? null), getPrompt: () => this.getPrompt(), logWarn: (message: string) => this.logger.warn(message), }; @@ -281,6 +285,7 @@ export class ChatService { id: row.id, role: row.role, content: row.content, + thinking: row.thinking, createdAt: row.createdAt.toISOString(), })), total: totalRows[0]?.count ?? 0, @@ -632,10 +637,11 @@ export class ChatService { principal: Principal, conversationId: string, content: string, + thinking: string | null = null, ): Promise<{ id: string }> { const [row] = await this.db .insert(chatMessage) - .values({ ownerId: principal.userId, conversationId, role: 'assistant', content }) + .values({ ownerId: principal.userId, conversationId, role: 'assistant', content, thinking }) .returning(); await this.touchConversation(conversationId, row!.createdAt); await this.maybeRequestTitle(principal, conversationId); diff --git a/project/src/chat/conversation-titler.integration.spec.ts b/project/src/chat/conversation-titler.integration.spec.ts index 32deab5a..e9027f98 100644 --- a/project/src/chat/conversation-titler.integration.spec.ts +++ b/project/src/chat/conversation-titler.integration.spec.ts @@ -4,6 +4,7 @@ import { afterAll, beforeAll, describe, expect, it } from 'vitest'; import { startTestDatabase } from '../testing/index'; import type { TestDatabase } from '../testing/index'; import { ModelGateway } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import { chatMessage, conversation } from './persistence/tables'; import { ConversationTitler, sanitizeTitle } from './conversation-titler'; @@ -20,7 +21,7 @@ class TitleGateway extends ModelGateway { throw new Error('unused'); } // eslint-disable-next-line require-yield -- unused - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(): Promise { diff --git a/project/src/chat/instance-context.integration.spec.ts b/project/src/chat/instance-context.integration.spec.ts index d000ad2b..d8210a8a 100644 --- a/project/src/chat/instance-context.integration.spec.ts +++ b/project/src/chat/instance-context.integration.spec.ts @@ -4,6 +4,7 @@ import { startTestDatabase } from '../testing/index'; import type { TestDatabase } from '../testing/index'; import { UserDirectory } from '../identity/index'; import { ModelGateway } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import { formatNow, UserContextService } from '../infrastructure/index'; import type { RetrievalService } from '../retrieval/index'; import { ChatService } from './chat.service'; @@ -34,9 +35,9 @@ class ScriptedGateway extends ModelGateway { complete(): never { throw new Error('no completion expected'); } - async *completeStream(request: { input: string }): AsyncIterable { + async *completeStream(request: { input: string }): AsyncIterable { this.streamCalls.push(request.input); - yield this.streamText; + yield { channel: 'text', text: this.streamText } as const; } async embed(texts: string[]): Promise { return texts.map(() => [0, 0, 0, 0]); diff --git a/project/src/chat/intents/answer.handler.ts b/project/src/chat/intents/answer.handler.ts index ad541401..4cb1d077 100644 --- a/project/src/chat/intents/answer.handler.ts +++ b/project/src/chat/intents/answer.handler.ts @@ -55,6 +55,7 @@ export class MemoryAnswerHandler { yield { type: 'sources', facts }; let answer: string; + let thinking = ''; if (retrieved.mode === 'open_loops' && (retrieved.openLoops?.length ?? 0) === 0) { // Zero open loops is an ANSWER (all clear), not a data gap. A // deterministic string cannot mirror; it follows the anchor (0052). @@ -82,9 +83,17 @@ export class MemoryAnswerHandler { }), tier: 'answer', }); - for await (const text of stream) { - buffer += text; - yield { type: 'token', text }; + // Two channels, two fates (Part C): thinking streams to the disclosure + // and is stored BESIDE the answer; only the text channel becomes the + // answer — it alone is sanitized, cited, capturable, and evaluated. + for await (const delta of stream) { + if (delta.channel === 'thinking') { + thinking += delta.text; + yield { type: 'thinking', text: delta.text }; + continue; + } + buffer += delta.text; + yield { type: 'token', text: delta.text }; } answer = buffer; } @@ -94,7 +103,12 @@ export class MemoryAnswerHandler { // Metadata only — never the answer content or tokens (pino rule). this.sink.logWarn(`citation_violation stripped=${violations}`); } - const row = await this.sink.storeAssistant(principal, conversationId, stored); + const row = await this.sink.storeAssistant( + principal, + conversationId, + stored, + thinking.trim() ? thinking : null, + ); // The research offer: every knowledge-class answer OFFERS // research as a one-tap bridge into the existing gate — never a silent // search. The offer carries the self-contained topic; tapping it proposes. diff --git a/project/src/chat/intents/intent-plumbing.ts b/project/src/chat/intents/intent-plumbing.ts index 6e16107d..8f40682c 100644 --- a/project/src/chat/intents/intent-plumbing.ts +++ b/project/src/chat/intents/intent-plumbing.ts @@ -14,6 +14,8 @@ export interface ChatTurnSink { principal: Principal, conversationId: string, content: string, + /** The displayed deliberation (Part C); null when the model produced none. */ + thinking?: string | null, ): Promise<{ id: string }>; getPrompt(): Promise; /** Metadata only — never answer content or tokens (pino rule). */ diff --git a/project/src/chat/intents/small-talk.handler.ts b/project/src/chat/intents/small-talk.handler.ts index 0a5d39b9..321bff68 100644 --- a/project/src/chat/intents/small-talk.handler.ts +++ b/project/src/chat/intents/small-talk.handler.ts @@ -48,20 +48,31 @@ export class SmallTalkHandler { yield { type: 'sources', facts: [] }; const prompt = await this.sink.getPrompt(); let buffer = ''; + let thinking = ''; const stream = this.gateway.completeStream({ system: prompt.content, input: buildSmallTalkInput(history, content, contextBlock), tier: 'answer', }); - for await (const text of stream) { - buffer += text; - yield { type: 'token', text }; + for await (const delta of stream) { + if (delta.channel === 'thinking') { + thinking += delta.text; + yield { type: 'thinking', text: delta.text }; + continue; + } + buffer += delta.text; + yield { type: 'token', text: delta.text }; } const { text: stored, violations } = toStoredAnswer(buffer, []); if (violations > 0) { this.sink.logWarn(`citation_violation stripped=${violations}`); } - const row = await this.sink.storeAssistant(principal, conversationId, stored); + const row = await this.sink.storeAssistant( + principal, + conversationId, + stored, + thinking.trim() ? thinking : null, + ); yield { type: 'done', messageId: row.id, content: stored, citationViolations: violations }; } } diff --git a/project/src/chat/persistence/tables.ts b/project/src/chat/persistence/tables.ts index 0d93a6fe..dfb190f6 100644 --- a/project/src/chat/persistence/tables.ts +++ b/project/src/chat/persistence/tables.ts @@ -44,6 +44,14 @@ export const chatMessage = pgTable( .references(() => conversation.id), role: chatRoleEnum('role').notNull(), content: text('content').notNull(), + /** + * The model's displayed deliberation (Part C of reasoning support, + * migration 0044): a CHANNEL beside the answer, never content. Null for + * user rows and non-reasoning models. Content-bearing, so the answer + * redaction cascade nulls it with the content overwrite, and row deletion + * takes it implicitly. + */ + thinking: text('thinking'), createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(), }, (t) => [ diff --git a/project/src/email/email-intake.integration.spec.ts b/project/src/email/email-intake.integration.spec.ts index 8ae288f2..6ce47639 100644 --- a/project/src/email/email-intake.integration.spec.ts +++ b/project/src/email/email-intake.integration.spec.ts @@ -14,6 +14,7 @@ import { } from '../testing/index'; import type { TestDatabase, TestMinio, TestQdrant } from '../testing/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { StructuredExtractionRequest } from '../model-gateway/index'; import { createMemoryReconciliation, MemoryFileStore, MemoryObjectStore } from '../memory/index'; import type { MemoryStore, MemoryReconciliation } from '../memory/index'; @@ -63,7 +64,7 @@ class ScriptedGateway extends ModelGateway { throw new Error('unused'); } // eslint-disable-next-line require-yield -- unused - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(texts: string[]): Promise { diff --git a/project/src/email/email-reply-triggers.integration.spec.ts b/project/src/email/email-reply-triggers.integration.spec.ts index 9d1b4f4e..5008bb30 100644 --- a/project/src/email/email-reply-triggers.integration.spec.ts +++ b/project/src/email/email-reply-triggers.integration.spec.ts @@ -10,6 +10,7 @@ import { } from '../testing/index'; import type { TestDatabase, TestMinio, TestQdrant } from '../testing/index'; import { ModelGateway } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { CompletionResult, StructuredExtractionRequest } from '../model-gateway/index'; import { createMemoryStore, MemoryFileStore, MemoryObjectStore } from '../memory/index'; import type { MemoryStore } from '../memory/index'; @@ -39,7 +40,7 @@ class ScriptedGateway extends ModelGateway { return { text: 'Thanks — Friday works for the delivery. Best regards.' }; } // eslint-disable-next-line require-yield -- unused - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(texts: string[]): Promise { diff --git a/project/src/email/email-thread-dedup.integration.spec.ts b/project/src/email/email-thread-dedup.integration.spec.ts index 471de06d..1f700a99 100644 --- a/project/src/email/email-thread-dedup.integration.spec.ts +++ b/project/src/email/email-thread-dedup.integration.spec.ts @@ -12,6 +12,7 @@ import { } from '../testing/index'; import type { TestDatabase, TestMinio, TestQdrant } from '../testing/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { StructuredExtractionRequest } from '../model-gateway/index'; import { createMemoryReconciliation, MemoryFileStore, MemoryObjectStore } from '../memory/index'; import type { MemoryReconciliation, MemoryStore } from '../memory/index'; @@ -50,7 +51,7 @@ class ScriptedGateway extends ModelGateway { throw new Error('unused'); } // eslint-disable-next-line require-yield -- unused - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(texts: string[]): Promise { diff --git a/project/src/entrypoints/demo/demo-seed.integration.spec.ts b/project/src/entrypoints/demo/demo-seed.integration.spec.ts index 629f4d20..52fabad4 100644 --- a/project/src/entrypoints/demo/demo-seed.integration.spec.ts +++ b/project/src/entrypoints/demo/demo-seed.integration.spec.ts @@ -12,6 +12,7 @@ import type { TestDatabase, TestMinio, TestQdrant } from '../../testing/index'; import { createMemoryStore, MemoryObjectStore, reindexMemories } from '../../memory/index'; import type { MemoryStore } from '../../memory/index'; import { ModelGateway } from '../../model-gateway/index'; +import type { StreamDelta } from '../../model-gateway/index'; import { assertEndState, inspectEndState } from './assertions'; import { fileObjectKeys, truncateDomainTables } from './ops'; @@ -33,7 +34,7 @@ class FakeGateway extends ModelGateway { throw new Error('unused'); } // eslint-disable-next-line require-yield -- unused - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(texts: string[]): Promise { diff --git a/project/src/entrypoints/eval-cache.ts b/project/src/entrypoints/eval-cache.ts index c55a59e0..b8e0d6e3 100644 --- a/project/src/entrypoints/eval-cache.ts +++ b/project/src/entrypoints/eval-cache.ts @@ -9,6 +9,7 @@ import type { CompletionResult, ResolvedModelProviders, StructuredExtractionRequest, + StreamDelta, } from '../model-gateway/index'; /** @@ -342,7 +343,7 @@ export class CachingModelGateway extends ModelGateway { * full text and score that, so chunk boundaries are not observable; nothing * that is scored depends on them. */ - async *completeStream(request: CompletionRequest): AsyncIterable { + async *completeStream(request: CompletionRequest): AsyncIterable { const model = this.modelFor(request.tier, 'answer'); const key = evalCacheKey({ op: 'complete', @@ -353,13 +354,16 @@ export class CachingModelGateway extends ModelGateway { }); const hit = this.options.store.getText(key); if (hit !== undefined) { - yield hit; + yield { channel: 'text', text: hit }; return; } if (this.options.mode === 'replay') throw this.miss('stream', request.input); + // The TEXT channel only is recorded (Part A): thinking is never evaluated + // (honesty rule 2), so a fixture that froze it would be a measurement of + // something the harness must not measure. let assembled = ''; for await (const delta of this.inner.completeStream(request)) { - assembled += delta; + if (delta.channel === 'text') assembled += delta.text; yield delta; } this.options.store.putText(key, { diff --git a/project/src/entrypoints/eval-chat.ts b/project/src/entrypoints/eval-chat.ts index f37e7a6b..26e270ec 100644 --- a/project/src/entrypoints/eval-chat.ts +++ b/project/src/entrypoints/eval-chat.ts @@ -37,7 +37,12 @@ import { import { ChatSkillResolver, SkillEngine, SkillPlanner, SkillRunService } from '../skills/index'; import type { ResearchOptions } from '../research/index'; import { InMemoryDailyCounters } from '../infrastructure/index'; -import { createModelGateway, loadPrompt, ModelGateway } from '../model-gateway/index'; +import { + createModelGateway, + loadPrompt, + ModelGateway, + probeReasoning, +} from '../model-gateway/index'; import type { ResolvedModelProviders } from '../model-gateway/index'; import { resolveEvalProviders, requireConfiguredProviders } from './eval-env'; import { EVAL_SCORING_VERSION, evalCacheModeFromEnv, wrapWithEvalCache } from './eval-cache'; @@ -1098,8 +1103,12 @@ async function main(): Promise { process.exit(2); } // The ACTIVE configuration, from the same resolver the gateway was built - // with — id and models are exact by construction. - const { id, models } = configurationForEmission(providers); + // with — id and models are exact by construction. The reasoning probe + // mirrors eval.ts exactly, so the partial-merge id guard holds (Part C). + const reasoningProbe = await probeReasoning(gateway, providers); + const { id, models } = configurationForEmission(providers, { + reasoning: reasoningProbe.reasoning, + }); emitPartial(emitPath, { schema_version: TRUST_SCORES_SCHEMA_VERSION, harness: `chat ${ANSWER_PROMPT.family}/${ANSWER_PROMPT.version} · grader ${COVERAGE_PROMPT.family}/${COVERAGE_PROMPT.version}`, diff --git a/project/src/entrypoints/eval.ts b/project/src/entrypoints/eval.ts index 54e65b1f..22c95d89 100644 --- a/project/src/entrypoints/eval.ts +++ b/project/src/entrypoints/eval.ts @@ -5,7 +5,7 @@ import { evalConfigSchema, runGoldenEval, runReconcileEval } from '../ingestion/ import type { EvalMetrics, ReconcileEvalMetrics } from '../ingestion/index'; import { runRewriteEval } from '../retrieval/index'; import type { RewriteEvalMetrics } from '../retrieval/index'; -import { createModelGateway } from '../model-gateway/index'; +import { createModelGateway, probeReasoning } from '../model-gateway/index'; import { resolveEvalProviders, requireConfiguredProviders } from './eval-env'; import { EVAL_SCORING_VERSION, evalCacheModeFromEnv, wrapWithEvalCache } from './eval-cache'; import { configurationForEmission, emitPartial } from './trust-scores'; @@ -227,8 +227,13 @@ async function main(): Promise { process.exit(2); } // The ACTIVE configuration, from the same resolver the gateway was built - // with — id and models are exact by construction. - const { id, models } = configurationForEmission(providers); + // with — id and models are exact by construction. Reasoning is PROBED so + // the emitted id labels what this run actually measured (Part C); replay + // never reaches this branch, so the probe only runs on live emissions. + const reasoningProbe = await probeReasoning(gateway, providers); + const { id, models } = configurationForEmission(providers, { + reasoning: reasoningProbe.reasoning, + }); const reconcileByLabel = new Map(reconcile.perLanguage.map((m) => [m.label, m])); const rewriteByLabel = new Map(rewrite.perLanguage.map((m) => [m.label, m])); emitPartial(emitPath, { diff --git a/project/src/entrypoints/trust-scores.spec.ts b/project/src/entrypoints/trust-scores.spec.ts index 7da7e821..1da42d8f 100644 --- a/project/src/entrypoints/trust-scores.spec.ts +++ b/project/src/entrypoints/trust-scores.spec.ts @@ -205,6 +205,15 @@ describe('eval_emission_config_correct', () => { expect(emission.id.endsWith('-redacted')).toBe(true); // The emitted id stays inside the published schema's pattern. expect(emission.id).toMatch(/^[a-z0-9][a-z0-9-]*$/); + + // The PROBED reasoning marker (Part C, honesty rule 3): a run with + // thinking on is a different measurement and its id says so; probed off + // emits the unchanged id, so every existing artifact and gate is + // untouched — a Mistral-routed run can never gain the marker by accident. + const marked = configurationForEmission(providers, { reasoning: true }); + expect(marked.id).toBe(`${providers.id}--reasoning`); + expect(marked.id).toMatch(/^[a-z0-9][a-z0-9-]*$/); + expect(configurationForEmission(providers, { reasoning: false }).id).toBe(providers.id); }); it('emits the local configurations with correct ids', () => { diff --git a/project/src/entrypoints/trust-scores.ts b/project/src/entrypoints/trust-scores.ts index 19eff57e..a4083482 100644 --- a/project/src/entrypoints/trust-scores.ts +++ b/project/src/entrypoints/trust-scores.ts @@ -161,12 +161,26 @@ export const indexSchema = z.array(indexEntrySchema); * construction. Ids are the website's join key — the derivation (preset name * or the full per-tier form, `-redacted` suffix) lives with the resolver. */ -export function configurationForEmission(providers: ResolvedModelProviders): { +export function configurationForEmission( + providers: ResolvedModelProviders, + options: { + /** + * PROBED reasoning state (reasoning support Part C, honesty rule 3): a run + * with thinking on is a different measurement, so the marker joins the id + * the way `--vis-` does. Appended HERE, at emission time, rather than in + * the resolver: whether a binding reasons is a runtime fact the static + * resolver cannot know, and only emission labels a measurement. A + * Mistral-routed run probes false and emits the unchanged id, so every + * existing artifact, gate and cached fixture is untouched. + */ + reasoning?: boolean; + } = {}, +): { id: string; models: { pipeline: string; answer: string; embedding: string }; } { return { - id: providers.id, + id: options.reasoning ? `${providers.id}--reasoning` : providers.id, models: { pipeline: providers.tiers.pipeline.model, answer: providers.tiers.answer.model, diff --git a/project/src/files/files.integration.spec.ts b/project/src/files/files.integration.spec.ts index 1c9a1f78..a81114ed 100644 --- a/project/src/files/files.integration.spec.ts +++ b/project/src/files/files.integration.spec.ts @@ -17,6 +17,7 @@ import { } from '../testing/index'; import type { TestDatabase, TestMinio, TestQdrant } from '../testing/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { StructuredExtractionRequest } from '../model-gateway/index'; import { createMemoryReconciliation, @@ -88,7 +89,7 @@ class ScriptedGateway extends ModelGateway { throw new Error('complete() is not used by the pipeline'); } // eslint-disable-next-line require-yield -- not used by the pipeline - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('completeStream() is not used by the pipeline'); } async embed(texts: string[]): Promise { diff --git a/project/src/files/reading/scan.integration.spec.ts b/project/src/files/reading/scan.integration.spec.ts index 2a074a71..8b8771d8 100644 --- a/project/src/files/reading/scan.integration.spec.ts +++ b/project/src/files/reading/scan.integration.spec.ts @@ -2,6 +2,7 @@ import { describe, expect, it } from 'vitest'; import { PDF_CONTENT_TYPE } from '@cogeto/shared'; import { makeScannedPdf } from '../../testing/index'; import { ModelGateway } from '../../model-gateway/index'; +import type { StreamDelta } from '../../model-gateway/index'; import type { CompletionResult, VisionRequest } from '../../model-gateway/index'; import { VisionUnavailableError } from '../../model-gateway/index'; import { ocrAvailable } from './ocr'; @@ -50,7 +51,7 @@ class ScriptedVision extends ModelGateway { throw new Error('unused'); } // eslint-disable-next-line require-yield -- unused here - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } extractStructured(): never { diff --git a/project/src/ingestion/auto-review-resolution.integration.spec.ts b/project/src/ingestion/auto-review-resolution.integration.spec.ts index 8b76787d..d4c3da42 100644 --- a/project/src/ingestion/auto-review-resolution.integration.spec.ts +++ b/project/src/ingestion/auto-review-resolution.integration.spec.ts @@ -14,6 +14,7 @@ import { import type { MemoryStore, SourceDeletion } from '../memory/index'; import type { Tx } from '../infrastructure/index'; import { ModelGateway } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { StructuredExtractionRequest } from '../model-gateway/index'; import { createSuppressedFactLog } from './persistence/suppressed-fact-log'; import type { SuppressedFactLog } from './persistence/suppressed-fact-log'; @@ -67,7 +68,7 @@ class ScriptedGateway extends ModelGateway { throw new Error('not used'); } // eslint-disable-next-line require-yield -- not used - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('not used'); } async embed(texts: string[]): Promise { diff --git a/project/src/ingestion/dreaming.integration.spec.ts b/project/src/ingestion/dreaming.integration.spec.ts index bc744014..da81100f 100644 --- a/project/src/ingestion/dreaming.integration.spec.ts +++ b/project/src/ingestion/dreaming.integration.spec.ts @@ -13,6 +13,7 @@ import type { NewFact, } from '../memory/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { StructuredExtractionRequest } from '../model-gateway/index'; import type { AuthenticatedRequest } from '../identity/index'; import { DreamingService } from './dreaming.service'; @@ -42,7 +43,7 @@ class CountingJudgeGateway extends ModelGateway { throw new Error('not used'); } // eslint-disable-next-line require-yield -- not used by dreaming - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('not used'); } async embed(): Promise { diff --git a/project/src/ingestion/pipeline.integration.spec.ts b/project/src/ingestion/pipeline.integration.spec.ts index 0d7eaec2..638b1efb 100644 --- a/project/src/ingestion/pipeline.integration.spec.ts +++ b/project/src/ingestion/pipeline.integration.spec.ts @@ -14,6 +14,7 @@ import type { ParseCaps } from '../infrastructure/index'; import { createMemoryStore, MemoryReconciliation } from '../memory/index'; import type { MemoryStore } from '../memory/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { StructuredExtractionRequest } from '../model-gateway/index'; import type { CandidateFact } from './domain/candidate-fact'; import { AnchorStage } from './pipeline/anchor.stage'; @@ -68,7 +69,7 @@ class ScriptedGateway extends ModelGateway { throw new Error('complete() is not used by the pipeline'); } // eslint-disable-next-line require-yield -- not used by the pipeline - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('completeStream() is not used by the pipeline'); } async embed(texts: string[]): Promise { diff --git a/project/src/ingestion/pipeline/anchor.spec.ts b/project/src/ingestion/pipeline/anchor.spec.ts index 29bc8312..0989480f 100644 --- a/project/src/ingestion/pipeline/anchor.spec.ts +++ b/project/src/ingestion/pipeline/anchor.spec.ts @@ -1,6 +1,7 @@ import { describe, expect, it } from 'vitest'; import type { ZodType } from 'zod'; import { ModelGateway } from '../../model-gateway/index'; +import type { StreamDelta } from '../../model-gateway/index'; import type { StructuredExtractionRequest } from '../../model-gateway/index'; import { AnchorStage, computeSourceContext } from './anchor.stage'; import { @@ -35,7 +36,7 @@ class OneShotGateway extends ModelGateway { throw new Error('not used'); } // eslint-disable-next-line require-yield -- not used - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('not used'); } async embed(): Promise { diff --git a/project/src/ingestion/pipeline/verify-batch.spec.ts b/project/src/ingestion/pipeline/verify-batch.spec.ts index ebbd1c7a..ff8e41d6 100644 --- a/project/src/ingestion/pipeline/verify-batch.spec.ts +++ b/project/src/ingestion/pipeline/verify-batch.spec.ts @@ -1,6 +1,7 @@ import { describe, expect, it } from 'vitest'; import type { ZodType } from 'zod'; import { ModelGateway } from '../../model-gateway/index'; +import type { StreamDelta } from '../../model-gateway/index'; import type { StructuredExtractionRequest } from '../../model-gateway/index'; import type { CandidateFact } from '../domain/candidate-fact'; import { buildVerificationBatchInput, VerifyStage } from './verify.stage'; @@ -31,7 +32,7 @@ class RecordingGateway extends ModelGateway { throw new Error('unused'); } // eslint-disable-next-line require-yield -- unused - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(): Promise { diff --git a/project/src/ingestion/reconcile.integration.spec.ts b/project/src/ingestion/reconcile.integration.spec.ts index 301f8bd6..c467d18c 100644 --- a/project/src/ingestion/reconcile.integration.spec.ts +++ b/project/src/ingestion/reconcile.integration.spec.ts @@ -7,6 +7,7 @@ import type { TestDatabase, TestQdrant } from '../testing/index'; import { createMemoryReconciliation } from '../memory/index'; import type { MemoryReconciliation, MemoryRow, MemoryStore, NewFact } from '../memory/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { StructuredExtractionRequest } from '../model-gateway/index'; import { ReconciliationService } from './pipeline/reconcile.stage'; import { noopLog } from './pipeline/pipeline-log'; @@ -55,7 +56,7 @@ class ScriptedJudgeGateway extends ModelGateway { throw new Error('not used'); } // eslint-disable-next-line require-yield -- not used by reconciliation - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('not used'); } async embed(texts: string[]): Promise { diff --git a/project/src/memory/conversation-deletion-cascade.integration.spec.ts b/project/src/memory/conversation-deletion-cascade.integration.spec.ts index 15db78ae..6eb72158 100644 --- a/project/src/memory/conversation-deletion-cascade.integration.spec.ts +++ b/project/src/memory/conversation-deletion-cascade.integration.spec.ts @@ -16,6 +16,7 @@ import { } from '../testing/index'; import type { TestDatabase, TestMinio, TestQdrant } from '../testing/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { StructuredExtractionRequest } from '../model-gateway/index'; import { ChatSourceReader, ConversationSourceDeletion } from '../chat/index'; import { UserSettingsService } from '../settings/index'; @@ -64,7 +65,7 @@ class ScriptedGateway extends ModelGateway { throw new Error('unused'); } // eslint-disable-next-line require-yield -- unused - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(texts: string[]): Promise { diff --git a/project/src/memory/deletion-race.integration.spec.ts b/project/src/memory/deletion-race.integration.spec.ts index 491c9af2..c97d3084 100644 --- a/project/src/memory/deletion-race.integration.spec.ts +++ b/project/src/memory/deletion-race.integration.spec.ts @@ -23,6 +23,7 @@ import { } from '../ingestion/index'; import type { IngestionPipeline } from '../ingestion/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { StructuredExtractionRequest } from '../model-gateway/index'; import { MemoryStore } from './memory.store'; import { MemoryReconciliation } from './reconciliation'; @@ -64,7 +65,7 @@ class SlowExtractionGateway extends ModelGateway { throw new Error('complete() is not used by the pipeline'); } // eslint-disable-next-line require-yield -- not used by the pipeline - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('completeStream() is not used by the pipeline'); } async embed(texts: string[]): Promise { diff --git a/project/src/memory/email-deletion-cascade.integration.spec.ts b/project/src/memory/email-deletion-cascade.integration.spec.ts index 4a840c72..f1ad18aa 100644 --- a/project/src/memory/email-deletion-cascade.integration.spec.ts +++ b/project/src/memory/email-deletion-cascade.integration.spec.ts @@ -16,6 +16,7 @@ import { } from '../testing/index'; import type { TestDatabase, TestMinio, TestQdrant } from '../testing/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { StructuredExtractionRequest } from '../model-gateway/index'; import { EmailAllowlistService, @@ -67,7 +68,7 @@ class ScriptedGateway extends ModelGateway { throw new Error('unused'); } // eslint-disable-next-line require-yield -- unused - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(texts: string[]): Promise { diff --git a/project/src/memory/governance.integration.spec.ts b/project/src/memory/governance.integration.spec.ts index e6be32e1..d3d59573 100644 --- a/project/src/memory/governance.integration.spec.ts +++ b/project/src/memory/governance.integration.spec.ts @@ -4,6 +4,7 @@ import type { Principal } from '@cogeto/shared'; import { fakeEmbedding, startTestDatabase, startTestQdrant } from '../testing/index'; import type { TestDatabase, TestQdrant } from '../testing/index'; import { ModelGateway } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import { MemoryStore } from './memory.store'; import type { NewFact } from './memory.store'; import { runMemoryEmbedJob } from './embed-job'; @@ -34,7 +35,7 @@ class FakeEmbedGateway extends ModelGateway { throw new Error('not used'); } // eslint-disable-next-line require-yield -- not used - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('not used'); } extractStructured(): Promise { diff --git a/project/src/memory/local-embeddings.integration.spec.ts b/project/src/memory/local-embeddings.integration.spec.ts index 4a9646ee..e6f1877a 100644 --- a/project/src/memory/local-embeddings.integration.spec.ts +++ b/project/src/memory/local-embeddings.integration.spec.ts @@ -3,6 +3,7 @@ import type { Principal } from '@cogeto/shared'; import { fakeEmbedding, startTestDatabase, startTestQdrant } from '../testing/index'; import type { TestDatabase, TestQdrant } from '../testing/index'; import { ModelGateway } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import { MemoryStore } from './memory.store'; import type { NewFact } from './memory.store'; import { vectorIndexDimensionMismatch } from './embedding-space'; @@ -44,7 +45,7 @@ class FakeEmbedGateway extends ModelGateway { throw new Error('not used'); } // eslint-disable-next-line require-yield -- not used - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('not used'); } extractStructured(): Promise { diff --git a/project/src/memory/upload-cascade.integration.spec.ts b/project/src/memory/upload-cascade.integration.spec.ts index bce5730b..2d7d4a38 100644 --- a/project/src/memory/upload-cascade.integration.spec.ts +++ b/project/src/memory/upload-cascade.integration.spec.ts @@ -22,6 +22,7 @@ import { } from '../testing/index'; import type { TestDatabase, TestMinio, TestQdrant } from '../testing/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { StructuredExtractionRequest } from '../model-gateway/index'; import { FilesService, FileSourceReader } from '../files/index'; import { @@ -79,7 +80,7 @@ class ScriptedGateway extends ModelGateway { throw new Error('unused'); } // eslint-disable-next-line require-yield -- unused by the pipeline - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(texts: string[]): Promise { diff --git a/project/src/memory/vector.integration.spec.ts b/project/src/memory/vector.integration.spec.ts index cbcc3f9b..130f238b 100644 --- a/project/src/memory/vector.integration.spec.ts +++ b/project/src/memory/vector.integration.spec.ts @@ -3,6 +3,7 @@ import type { Principal } from '@cogeto/shared'; import { fakeEmbedding, startTestDatabase, startTestQdrant } from '../testing/index'; import type { TestDatabase, TestQdrant } from '../testing/index'; import { ModelGateway } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import { MemoryStore } from './memory.store'; import type { NewFact } from './memory.store'; import { buildGateFilter, MemoryVectorStore } from './persistence/vector-store'; @@ -29,7 +30,7 @@ class FakeEmbedGateway extends ModelGateway { throw new Error('not used'); } // eslint-disable-next-line require-yield -- not used - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('not used'); } extractStructured(): Promise { diff --git a/project/src/memory/web-deletion-cascade.integration.spec.ts b/project/src/memory/web-deletion-cascade.integration.spec.ts index 6d6034ea..f7e23be3 100644 --- a/project/src/memory/web-deletion-cascade.integration.spec.ts +++ b/project/src/memory/web-deletion-cascade.integration.spec.ts @@ -20,6 +20,7 @@ import { } from '../testing/index'; import type { TestDatabase, TestMinio, TestQdrant } from '../testing/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { StructuredExtractionRequest } from '../model-gateway/index'; import { ResearchService, @@ -71,7 +72,7 @@ class ScriptedGateway extends ModelGateway { throw new Error('unused'); } // eslint-disable-next-line require-yield -- unused - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(texts: string[]): Promise { diff --git a/project/src/migrations/0044_chat_thinking.sql b/project/src/migrations/0044_chat_thinking.sql new file mode 100644 index 00000000..43826498 --- /dev/null +++ b/project/src/migrations/0044_chat_thinking.sql @@ -0,0 +1,17 @@ +-- 0044: the displayed thinking channel (reasoning support Part C). +-- +-- A reasoning model returns its deliberation in a separate stream beside the +-- answer. Cogeto displays it, live and afterwards, because hiding what the +-- instance's own model said while deciding would be the opposite of the +-- product's posture; but it is a CHANNEL, never content. The three honesty +-- rules are structural elsewhere: capture reads user rows only, citations and +-- the answer sanitizer read `content` only, and the eval harness never sees +-- the column. +-- +-- Content-bearing, with the same erasure story as the answer it explains: the +-- answer-redaction cascade (chat-answer-cascade) nulls `thinking` in the same +-- UPDATE that overwrites `content`, because reasoning ABOUT an erased memory +-- must not survive the citation that grounded it; conversation and message +-- deletion remove the row wholesale, receipts unchanged. + +ALTER TABLE chat_message ADD COLUMN thinking text; diff --git a/project/src/model-gateway/anthropic.gateway.ts b/project/src/model-gateway/anthropic.gateway.ts index cdec3edf..6c5280cc 100644 --- a/project/src/model-gateway/anthropic.gateway.ts +++ b/project/src/model-gateway/anthropic.gateway.ts @@ -7,6 +7,7 @@ import type { ModelTier, StructuredExtractionRequest, TokenUsage, + StreamDelta, } from './model-gateway.service'; import { ModelGatewayError } from './errors'; import { @@ -106,7 +107,7 @@ export class AnthropicModelGateway extends ModelGateway { return { text: textOf(response), ...usageOf(response) }; } - async *completeStream(request: CompletionRequest): AsyncIterable { + async *completeStream(request: CompletionRequest): AsyncIterable { const response = await callWithRetry('anthropic', () => postStream( `${this.baseUrl}/v1/messages`, @@ -134,7 +135,14 @@ export class AnthropicModelGateway extends ModelGateway { } if (event.type === 'content_block_delta' && event.delta?.type === 'text_delta') { const text = event.delta.text; - if (typeof text === 'string' && text) yield text; + if (typeof text === 'string' && text) yield { channel: 'text', text }; + } + // Anthropic thinking blocks arrive only when extended thinking is + // requested, which Cogeto does not do; mapped anyway (Part A) so a + // future opt-in cannot silently lose the channel. + if (event.type === 'content_block_delta' && event.delta?.type === 'thinking_delta') { + const thinking = (event.delta as { thinking?: unknown }).thinking; + if (typeof thinking === 'string' && thinking) yield { channel: 'thinking', text: thinking }; } } } diff --git a/project/src/model-gateway/audited.gateway.spec.ts b/project/src/model-gateway/audited.gateway.spec.ts index 0044ab5f..7a631c31 100644 --- a/project/src/model-gateway/audited.gateway.spec.ts +++ b/project/src/model-gateway/audited.gateway.spec.ts @@ -2,6 +2,7 @@ import { describe, expect, it } from 'vitest'; import { z } from 'zod'; import type { ZodType } from 'zod'; import { ModelGateway } from './model-gateway.service'; +import type { StreamDelta } from './model-gateway.service'; import type { CompletionRequest, StructuredExtractionRequest } from './model-gateway.service'; import { AuditedModelGateway } from './audited.gateway'; import { createModelGateway } from './factory'; @@ -23,9 +24,9 @@ class RecordingGateway extends ModelGateway { async complete(_request: CompletionRequest) { return { text: SECRET_OUTPUT, usage: { inputTokens: 11, outputTokens: 7 } }; } - async *completeStream(_request: CompletionRequest): AsyncIterable { - yield SECRET_OUTPUT.slice(0, 5); - yield SECRET_OUTPUT.slice(5); + async *completeStream(_request: CompletionRequest): AsyncIterable { + yield { channel: 'text', text: SECRET_OUTPUT.slice(0, 5) } as const; + yield { channel: 'text', text: SECRET_OUTPUT.slice(5) } as const; } async extractStructured(schema: ZodType, _r: StructuredExtractionRequest) { return schema.parse({ claim: SECRET_OUTPUT }); diff --git a/project/src/model-gateway/audited.gateway.ts b/project/src/model-gateway/audited.gateway.ts index 5beb096e..b7c57f32 100644 --- a/project/src/model-gateway/audited.gateway.ts +++ b/project/src/model-gateway/audited.gateway.ts @@ -7,6 +7,7 @@ import type { ModelTier, StructuredExtractionRequest, VisionRequest, + StreamDelta, } from './model-gateway.service'; import type { ModelEgressAudit } from '../infrastructure/index'; import type { ZodType } from 'zod'; @@ -70,14 +71,16 @@ export class AuditedModelGateway extends ModelGateway { } } - async *completeStream(request: CompletionRequest): AsyncIterable { + async *completeStream(request: CompletionRequest): AsyncIterable { const started = Date.now(); const tier = request.tier ?? 'answer'; + // Structural counts only, as ever — but BOTH channels moved over the wire, + // so both count as egress volume (Part A). let outputChars = 0; let failure: unknown; try { for await (const delta of this.inner.completeStream(request)) { - outputChars += delta.length; + outputChars += delta.text.length; yield delta; } } catch (error) { diff --git a/project/src/model-gateway/budgeted.gateway.spec.ts b/project/src/model-gateway/budgeted.gateway.spec.ts index 55122afb..bbe5a3b6 100644 --- a/project/src/model-gateway/budgeted.gateway.spec.ts +++ b/project/src/model-gateway/budgeted.gateway.spec.ts @@ -2,6 +2,7 @@ import { describe, expect, it } from 'vitest'; import type { ZodType } from 'zod'; import { z } from 'zod'; import { ModelGateway } from './model-gateway.service'; +import type { StreamDelta } from './model-gateway.service'; import type { CompletionRequest, StructuredExtractionRequest } from './model-gateway.service'; import { BudgetedModelGateway } from './budgeted.gateway'; import { ModelBudgetExceededError } from './errors'; @@ -10,14 +11,18 @@ import type { ModelUsageMeter } from '../infrastructure/index'; /** A gateway that records what it was asked and returns canned output. */ class RecordingGateway extends ModelGateway { calls = 0; + /** Thinking deltas emitted before the text (Part A: charged like text). */ + thinkingDeltas: string[] = []; async complete(_request: CompletionRequest) { this.calls++; return { text: 'the answer text' }; } - async *completeStream(_request: CompletionRequest): AsyncIterable { + async *completeStream(_request: CompletionRequest): AsyncIterable { this.calls++; - yield 'hello '; - yield 'world'; + for (const thinking of this.thinkingDeltas) + yield { channel: 'thinking', text: thinking } as const; + yield { channel: 'text', text: 'hello ' } as const; + yield { channel: 'text', text: 'world' } as const; } async extractStructured( schema: ZodType, @@ -95,11 +100,32 @@ describe('BudgetedModelGateway', () => { const gateway = new BudgetedModelGateway(inner, meter); let text = ''; - for await (const delta of gateway.completeStream({ input: 'question' })) text += delta; + for await (const delta of gateway.completeStream({ input: 'question' })) text += delta.text; expect(text).toBe('hello world'); expect(meter.records).toHaveLength(1); }); + it('thinking is CHARGED (Part A ruling): reasoning tokens cost real money', async () => { + const plain = new RecordingGateway(); + const plainMeter = new FakeMeter(); + for await (const _ of new BudgetedModelGateway(plain, plainMeter).completeStream({ + input: 'question', + })) { + void _; + } + + const reasoning = new RecordingGateway(); + reasoning.thinkingDeltas = ['a long deliberation before the same answer ']; + const reasoningMeter = new FakeMeter(); + for await (const _ of new BudgetedModelGateway(reasoning, reasoningMeter).completeStream({ + input: 'question', + })) { + void _; + } + + expect(reasoningMeter.records[0]!.tokens).toBeGreaterThan(plainMeter.records[0]!.tokens); + }); + it('extractStructured stays validated and metered', async () => { const inner = new RecordingGateway(); const meter = new FakeMeter(); diff --git a/project/src/model-gateway/budgeted.gateway.ts b/project/src/model-gateway/budgeted.gateway.ts index ff6d494f..e2354146 100644 --- a/project/src/model-gateway/budgeted.gateway.ts +++ b/project/src/model-gateway/budgeted.gateway.ts @@ -6,6 +6,7 @@ import type { GatewayReachability, StructuredExtractionRequest, VisionRequest, + StreamDelta, } from './model-gateway.service'; import { ModelBudgetExceededError } from './errors'; import type { ModelUsageMeter } from '../infrastructure/index'; @@ -52,11 +53,14 @@ export class BudgetedModelGateway extends ModelGateway { return result; } - async *completeStream(request: CompletionRequest): AsyncIterable { + async *completeStream(request: CompletionRequest): AsyncIterable { const userId = await this.gate(); + // BOTH channels are charged (Part A ruling): thinking costs real tokens + // at the provider, and on a reasoning model it is most of them — leaving + // it out would under-report spend several times over. let output = ''; for await (const delta of this.inner.completeStream(request)) { - output += delta; + output += delta.text; yield delta; } await this.charge(userId, request.input, output); diff --git a/project/src/model-gateway/index.ts b/project/src/model-gateway/index.ts index aa59e1fb..a945ca30 100644 --- a/project/src/model-gateway/index.ts +++ b/project/src/model-gateway/index.ts @@ -37,6 +37,7 @@ export type { VisionUnavailableReason } from './errors'; export type { CompletionRequest, CompletionResult, + StreamDelta, StructuredExtractionRequest, VisionImage, VisionRequest, diff --git a/project/src/model-gateway/mistral.gateway.ts b/project/src/model-gateway/mistral.gateway.ts index f1ac27d1..db127ffe 100644 --- a/project/src/model-gateway/mistral.gateway.ts +++ b/project/src/model-gateway/mistral.gateway.ts @@ -7,6 +7,7 @@ import type { GatewayReachability, StructuredExtractionRequest, TokenUsage, + StreamDelta, } from './model-gateway.service'; import { ModelGatewayError, ModelGatewayNotConfiguredError } from './errors'; import type { ModelTier } from './model-gateway.service'; @@ -73,7 +74,7 @@ export class MistralModelGateway extends ModelGateway { return { text: contentToText(response.choices?.[0]?.message?.content), ...usageOf(response) }; } - async *completeStream(request: CompletionRequest): AsyncIterable { + async *completeStream(request: CompletionRequest): AsyncIterable { const stream = await callWithRetry('mistral', () => this.client.chat.stream({ model: this.models[request.tier ?? 'answer'], @@ -87,7 +88,7 @@ export class MistralModelGateway extends ModelGateway { ); for await (const event of stream) { const text = contentToText(event.data.choices?.[0]?.delta?.content); - if (text) yield text; + if (text) yield { channel: 'text', text }; } } @@ -175,7 +176,7 @@ export class UnconfiguredModelGateway extends ModelGateway { throw new ModelGatewayNotConfiguredError(); } // eslint-disable-next-line require-yield -- fails on first pull, like the rest - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new ModelGatewayNotConfiguredError(); } extractStructured(): Promise { diff --git a/project/src/model-gateway/model-gateway.service.ts b/project/src/model-gateway/model-gateway.service.ts index 02bd0513..3d280ca4 100644 --- a/project/src/model-gateway/model-gateway.service.ts +++ b/project/src/model-gateway/model-gateway.service.ts @@ -76,13 +76,29 @@ export interface StructuredExtractionRequest { tier?: ModelTier; } +/** + * One streamed delta (Part A of reasoning support): the seam yields + * channel-tagged text instead of bare strings, because a reasoning model + * produces two interleaved streams and only one of them is the answer. + * + * `thinking` is a CHANNEL, not content: it is displayed live and stored beside + * the chat message it explains, and it is never captured, cited, verified, or + * evaluated. A non-reasoning model only ever yields `text` deltas — the same + * bytes it always yielded, one field deeper. + */ +export interface StreamDelta { + channel: 'thinking' | 'text'; + text: string; +} + export abstract class ModelGateway { abstract complete(request: CompletionRequest): Promise; /** - * Streaming completion for the fast path (chat, spec §3.4): yields text deltas in - * order. Same seam rule as everything else — no provider types leak out. + * Streaming completion for the fast path (chat, spec §3.4): yields + * channel-tagged deltas in order (Part A). Same seam rule as everything + * else — no provider types leak out. */ - abstract completeStream(request: CompletionRequest): AsyncIterable; + abstract completeStream(request: CompletionRequest): AsyncIterable; /** * Requests JSON output, parses it, and validates it against the Zod schema. * The input type is free so schemas may use.default for omitted fields. diff --git a/project/src/model-gateway/openai.gateway.ts b/project/src/model-gateway/openai.gateway.ts index 3bbc5131..339762c1 100644 --- a/project/src/model-gateway/openai.gateway.ts +++ b/project/src/model-gateway/openai.gateway.ts @@ -5,6 +5,7 @@ import type { CompletionResult, GatewayReachability, ModelTier, + StreamDelta, StructuredExtractionRequest, TokenUsage, VisionRequest, @@ -267,9 +268,17 @@ export class OpenAiCompatibleModelGateway extends ModelGateway { }; } - async *completeStream(request: CompletionRequest): AsyncIterable { + /** + * Streams both channels (Part A of reasoning support): a reasoning model + * interleaves `reasoning_content` (llama.cpp, DeepSeek) / `reasoning` + * (OpenAI-style) / `thinking` (Ollama) deltas with `content` deltas, and the + * seam labels each. A thinking delta also marks the model as reasoning, so + * the maxTokens headroom (Part B) arms from live chat traffic too. + */ + async *completeStream(request: CompletionRequest): AsyncIterable { const tier = request.tier ?? 'answer'; - const response = await this.call(tier, this.modelFor(tier), (signal) => + const model = this.modelFor(tier); + const response = await this.call(tier, model, (signal) => postStream( `${this.baseUrl}/chat/completions`, this.headers, @@ -279,14 +288,22 @@ export class OpenAiCompatibleModelGateway extends ModelGateway { ); for await (const data of sseData(response)) { if (data === '[DONE]') break; - let event: { choices?: { delta?: { content?: unknown } }[] }; + let event: { choices?: { delta?: ChatMessage }[] }; try { event = JSON.parse(data) as typeof event; } catch { continue; } - const text = contentToText(event.choices?.[0]?.delta?.content); - if (text) yield text; + const delta = event.choices?.[0]?.delta; + const thinking = contentToText( + delta?.reasoning_content ?? delta?.reasoning ?? delta?.thinking, + ); + if (thinking) { + this.reasoningModels.add(model); + yield { channel: 'thinking', text: thinking }; + } + const text = contentToText(delta?.content); + if (text) yield { channel: 'text', text }; } } diff --git a/project/src/model-gateway/provider-adapters.spec.ts b/project/src/model-gateway/provider-adapters.spec.ts index 8a56270d..47e8b4ac 100644 --- a/project/src/model-gateway/provider-adapters.spec.ts +++ b/project/src/model-gateway/provider-adapters.spec.ts @@ -2,6 +2,7 @@ import { afterEach, describe, expect, it, vi } from 'vitest'; import { z } from 'zod'; import type { ZodType } from 'zod'; import { ModelGateway } from './model-gateway.service'; +import type { StreamDelta } from './model-gateway.service'; import type { CompletionRequest, CompletionResult, @@ -346,9 +347,9 @@ class FakeGateway extends ModelGateway { this.seen.push(`complete:${request.tier ?? 'answer'}`); return { text: this.name, usage: { inputTokens: 100, outputTokens: 50 } }; } - async *completeStream(request: CompletionRequest): AsyncIterable { + async *completeStream(request: CompletionRequest): AsyncIterable { this.seen.push(`stream:${request.tier ?? 'answer'}`); - yield this.name; + yield { channel: 'text', text: this.name } as const; } async extractStructured( schema: ZodType, diff --git a/project/src/model-gateway/provider-config.spec.ts b/project/src/model-gateway/provider-config.spec.ts index d7fbfea0..b990dc2c 100644 --- a/project/src/model-gateway/provider-config.spec.ts +++ b/project/src/model-gateway/provider-config.spec.ts @@ -290,7 +290,7 @@ describe('reasoning_headroom_config — the maxTokens multiplier (Part B)', () = it('defaults to 4 and never joins the configuration id', () => { const providers = resolve({ COGETO_MISTRAL_API_KEY: 'k' }); expect(providers.reasoningHeadroom).toBe(4); - expect(providers.id).toBe('mistral-default'); // no reasoning marker (Part C) + expect(providers.id).toBe('mistral-default'); // marker joins at emission, never here }); it('is configurable', () => { diff --git a/project/src/model-gateway/provider-config.ts b/project/src/model-gateway/provider-config.ts index fad64395..d9cbe114 100644 --- a/project/src/model-gateway/provider-config.ts +++ b/project/src/model-gateway/provider-config.ts @@ -410,8 +410,8 @@ export function resolveModelProviders( // Reasoning headroom (Part B). Deliberately NOT part of the configuration id: // the id fingerprints what a measurement ran against, and whether a binding // reasons is a PROBED runtime fact, not configuration — the id is derived - // before any probe can run. The fingerprint marker is Part C's, alongside the - // channel, when a reasoning-routed configuration can publish a measurement. + // before any probe can run. The fingerprint marker is appended at trust + // EMISSION time from the probe (Part C, configurationForEmission), never here. const headroomRaw = read(env, 'COGETO_REASONING_HEADROOM'); let reasoningHeadroom = 4; if (headroomRaw !== undefined) { diff --git a/project/src/model-gateway/reasoning.spec.ts b/project/src/model-gateway/reasoning.spec.ts index b8b50889..c0498d47 100644 --- a/project/src/model-gateway/reasoning.spec.ts +++ b/project/src/model-gateway/reasoning.spec.ts @@ -395,3 +395,55 @@ describe('reasoning_probe_unreachable_error', () => { expect(error.reason).toBe('reasoning_exhausted'); }); }); + +describe('reasoning_stream_channel', () => { + const sse = (...events: object[]): Response => + new Response( + new ReadableStream({ + start(controller) { + const encoder = new TextEncoder(); + for (const event of events) { + controller.enqueue(encoder.encode(`data: ${JSON.stringify(event)}\n\n`)); + } + controller.enqueue(encoder.encode('data: [DONE]\n\n')); + controller.close(); + }, + }), + { status: 200, headers: { 'content-type': 'text/event-stream' } }, + ); + const delta = (fields: object): object => ({ choices: [{ delta: fields }] }); + + it('yields thinking and text as labeled channels, in order, and arms headroom', async () => { + const { calls } = stubFetch( + sse( + delta({ reasoning_content: 'Thinking about it. ' }), + delta({ reasoning_content: 'Still thinking. ' }), + delta({ content: 'OK' }), + delta({ content: '.' }), + ), + chat('follow-up'), + ); + const g = gateway(); + const seen: { channel: string; text: string }[] = []; + for await (const d of g.completeStream({ input: 'q', maxTokens: 64 })) seen.push(d); + expect(seen).toEqual([ + { channel: 'thinking', text: 'Thinking about it. ' }, + { channel: 'thinking', text: 'Still thinking. ' }, + { channel: 'text', text: 'OK' }, + { channel: 'text', text: '.' }, + ]); + // A thinking delta marked the model: the NEXT capped call gets headroom. + await g.complete({ input: 'q', maxTokens: 64 }); + expect(calls[1]!.body.max_tokens).toBe(256); + }); + + it('a non-reasoning stream yields text deltas only — the same bytes as ever', async () => { + stubFetch(sse(delta({ content: 'plain ' }), delta({ content: 'answer' }))); + const seen: { channel: string; text: string }[] = []; + for await (const d of gateway().completeStream({ input: 'q' })) seen.push(d); + expect(seen).toEqual([ + { channel: 'text', text: 'plain ' }, + { channel: 'text', text: 'answer' }, + ]); + }); +}); diff --git a/project/src/model-gateway/redacting.gateway.ts b/project/src/model-gateway/redacting.gateway.ts index eeef3100..f80c19a0 100644 --- a/project/src/model-gateway/redacting.gateway.ts +++ b/project/src/model-gateway/redacting.gateway.ts @@ -6,6 +6,7 @@ import type { CompletionResult, GatewayReachability, StructuredExtractionRequest, + StreamDelta, } from './model-gateway.service'; import type { RedactionPort } from './redaction-client'; import { reidentifyDeep, reidentifyStream, reidentifyText } from './redaction-utils'; @@ -39,9 +40,24 @@ export class RedactingModelGateway extends ModelGateway { return { ...result, text: reidentifyText(result.text, mapping) }; } - async *completeStream(request: CompletionRequest): AsyncIterable { + /** + * Thinking is STRIPPED under redaction (Part A) — the vision posture, fail + * closed. Re-identification maps pseudonyms back into the TEXT the user + * reads; a reasoning model's deliberation interleaves pseudonym fragments + * the flush logic cannot bound, and half-re-identified reasoning about + * redacted entities is exactly what redaction promises can never surface. + * So under redaction the thinking channel does not exist: no delta, no + * empty disclosure, nothing. + */ + async *completeStream(request: CompletionRequest): AsyncIterable { const { text: input, mapping } = await this.redactor.pseudonymize(request.input); - yield* reidentifyStream(this.inner.completeStream({ ...request, input }), mapping); + const inner = this.inner.completeStream({ ...request, input }); + const textOnly = async function* (): AsyncIterable { + for await (const delta of inner) if (delta.channel === 'text') yield delta.text; + }; + for await (const text of reidentifyStream(textOnly(), mapping)) { + yield { channel: 'text', text }; + } } async extractStructured( diff --git a/project/src/model-gateway/redaction.spec.ts b/project/src/model-gateway/redaction.spec.ts index 95e71745..2a24fa67 100644 --- a/project/src/model-gateway/redaction.spec.ts +++ b/project/src/model-gateway/redaction.spec.ts @@ -2,6 +2,7 @@ import { describe, expect, it } from 'vitest'; import { z } from 'zod'; import type { ZodType } from 'zod'; import { ModelGateway } from './model-gateway.service'; +import type { StreamDelta } from './model-gateway.service'; import type { CompletionRequest, CompletionResult, @@ -54,6 +55,8 @@ class RecordingUpstream extends ModelGateway { lastInput: string | null = null; embedInputs: string[] = []; called = false; + /** Thinking deltas the fake emits before the text (Part A tests). */ + thinkingDeltas: string[] = []; async complete(request: CompletionRequest): Promise { this.called = true; @@ -61,10 +64,13 @@ class RecordingUpstream extends ModelGateway { // The model echoes the pseudonymized entities in its answer. return { text: `Noted: ${request.input}` }; } - async *completeStream(request: CompletionRequest): AsyncIterable { + async *completeStream(request: CompletionRequest): AsyncIterable { this.called = true; this.lastInput = request.input; - for (const piece of ['Sending to ', '[person2]', ' now.']) yield piece; + for (const thinking of this.thinkingDeltas) + yield { channel: 'thinking', text: thinking } as const; + for (const piece of ['Sending to ', '[person2]', ' now.']) + yield { channel: 'text', text: piece } as const; } async extractStructured( _schema: ZodType, @@ -131,10 +137,22 @@ describe('redaction_in_path', () => { const gateway = new RedactingModelGateway(upstream, new FakeRedactor()); let text = ''; - for await (const delta of gateway.completeStream({ input: NOTE })) text += delta; + for await (const delta of gateway.completeStream({ input: NOTE })) text += delta.text; expect(text).toBe('Sending to Marko now.'); }); + + it('thinking is STRIPPED under redaction — fail closed, the vision posture (Part A)', async () => { + const upstream = new RecordingUpstream(); + upstream.thinkingDeltas = ['deliberating about [person1]... ']; + const gateway = new RedactingModelGateway(upstream, new FakeRedactor()); + + const seen: { channel: string; text: string }[] = []; + for await (const delta of gateway.completeStream({ input: NOTE })) seen.push(delta); + + expect(seen.every((delta) => delta.channel === 'text')).toBe(true); + expect(seen.map((delta) => delta.text).join('')).toBe('Sending to Marko now.'); + }); }); describe('redaction_fail_closed', () => { diff --git a/project/src/model-gateway/routed.gateway.ts b/project/src/model-gateway/routed.gateway.ts index 3e3acac7..8b57b59f 100644 --- a/project/src/model-gateway/routed.gateway.ts +++ b/project/src/model-gateway/routed.gateway.ts @@ -7,6 +7,7 @@ import type { GatewayReachability, StructuredExtractionRequest, VisionRequest, + StreamDelta, } from './model-gateway.service'; export interface TierRoutes { @@ -34,7 +35,7 @@ export class TierRoutedModelGateway extends ModelGateway { return this.routes[request.tier ?? 'answer'].complete(request); } - completeStream(request: CompletionRequest): AsyncIterable { + completeStream(request: CompletionRequest): AsyncIterable { return this.routes[request.tier ?? 'answer'].completeStream(request); } diff --git a/project/src/model-gateway/vision-probe.spec.ts b/project/src/model-gateway/vision-probe.spec.ts index 7a3e3518..94b7af81 100644 --- a/project/src/model-gateway/vision-probe.spec.ts +++ b/project/src/model-gateway/vision-probe.spec.ts @@ -1,5 +1,6 @@ import { describe, expect, it } from 'vitest'; import { ModelGateway } from './model-gateway.service'; +import type { StreamDelta } from './model-gateway.service'; import type { CompletionResult, VisionRequest } from './model-gateway.service'; import { VisionUnavailableError } from './errors'; import { probeImagePng, probeVision } from './vision-probe'; @@ -25,7 +26,7 @@ class StubGateway extends ModelGateway { throw new Error('unused'); } // eslint-disable-next-line require-yield -- unused by the probe - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } extractStructured(): never { diff --git a/project/src/research/research-conclusion.integration.spec.ts b/project/src/research/research-conclusion.integration.spec.ts index 06aadb9c..14ef7eff 100644 --- a/project/src/research/research-conclusion.integration.spec.ts +++ b/project/src/research/research-conclusion.integration.spec.ts @@ -9,6 +9,7 @@ import type { TestDatabase, TestQdrant } from '../testing/index'; import { createMemoryStore, MemoryReconciliation } from '../memory/index'; import type { MemoryObjectStore, MemoryStore } from '../memory/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { CompletionResult, StructuredExtractionRequest } from '../model-gateway/index'; import { createSuppressedFactLog, @@ -76,7 +77,7 @@ class ConcludeGateway extends ModelGateway { return { text: 'The harbour day fee is 12 EUR. [W1] Permits cost 40 EUR. [W2] [W9]' }; } // eslint-disable-next-line require-yield -- unused - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(texts: string[]): Promise { diff --git a/project/src/research/research-flow.integration.spec.ts b/project/src/research/research-flow.integration.spec.ts index 5d08ad07..47ab31fd 100644 --- a/project/src/research/research-flow.integration.spec.ts +++ b/project/src/research/research-flow.integration.spec.ts @@ -9,6 +9,7 @@ import type { TestDatabase, TestQdrant } from '../testing/index'; import { createMemoryStore, MemoryReconciliation } from '../memory/index'; import type { MemoryObjectStore, MemoryStore } from '../memory/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { CompletionResult, StructuredExtractionRequest } from '../model-gateway/index'; import { createSuppressedFactLog, @@ -60,7 +61,7 @@ class FlowGateway extends ModelGateway { return { text: this.answerText }; } // eslint-disable-next-line require-yield -- unused - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(texts: string[]): Promise { diff --git a/project/src/research/research-gate.integration.spec.ts b/project/src/research/research-gate.integration.spec.ts index 1e55f395..c3ff3ea4 100644 --- a/project/src/research/research-gate.integration.spec.ts +++ b/project/src/research/research-gate.integration.spec.ts @@ -6,6 +6,7 @@ import { startTestDatabase } from '../testing/index'; import type { TestDatabase } from '../testing/index'; import type { MemoryObjectStore } from '../memory/index'; import { ModelGateway } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import { ResearchService } from './research.service'; import { WebDiscoveryService } from './web-discovery.service'; import { WebFetchService } from './web-fetch'; @@ -41,7 +42,7 @@ class MinimiserGateway extends ModelGateway { throw new Error('unused'); } // eslint-disable-next-line require-yield -- unused - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(): Promise { diff --git a/project/src/research/research-minimise.spec.ts b/project/src/research/research-minimise.spec.ts index 0265932f..d11d731a 100644 --- a/project/src/research/research-minimise.spec.ts +++ b/project/src/research/research-minimise.spec.ts @@ -3,6 +3,7 @@ import * as path from 'node:path'; import { describe, expect, it } from 'vitest'; import type { ZodType } from 'zod'; import { ModelGateway } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { StructuredExtractionRequest } from '../model-gateway/index'; import { minimiseQuery, RESEARCH_MINIMISE_PROMPT } from './research-minimise'; @@ -26,7 +27,7 @@ class ScriptedMinimiser extends ModelGateway { throw new Error('unused'); } // eslint-disable-next-line require-yield -- unused - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(): Promise { diff --git a/project/src/research/web-research.integration.spec.ts b/project/src/research/web-research.integration.spec.ts index 85f4f343..64d08a75 100644 --- a/project/src/research/web-research.integration.spec.ts +++ b/project/src/research/web-research.integration.spec.ts @@ -11,6 +11,7 @@ import type { TestDatabase, TestQdrant } from '../testing/index'; import { createMemoryStore, MemoryReconciliation } from '../memory/index'; import type { MemoryObjectStore, MemoryStore } from '../memory/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { StructuredExtractionRequest } from '../model-gateway/index'; import { createSuppressedFactLog, @@ -70,7 +71,7 @@ class RecordingGateway extends ModelGateway { throw new Error('the pipeline must never use the answer tier'); } // eslint-disable-next-line require-yield -- must never be reached - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { this.answerTierCalls += 1; throw new Error('the pipeline must never use the answer tier'); } diff --git a/project/src/retrieval/query-rewrite.spec.ts b/project/src/retrieval/query-rewrite.spec.ts index ee959fa2..995700d3 100644 --- a/project/src/retrieval/query-rewrite.spec.ts +++ b/project/src/retrieval/query-rewrite.spec.ts @@ -2,6 +2,7 @@ import { describe, expect, it } from 'vitest'; import type { ZodType } from 'zod'; import type { Principal } from '@cogeto/shared'; import { ModelGateway } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { StructuredExtractionRequest } from '../model-gateway/index'; import type { MemoryRow, MemoryStore } from '../memory/index'; import { RetrievalService } from './retrieval.service'; @@ -43,7 +44,7 @@ class RewriteGateway extends ModelGateway { throw new Error('not used'); } // eslint-disable-next-line require-yield -- not used - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('not used'); } async embed(texts: string[]): Promise { diff --git a/project/src/settings/context-suggestions.spec.ts b/project/src/settings/context-suggestions.spec.ts index f9969daf..ecff52c0 100644 --- a/project/src/settings/context-suggestions.spec.ts +++ b/project/src/settings/context-suggestions.spec.ts @@ -5,6 +5,7 @@ import { startTestDatabase } from '../testing/index'; import type { TestDatabase } from '../testing/index'; import { readAuditEntries, UserContextService } from '../infrastructure/index'; import { ModelGateway } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { MemoryRow, MemoryStore } from '../memory/index'; import { ContextSuggestionsService } from './context-suggestions.service'; @@ -59,7 +60,7 @@ class VerdictGateway extends ModelGateway { throw new Error('unexpected'); } // eslint-disable-next-line require-yield - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unexpected'); } async embed(texts: string[]): Promise { diff --git a/project/src/skills/skill-brief.integration.spec.ts b/project/src/skills/skill-brief.integration.spec.ts index 6ae43b40..19874385 100644 --- a/project/src/skills/skill-brief.integration.spec.ts +++ b/project/src/skills/skill-brief.integration.spec.ts @@ -10,6 +10,7 @@ import type { TestDatabase, TestQdrant } from '../testing/index'; import { createMemoryStore, createMemorySystemStore, MemoryReconciliation } from '../memory/index'; import type { MemoryObjectStore, MemoryStore, MemorySystemStore } from '../memory/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { CompletionRequest, CompletionResult, @@ -86,7 +87,7 @@ class BriefGateway extends ModelGateway { return { text: this.briefText }; } // eslint-disable-next-line require-yield -- unused - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(texts: string[]): Promise { diff --git a/project/src/skills/skill-run.integration.spec.ts b/project/src/skills/skill-run.integration.spec.ts index 75ec9d58..489dcebb 100644 --- a/project/src/skills/skill-run.integration.spec.ts +++ b/project/src/skills/skill-run.integration.spec.ts @@ -10,6 +10,7 @@ import type { TestDatabase, TestQdrant } from '../testing/index'; import { createMemoryStore, createMemorySystemStore, MemoryReconciliation } from '../memory/index'; import type { MemoryObjectStore, MemoryStore, MemorySystemStore } from '../memory/index'; import { ModelGateway, ModelGatewayError } from '../model-gateway/index'; +import type { StreamDelta } from '../model-gateway/index'; import type { CompletionRequest, CompletionResult, @@ -95,7 +96,7 @@ class SkillGateway extends ModelGateway { return { text: this.briefText }; } // eslint-disable-next-line require-yield -- unused - async *completeStream(): AsyncIterable { + async *completeStream(): AsyncIterable { throw new Error('unused'); } async embed(texts: string[]): Promise { diff --git a/project/web/src/locales/de/chat.json b/project/web/src/locales/de/chat.json index e71a9ad5..71144b4d 100644 --- a/project/web/src/locales/de/chat.json +++ b/project/web/src/locales/de/chat.json @@ -96,5 +96,9 @@ "approvedNote_other": "Hinweis: {{count}} dieser Erinnerungen wurden von Ihnen genehmigt.", "alternative": "Beim Archivieren bleibt stattdessen alles erhalten. Dies kann nicht rückgängig gemacht werden." } + }, + "reasoning": { + "heading": "Thinking", + "streaming": "Thinking, streaming live" } } diff --git a/project/web/src/locales/en/chat.json b/project/web/src/locales/en/chat.json index b661d3ae..f1e4ca2b 100644 --- a/project/web/src/locales/en/chat.json +++ b/project/web/src/locales/en/chat.json @@ -96,5 +96,9 @@ "approvedNote_other": "Note: {{count}} of those memories were approved by you.", "alternative": "Archiving keeps everything instead. This cannot be undone." } + }, + "reasoning": { + "heading": "Thinking", + "streaming": "Thinking, streaming live" } } diff --git a/project/web/src/locales/fr/chat.json b/project/web/src/locales/fr/chat.json index ec7dafaa..d33a0b0f 100644 --- a/project/web/src/locales/fr/chat.json +++ b/project/web/src/locales/fr/chat.json @@ -100,5 +100,9 @@ "approvedNote_other": "Remarque : vous avez approuvé {{count}} de ces souvenirs.", "alternative": "L’archivage conserve tout. Cette action est irréversible." } + }, + "reasoning": { + "heading": "Thinking", + "streaming": "Thinking, streaming live" } } diff --git a/project/web/src/locales/hr/chat.json b/project/web/src/locales/hr/chat.json index 5c6e9d9b..9770bd5d 100644 --- a/project/web/src/locales/hr/chat.json +++ b/project/web/src/locales/hr/chat.json @@ -100,5 +100,9 @@ "approvedNote_other": "Napomena: vi ste odobrili {{count}} od tih sjećanja.", "alternative": "Arhiviranjem se umjesto toga sve zadržava. Ovo se ne može poništiti." } + }, + "reasoning": { + "heading": "Thinking", + "streaming": "Thinking, streaming live" } } diff --git a/project/web/src/pages/Chat.tsx b/project/web/src/pages/Chat.tsx index c8c50662..9518dbfb 100644 --- a/project/web/src/pages/Chat.tsx +++ b/project/web/src/pages/Chat.tsx @@ -227,6 +227,26 @@ function RememberAction({ session, messageId }: { session: Session; messageId: s } /** The question as a confident heading. */ +/** + * The reasoning channel (Part C of reasoning support): the model's own + * deliberation, collapsed by default, streaming live while it thinks and + * reopenable on a stored answer. Renders nothing when there is no thinking — + * a non-reasoning model must leave no empty affordance. + */ +function ReasoningDisclosure({ text, streaming = false }: { text: string; streaming?: boolean }) { + const { t } = useTranslation('chat'); + return ( +
+ + {streaming ? t('reasoning.streaming') : t('reasoning.heading')} + +
+        {text}
+      
+
+ ); +} + function AskHeading({ time, children }: { time?: string; children: ReactNode }) { const { t } = useTranslation('chat'); return ( @@ -318,7 +338,12 @@ function CogetoMark() { */ const SUGGESTED_PROMPT_IDS = ['promisedThisWeek', 'openCommitments', 'changedSince', 'whoInvolved']; -type ChatMessage = { id: string; role: 'user' | 'assistant'; content: string }; +type ChatMessage = { + id: string; + role: 'user' | 'assistant'; + content: string; + thinking?: string | null; +}; type Turn = { key: string; question?: ChatMessage; answer?: ChatMessage }; /** Pair the alternating message stream into ask → briefing turns. */ @@ -380,6 +405,9 @@ export function Chat({ session }: { session: Session }) { const [failMessage, setFailMessage] = useState(null); const [liveQuestion, setLiveQuestion] = useState(null); const [liveText, setLiveText] = useState(''); + /** The reasoning channel, streamed live (Part C): shown collapsed above the + * answer, and nothing renders when the model does not think. */ + const [liveThinking, setLiveThinking] = useState(''); const [liveFacts, setLiveFacts] = useState([]); /** The latest answer's research offer (0046) — ephemeral, cleared on the next ask. */ const [offer, setOffer] = useState(null); @@ -447,6 +475,7 @@ export function Chat({ session }: { session: Session }) { setBusy(false); setLiveQuestion(null); setLiveText(''); + setLiveThinking(''); setLiveFacts([]); setOffer(null); setInlineRun(null); @@ -492,6 +521,7 @@ export function Chat({ session }: { session: Session }) { setDraft(''); setLiveQuestion(content); setLiveText(''); + setLiveThinking(''); setLiveFacts([]); setOffer(null); setSkillRunId(null); @@ -548,6 +578,7 @@ export function Chat({ session }: { session: Session }) { void queryClient.invalidateQueries({ queryKey: ['conversations'] }); setLiveQuestion(null); setLiveText(''); + setLiveThinking(''); setLiveFacts([]); setBusy(false); }; @@ -663,6 +694,9 @@ export function Chat({ session }: { session: Session }) { {turn.answer && (
+ {turn.answer.thinking && ( + + )} {liveQuestion} + {liveThinking && ( + + )}
{liveText ? ( - ) : ( + ) : liveThinking ? null : ( 0