Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,10 +188,15 @@ Re-anchoring after an edit is the reprocess action, superseding via reconcile.
The plan-named golden cases gate it and the eval fixtures were refreshed
Mistral-routed: [`docs/features/anchoring.md`](docs/features/anchoring.md).

Work proceeds through the V2 plan in order, with one owner-approved insertion:
reasoning-model support Part B (the probed `reasoning` capability and the
maxTokens headroom) landed 2026-08-04; Parts A (the thinking channel) and C
(storage and UI) follow after V2.1 item 4.2.
Work proceeds through the V2 plan in order, with one owner-approved insertion,
now complete: **reasoning-model support** (Parts A, B and C, 2026-08-04).
Thinking is a CHANNEL, not content: `completeStream` yields channel-tagged
deltas, the budget charges thinking, redaction strips it fail-closed, chat
stores it (`chat_message.thinking`, migration 0044) and shows it as a
collapsed live disclosure, the answer-redaction cascade erases it with its
answer, and the trust-artifact id gains a probed `--reasoning` marker at
emission time only. Never captured, cited, verified, or evaluated:
[`docs/features/reasoning.md`](docs/features/reasoning.md).

## Delivery loop

Expand Down
9 changes: 5 additions & 4 deletions docs/features/capabilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,11 @@ The probe reads the reasoning field only as a yes/no. The thinking text is
discarded in the adapter: it is never stored, verified, cited, displayed or
evaluated, and it can never reach the JSON parser behind structured
extraction. Displaying it as a channel is Parts A and C of the reasoning
design, deliberately not this. The configuration fingerprint does not yet
carry a reasoning marker for the same reason: whether a binding reasons is a
probed runtime fact, and the fingerprint is derived before any probe can run;
the marker lands with the channel in Part C.
design, deliberately not this. The configuration fingerprint carries the
reasoning marker at EMISSION time (Parts A and C): whether a binding reasons
is a probed runtime fact the static resolver cannot know, so the trust-artifact
id gains `--reasoning` from the same probe when a measurement is emitted, and
never anywhere else. Full narrative: [`reasoning.md`](reasoning.md).

## Thresholds

Expand Down
103 changes: 103 additions & 0 deletions docs/features/reasoning.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# Reasoning models: the thinking channel

**Owner-approved design, delivered in three parts: B (2026-08-04, the probed
capability and the maxTokens headroom), then A and C together (the channel,
its storage, and the display). Migration 0044.**

A reasoning model deliberates before it answers, and serves that deliberation
as a separate stream beside the answer. Cogeto supports this first-class
rather than working around it, under one governing principle: **thinking is a
CHANNEL, not content.** It is displayed, live and afterwards, because hiding
what the instance's own model said while deciding would be the opposite of
the product's posture; and it is never a source, never a citation, never a
measurement.

## The three honesty rules

1. **Thinking is never a source.** "Remember this" captures the user's own
words; capture reads user rows only, structurally. Thinking cannot be
captured, cited, verified, or contradicted, and the answer sanitizer and
citation machinery read `content` alone.
2. **Thinking is never evaluated.** The golden set and the trust artifact
measure answers. The eval cache records the text channel only, so a
fixture cannot freeze what the harness must not measure.
3. **A run with thinking on is a different measurement.** The trust-artifact
configuration id carries a `--reasoning` marker, appended at EMISSION time
from the same probe the capability panel uses, because whether a binding
reasons is a runtime fact the static resolver cannot know. A Mistral-routed
run probes off and emits the unchanged id, so every existing artifact,
gate, and cached fixture is untouched.

## Part B: the probed capability (delivered first, separately)

Whether a configuration reasons cannot be read off a model name: the same
weights are served both ways. So it is probed, like vision: a trivial prompt
at boot and per registry window, the answer surfaced on the capability panel
and boot banner. When on, every `maxTokens` is multiplied by
`COGETO_REASONING_HEADROOM` (default 4) for the bindings that reasoned,
because a cap sized for an answer is not sized for an answer plus its
deliberation; the exhausted-budget failure has its own named error instead of
masquerading as "returned no text". Details:
[`capabilities.md`](capabilities.md).

## Part A: the channel

`ModelGateway.completeStream` yields channel-tagged deltas
(`{channel: 'thinking' | 'text', text}`). The OpenAI-compatible adapter
surfaces `reasoning_content` (llama.cpp, DeepSeek), `reasoning` (OpenAI-style)
and `thinking` (Ollama) deltas on the thinking channel, and a thinking delta
also arms the Part B headroom, so live chat traffic teaches the adapter too.
Mistral yields text only; Anthropic maps `thinking_delta` blocks although
Cogeto never requests extended thinking. A non-reasoning model yields the
same bytes it always did, one field deeper.

The four decorators keep their contracts, two by explicit ruling:

- **The budget charges BOTH channels.** Thinking costs real tokens at the
provider, and on the reference reasoning model it is most of them; a
meter that ignored it would under-report spend several times over.
- **Redaction strips thinking, fail closed**, the vision posture.
Re-identification maps pseudonyms back into the text a user reads; a
reasoning model's deliberation interleaves pseudonym fragments the flush
logic cannot bound. Under redaction the thinking channel does not exist:
no delta, no empty disclosure.
- The egress audit counts both channels' characters (structural, never
content); the tier router dispatches untouched.

## Part C: stored, streamed, shown

`chat_message.thinking` (migration 0044, nullable text) stores the
deliberation beside the answer it produced. A `thinking` SSE event streams
deltas live, interleaved with `token` events. The chat UI renders a collapsed
**Thinking** disclosure above the answer (streaming while the model
deliberates, expandable, reopenable on a stored answer) and renders NOTHING
when there is no thinking: a non-reasoning model leaves no empty affordance.

Erasure follows the answer. The answer-redaction cascade nulls `thinking` in
the same UPDATE that overwrites a citing answer's content, because reasoning
ABOUT an erased memory must not survive the citation that grounded it; row
deletion (message, conversation, source cascade) removes it implicitly, and
receipts are unchanged.

## What is deliberately NOT here

- No thinking in the answer prompt, the reply drafts, research synthesis, or
any prompt assembly: the channel ends at the disclosure and the column.
- No evaluation surface: the harness consumes answer text and never reads the
column or the channel.
- No configuration flag: display follows the model. A non-reasoning
configuration behaves byte-identically to the pre-channel system, held to
the reader-seam standard and tested as such.

## Tests

- `model-gateway/reasoning.spec.ts` (`reasoning_stream_channel`): labeled
channels in order, headroom armed from a stream, non-reasoning streams
unchanged.
- `model-gateway/redaction.spec.ts`: thinking stripped under redaction.
- `model-gateway/budgeted.gateway.spec.ts`: thinking charged.
- `chat/chat.integration.spec.ts` (`chat_thinking`): the SSE event, the
stored column, the DTO, and the answer never containing the deliberation.
- `chat/chat-answer-cascade.integration.spec.ts`: thinking nulled with the
redacted answer.
- `entrypoints/trust-scores.spec.ts`: the emission marker, on and off.
8 changes: 8 additions & 0 deletions project/shared/src/chat.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,10 @@ export interface ChatMessageDto {
id: string;
role: ChatRole;
content: string;
/** The model's displayed deliberation (Part C of reasoning support), when a
* reasoning model produced one. A CHANNEL, never content: not capturable,
* not citable, not evaluated. Null for user rows and non-reasoning models. */
thinking: string | null;
createdAt: string;
}

Expand Down Expand Up @@ -105,6 +109,10 @@ export interface ChatResearchProposalRef {

/** Server-sent events on POST /api/chat, in order: sources → token* → done. */
export type ChatStreamEvent =
/** A reasoning delta (Part C): displayed live in the collapsed Thinking
* disclosure. Interleaves with `token` events; absent entirely for a
* non-reasoning model. */
| { type: 'thinking'; text: string }
| { type: 'sources'; facts: ChatFactDto[] }
| { type: 'token'; text: string }
| {
Expand Down
18 changes: 16 additions & 2 deletions project/src/chat/chat-answer-cascade.integration.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,12 @@ describe(' chat-answer cascade (integration: real Postgres, real saga)', () => {

// Messages need a container since — one per owner is enough here.
const conversationIds = new Map<string, string>();
const insertMessage = async (ownerId: string, role: 'user' | 'assistant', content: string) => {
const insertMessage = async (
ownerId: string,
role: 'user' | 'assistant',
content: string,
thinking: string | null = null,
) => {
let conversationId = conversationIds.get(ownerId);
if (!conversationId) {
const [conv] = await tdb.db.insert(conversation).values({ ownerId }).returning();
Expand All @@ -50,7 +55,7 @@ describe(' chat-answer cascade (integration: real Postgres, real saga)', () => {
}
const [row] = await tdb.db
.insert(chatMessage)
.values({ ownerId, conversationId, role, content })
.values({ ownerId, conversationId, role, content, thinking })
.returning({ id: chatMessage.id });
return row!.id;
};
Expand Down Expand Up @@ -82,6 +87,8 @@ describe(' chat-answer cascade (integration: real Postgres, real saga)', () => {
userA.userId,
'assistant',
`The renewal is agreed {{cite:${m1.id}}}.`,
// Reasoning about the memory being erased (Part C): must go with it.
'The Novira renewal fact seems most relevant here.',
);
const citingOther = await insertMessage(
userA.userId,
Expand All @@ -105,6 +112,13 @@ describe(' chat-answer cascade (integration: real Postgres, real saga)', () => {
// owner's AND the peer's (erasure is erasure); the timeline rows survive.
expect(await contentOf(citing)).toBe(CHAT_ANSWER_REDACTED);
expect(await contentOf(peerCiting)).toBe(CHAT_ANSWER_REDACTED);
// The thinking channel goes with the answer it deliberated (Part C):
// reasoning ABOUT an erased memory must not survive its citation.
const { rows: thinkingRows } = await tdb.pool.query<{ thinking: string | null }>(
`SELECT thinking FROM chat_message WHERE id = $1`,
[citing],
);
expect(thinkingRows[0]!.thinking).toBeNull();
// An answer citing a different memory and the user's own words are untouched.
expect(await contentOf(citingOther)).toContain(other.id);
expect(await contentOf(userTurn)).toContain(m1.id);
Expand Down
8 changes: 7 additions & 1 deletion project/src/chat/chat-answer-cascade.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,13 @@ export class ChatAnswerCascade implements DerivedCascade {
);
const updated = await tx
.update(chatMessage)
.set({ content: CHAT_ANSWER_REDACTED })
.set({
content: CHAT_ANSWER_REDACTED,
// Reasoning ABOUT an erased memory goes with the citation that
// grounded it (Part C): the thinking channel is content-bearing and
// must not outlive the answer it deliberated.
thinking: null,
})
.where(and(eq(chatMessage.role, 'assistant'), or(...citeMatches)))
.returning({ id: chatMessage.id });
redacted += updated.length;
Expand Down
3 changes: 2 additions & 1 deletion project/src/chat/chat-capture.integration.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { createIngestionPipeline, createSuppressedFactLog } from '../ingestion/i
import type { IngestionPipeline } from '../ingestion/index';
import { UserDirectory } from '../identity/index';
import { ModelGateway, ModelGatewayError } from '../model-gateway/index';
import type { StreamDelta } from '../model-gateway/index';
import type { StructuredExtractionRequest } from '../model-gateway/index';
import { RetrievalService } from '../retrieval/index';
import { ChatService } from './chat.service';
Expand All @@ -30,7 +31,7 @@ class ScriptedGateway extends ModelGateway {
throw new Error('unused');
}
// eslint-disable-next-line require-yield -- unused
async *completeStream(): AsyncIterable<string> {
async *completeStream(): AsyncIterable<StreamDelta> {
throw new Error('unused');
}
async embed(texts: string[]): Promise<number[][]> {
Expand Down
5 changes: 3 additions & 2 deletions project/src/chat/chat-conversation.integration.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { startTestDatabase } from '../testing/index';
import type { TestDatabase } from '../testing/index';
import { UserDirectory } from '../identity/index';
import { ModelGateway } from '../model-gateway/index';
import type { StreamDelta } from '../model-gateway/index';
import type { RetrievalService } from '../retrieval/index';
import type { ChatReplyResolverPort } from './chat-reply-resolver.port';
import type { ChatResearchProposal, ChatResearchResolverPort } from './chat-research-resolver.port';
Expand Down Expand Up @@ -38,9 +39,9 @@ class ScriptedGateway extends ModelGateway {
complete(): never {
throw new Error('no completion expected');
}
async *completeStream(request: { input: string }): AsyncIterable<string> {
async *completeStream(request: { input: string }): AsyncIterable<StreamDelta> {
this.streamCalls.push(request.input);
yield this.streamText;
yield { channel: 'text', text: this.streamText } as const;
}
async embed(texts: string[]): Promise<number[][]> {
return texts.map(() => [0, 0, 0, 0]);
Expand Down
7 changes: 4 additions & 3 deletions project/src/chat/chat-conversations.integration.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import type { MemoryStore } from '../memory/index';
import { createIngestionPipeline, createSuppressedFactLog } from '../ingestion/index';
import { UserDirectory } from '../identity/index';
import { ModelGateway, ModelGatewayError } from '../model-gateway/index';
import type { StreamDelta } from '../model-gateway/index';
import type { StructuredExtractionRequest } from '../model-gateway/index';
import type { ZodType } from 'zod';
import { RetrievalService } from '../retrieval/index';
Expand Down Expand Up @@ -60,9 +61,9 @@ class ScriptedGateway extends ModelGateway {
complete(): never {
throw new Error('unused');
}
async *completeStream(request: { input: string }): AsyncIterable<string> {
async *completeStream(request: { input: string }): AsyncIterable<StreamDelta> {
this.streamCalls.push(request.input);
yield this.streamText;
yield { channel: 'text', text: this.streamText } as const;
}
async embed(texts: string[]): Promise<number[][]> {
return texts.map((t) => fakeEmbedding(t, DIMS));
Expand All @@ -88,7 +89,7 @@ class CaptureGateway extends ModelGateway {
throw new Error('unused');
}
// eslint-disable-next-line require-yield -- unused
async *completeStream(): AsyncIterable<string> {
async *completeStream(): AsyncIterable<StreamDelta> {
throw new Error('unused');
}
async embed(texts: string[]): Promise<number[][]> {
Expand Down
3 changes: 2 additions & 1 deletion project/src/chat/chat-reply-intent.integration.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import type { TestDatabase, TestQdrant } from '../testing/index';
import { createMemoryStore } from '../memory/index';
import type { MemoryStore } from '../memory/index';
import { ModelGateway } from '../model-gateway/index';
import type { StreamDelta } from '../model-gateway/index';
import { UserDirectory } from '../identity/index';
import { ChatService } from './chat.service';
import { RetrievalService } from '../retrieval/index';
Expand Down Expand Up @@ -42,7 +43,7 @@ class NoAnswerGateway extends ModelGateway {
return 'test-embed';
}
// eslint-disable-next-line require-yield -- must not be used
async *completeStream(): AsyncIterable<string> {
async *completeStream(): AsyncIterable<StreamDelta> {
this.streamCalls += 1;
throw new Error('reply intent must not stream an answer');
}
Expand Down
3 changes: 2 additions & 1 deletion project/src/chat/chat-research-intent.integration.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { startTestDatabase } from '../testing/index';
import type { TestDatabase } from '../testing/index';
import { UserDirectory } from '../identity/index';
import { ModelGateway } from '../model-gateway/index';
import type { StreamDelta } from '../model-gateway/index';
import type { RetrievalService } from '../retrieval/index';
import type { ChatResearchProposal, ChatResearchResolverPort } from './chat-research-resolver.port';
import { ChatService } from './chat.service';
Expand Down Expand Up @@ -49,7 +50,7 @@ class InertGateway extends ModelGateway {
throw new Error('no completion expected');
}
// eslint-disable-next-line require-yield -- ordinary-question path is stubbed to no facts
async *completeStream(): AsyncIterable<string> {
async *completeStream(): AsyncIterable<StreamDelta> {
throw new Error('no stream expected');
}
async embed(texts: string[]): Promise<number[][]> {
Expand Down
52 changes: 48 additions & 4 deletions project/src/chat/chat.integration.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import type { TestDatabase, TestQdrant } from '../testing/index';
import { createMemoryStore } from '../memory/index';
import type { MemoryStore, NewFact } from '../memory/index';
import { ModelGateway } from '../model-gateway/index';
import type { StreamDelta } from '../model-gateway/index';
import type { CompletionRequest } from '../model-gateway/index';
import { UserDirectory } from '../identity/index';
import { ChatService } from './chat.service';
Expand Down Expand Up @@ -49,11 +50,16 @@ class ScriptedChatGateway extends ModelGateway {
embeddingModelId(): string {
return MODEL;
}
async *completeStream(request: CompletionRequest): AsyncIterable<string> {
/** Thinking deltas emitted before the answer (Part C tests). */
thinkingDeltas: string[] = [];

async *completeStream(request: CompletionRequest): AsyncIterable<StreamDelta> {
this.streamRequests.push(request);
yield 'You owe Maja the draft contract before Thursday ';
yield '[F1]';
yield '.';
for (const thinking of this.thinkingDeltas)
yield { channel: 'thinking', text: thinking } as const;
yield { channel: 'text', text: 'You owe Maja the draft contract before Thursday ' } as const;
yield { channel: 'text', text: '[F1]' } as const;
yield { channel: 'text', text: '.' } as const;
}
}

Expand Down Expand Up @@ -212,4 +218,42 @@ describe('chat (integration, real Postgres + real Qdrant, gateway mocked)', () =
}
expect(await counts()).toEqual(afterFirst);
});

it('chat_thinking (Part C): streamed as its own event, stored beside the answer, never inside it', async () => {
gateway.thinkingDeltas = ['Considering the retrieved facts. ', 'Deciding on the citation. '];
try {
const conversationId = (await chat.createConversation(userA)).id;
const events: import('@cogeto/shared').ChatStreamEvent[] = [];
for await (const event of chat.ask(userA, 'What do I owe Maja?', conversationId)) {
events.push(event);
}

const thinkingEvents = events.filter((event) => event.type === 'thinking');
expect(thinkingEvents.map((event) => (event as { text: string }).text)).toEqual([
'Considering the retrieved facts. ',
'Deciding on the citation. ',
]);
// The thinking never leaks into the answer text or its stored form.
const done = events.find((event) => event.type === 'done') as { content: string };
expect(done.content).not.toContain('Considering the retrieved facts.');

const row = await tdb.pool.query<{ thinking: string | null; content: string }>(
`SELECT thinking, content FROM chat_message
WHERE conversation_id = $1 AND role = 'assistant'
ORDER BY created_at DESC LIMIT 1`,
[conversationId],
);
expect(row.rows[0]!.thinking).toBe(
'Considering the retrieved facts. Deciding on the citation. ',
);
expect(row.rows[0]!.content).not.toContain('Considering');

// The message page DTO carries it for reopening the conversation.
const page = await chat.listMessages(userA, conversationId, { limit: 10 });
const assistant = page.items.find((message) => message.role === 'assistant');
expect(assistant?.thinking).toContain('Deciding on the citation.');
} finally {
gateway.thinkingDeltas = [];
}
});
});
Loading
Loading