features streaming chat

Streaming chat

Active contributors: Zachary BENSALEM

Purpose

Fleet uses two streams for one session. POST /api/chat returns the active turn as NDJSON. GET /api/chat/events carries out-of-turn updates, pending dialogs, presentation changes, and bounded replay. Both transports use the ChatStreamEvent union from web/protocol/src/chat-protocol.ts.

Turn flow

sequenceDiagram
    participant UI as web/app
    participant Chat as POST /api/chat
    participant Bridge as PrimeBridge
    participant Daemon as Prime daemon
    UI->>Chat: ChatRequest
    Chat-->>UI: start frame
    Chat->>Bridge: promptAndWait
    Bridge->>Daemon: prompt with queueIfBusy
    Daemon-->>Bridge: connection events
    Bridge-->>Chat: mapped frames
    Chat-->>UI: NDJSON delta/tool/state frames
    Bridge-->>UI: SSE presentation and out-of-turn frames
    Bridge-->>Chat: done or error
    Chat-->>UI: terminal frame

Loading

web/server/src/handlers/chat.ts validates the request, resolves the live or persisted session, validates attachments, writes start, subscribes to bridge events, and closes on done or error. It can build a terminal transcript fallback if a hot reload loses the listener.

SSE replay

web/server/src/handlers/chat-events.ts registers its listener before bootstrapping. It writes a connected frame, an optional child snapshot, then replays RingBuffer entries newer than the requested Last-Event-ID or lastEventId query. Every session has a bounded buffer from web/server/src/ring-buffer.ts; a 15-second heartbeat keeps idle connections open.

The cursor is valid only for its stream generation. If the cursor falls outside the buffer, the server emits a state frame with message: "resync-required". The browser reloads the session snapshot and Fleet sidecars. First-time connections replay still-pending question frames through web/server/src/sse-replay.ts; answered dialogs are not replayed.

Browser state

web/app/src/lib/pi/chat-fetch.ts reads newline-delimited response chunks, validates structural frames, and tracks sequence gaps. web/app/src/lib/pi/use-pi-chat.ts opens the SSE channel, persists the per-session cursor in sessionStorage, and reconnects with generation/cursor metadata. web/app/src/lib/pi/chat-stream-state.ts applies frames to the message and presentation snapshot.

The reducer gates presentation frames by revision, ignores legacy raw thinking frames, reconciles optimistic assistant IDs, and treats synthetic session-reset completions differently from real turn completion.

Cancellation and failure

POST /api/chat/abort cancels the active connection and pending dialogs. ChatRequestError in web/app/src/lib/pi/chat-fetch.ts parses the shared error envelope. Daemon disconnects, unknown sessions, stream errors, and buffer overflow have separate recovery paths.

Entry points for modification

Change frame meaning in web/protocol/src/chat-protocol.ts and web/protocol/src/schemas/chat.ts, map upstream events in web/server/src/event-mapper.ts, and update both NDJSON and SSE consumers. Add deterministic tests for ordering, reconnect, overflow, and hydration in the server and app test directories.

Key source files

File	Purpose
web/server/src/handlers/chat.ts	Active-turn NDJSON response
web/server/src/handlers/chat-events.ts	Root and child SSE response
web/server/src/ring-buffer.ts	Bounded sequence buffer
web/server/src/sse-replay.ts	Replay filtering and normalization
web/server/src/event-mapper.ts	Runtime events to frames
web/app/src/lib/pi/chat-fetch.ts	Fetch, parsing, and cursor diagnostics
web/app/src/lib/pi/use-pi-chat.ts	SSE lifecycle and hydration
web/app/src/lib/pi/chat-stream-state.ts	Pure frame reducer
web/protocol/src/chat-protocol.ts	Event union and capabilities

See the adapter contract and chat state.