Skip to content

Keep durable chat images once, out of Pi Durable's transcript rows - #283

Merged
jonaslsaa merged 3 commits into
mainfrom
feature/durable-image-offload
Nov 3, 2026
Merged

jonaslsaa merged 3 commits into
mainfrom
feature/durable-image-offload

Conversation

@jonaslsaa

@jonaslsaa jonaslsaa commented Nov 3, 2026 •

Copy link
Copy Markdown
Owner

With SqliteStore's images option (R2, say), memory keeps its copy of an image outside the database, but Pi Durable still stores every image as base64 in its transcript rows. The database grows with each image, and a message's images must fit one 2 MB row together.

What changes

Each image in a durable chat now lives once, in memory's store; Pi's transcript row holds only its ref, [image <16 hex digits>], the same ref memory already uses. The code is in src/durable/images.ts; optchat.ts only wires the hooks.

  • Tool results (afterTool, always on): a model-issued tool's images are put in the store and replaced by their refs. Nested calls are left alone, and so is a result whose text could pass Pi Durable's default bound (50 KB or 2,000 lines) with the refs: Pi bounds a result after the hook by keeping one of its text blocks, which would drop them.
  • User messages (offloadImages(input), new, opt-in): Pi Durable has no user-input hook, so the app passes an input through it before submit. If it doesn't, images stay inline as today.
  • Requests (beforeRequest): once a request is built, exact ref blocks in its user messages and tool results are swapped back for the images, each read once per request; steps a refresh dropped are never read. A ref counts as an image in the refresh estimate and survives boundedMessage's cap (core messages.ts). Restoring never throws: an unknown ref or a failed read is sent as the text.

A failed put keeps that image inline. Memory's text is byte-identical to before (zoom's own results drop the refs they would add), and zoom still returns the image, also after a restart. The compactor only switches its image labels to ref (one line in src/core/compactor.ts).

Hosts that read Pi's transcript themselves see refs instead of images; docs/durable.md says how to get them (memory.store.image(name)) and how hook order works.

pi-optchat/core now exports ref(name) and refName(text), the one ref format Pi and durable both use, built on core's existing REF pattern, plus loadImage(store, name). The compactor's image labels (#284) use ref too, so there is one ref format.

Closes #282

jonaslsaa added a commit that referenced this pull request Nov 3, 2026
@jonaslsaa
jonaslsaa requested a balanced review from Copilot November 3, 2026 01:11
jonaslsaa added a commit that referenced this pull request Nov 3, 2026
…they show (#284)

* Let the compactor see the images a message names

A message's leaf summary now gets the images it names, after the task text,
so its line says what they show; a message with images is summarized even
when short. A text-only compactor model gets none, and a provider that
rejects them is asked once more by name only. Merges stay text.

* Compactor images: retry short ones by name before giving up, skip zoom pages, tidy

- A short image message whose request fails for any non-retryable reason is
  asked once more by name only; if that fails too, the leaf is retried later.
- A zoom result's page doesn't send its images again.
- One short flag, required Compression.images, task() reads the current images.
- Export imageNames from core/images.ts.
- Split the compactor tests; add tests for a failed image read, a refused
  short image summary, and the Anthropic cache marks.

* Drop imageNames: #283 brings core's marker helpers

* Compactor images: label each image with its name, load from the capped source, keep the names-only call out of the 5 tries
Tool results' images go to memory's store in afterTool and leave their
name in Pi's row; offloadImages does the same for a user's input.
beforeRequest puts the images back for each request.

Closes #282
…de to durable/images.ts

Pi Durable bounds a tool result after afterTool by keeping one text block,
which dropped the refs of a result over 50 KB or 2,000 lines. Decide
first, then put; zoom's images skip the put. One ref format from core
(ref, refName), one keep helper, tests for the bound, a failed read and
the restart split out.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved image-handling defects can discard trailing newlines, replace zoom images with text, and repeatedly reload images excluded after refresh.

4 open issues
What changed in this PR

This PR reduces Pi Durable transcript storage by keeping images in memory’s image store and recording references in transcript rows, addressing #282.

Changes:

  • Offloads tool-result images automatically and adds opt-in offloading for user inputs.
  • Restores image references for model requests, with inline or text fallbacks when storage operations fail.
  • Shares core reference helpers and adds integration tests and host documentation.
File Description
test/​durable.test.ts Tests offloading, restoration, restart behavior, and storage failures.
src/​images.ts Uses the shared core image-reference formatter.
src/​durable/​optchat.ts Wires image hooks and exposes offloadImages.
src/​durable/​images.ts Implements image naming, storage, restoration, and bounds checks.
src/​core/​index.ts Exports the shared reference helpers.
src/​core/​images.ts Adds reference formatting and parsing helpers.
docs/​durable.md Documents transcript references, fallback behavior, and hook ordering.

Review effort: Balanced

Comment thread src/core/images.ts Resolved
Comment thread src/durable/images.ts Resolved
Comment thread src/durable/optchat.ts Resolved
Comment thread docs/durable.md Resolved
…n; one ref format with the compactor

Images go back after the request is built, so steps a refresh dropped are
never read; a ref counts as an image in the refresh estimate and survives
boundedMessage's cap. Zoom's images are put again, since the store gives
them back as canonical base64 under another name. The compactor labels
images with core's ref(). Docs: core's ref helpers, the compactor's image
cost, beforeRequest hook order.
@jonaslsaa
jonaslsaa force-pushed the feature/durable-image-offload branch from e42978e to d1e6bb5 Compare November 3, 2026 01:21
@jonaslsaa
jonaslsaa merged commit 1cfe76c into main Nov 3, 2026
4 checks passed
@jonaslsaa
jonaslsaa deleted the feature/durable-image-offload branch November 3, 2026 01:26
@jonaslsaa jonaslsaa mentioned this pull request Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Durable: images in tool results stay in Pi's transcript, even when an ImageStore keeps them outside the database

2 participants