stable mode: stop invalidating the KV cache mid-session

[Radu0120]

PI_MEMORY_SNAPSHOT=stable is documented as snapshotting memory "for KV cache stability", but three things re-render or move the injected block during a session, and each one costs a full prompt reprocess:

1. snapshotDirty on a long-term write (index.ts:1713) — writing memory is what a session does, so the stable path breaks on first use. The comment justifying it notes the model "should always see" the write; the header two lines above already says recent writes are visible in tool-call history, which is where the fact actually is, a few messages back.
2. snapshotTakenOnDate !== today — a midnight rollover dumps a live session's cache for a date change.
3. The caveat line Snapshot ${snapshotReason} at ${snapshotTakenAt} — a second-precision wall clock inside the cached prefix, plus a reason word whose length changes, so even a content-identical refresh moves every byte after it.

Measured, llama.cpp + a 27B GatedDeltaNet hybrid, ~15k-token session:

	before	after
turn after a target: long_term write	51,658 tokens reprocessed, 82.0 s	13 tokens, 0.26 s
turn after memory_forget	15,526 tokens, 17.8 s	108 tokens, 0.40 s

The cost is architecture-dependent and worst on recurrent/hybrid models (GatedDeltaNet, Mamba): their state can only be continued forward, never rewound to a partial match, so a changed prefix reuses zero tokens rather than the shared head. On plain attention the same change costs the tail only — still real on a long session.

Changes

• needsRefresh fires only when there is no snapshot. session_start and session_before_compact remain the checkpoints; compaction rewrites the prompt anyway, so that boundary is free.
• The caveat is now constant text with no timestamp and no reason word.
• Deletions and restores no longer re-render the block. They queue a line and emit it once as an injected message (before_agent_start → message, customType: "pi-memory-correction"), which lands at the tail of the history and costs only its own tokens. A forgotten memory still stops being authoritative — that part of the original reasoning is right, it just cannot be paid for in the prompt prefix. My first attempt appended the corrections to the system prompt instead; that is the 15.5k-token row in the table above, measured, which is what motivated the message channel.
• PI_MEMORY_SNAPSHOT=refresh restores the previous behaviour exactly; stable and per-turn are unchanged in name and meaning. README env table, snapshot section and troubleshooting row updated.

Tests

bun test test/unit.test.ts → 184 pass, 0 fail. Updated the two tests that encoded the old contract (long_term write now asserts the prompt is unchanged; the caveat test asserts no hh:mm:ss appears), and added two: PI_MEMORY_SNAPSHOT=refresh still refreshes on a long-term write, and memory_forget sends a correction message once, drains it, and never touches the systemPrompt. npm run build and biome check are clean.

Note for review

pi delivers the injected message with role user, so the correction reads as if the user said it. It is tagged with customType and the content is explicit about what it is, but if you would rather it went through a different channel, say so and I will rework it.

One design call worth confirming: I made the cache-safe behaviour the default and put the old one behind refresh, on the grounds that a mode named stable should be stable. Happy to invert that if you would rather the change be strictly opt-in.