pi-deepseek-cache

Pi extension for DeepSeek cache-stable invariants. Enforces byte-stable system prompts, detects and blocks tool-call storms, shrinks oversized tool results, and recovers lost tool calls from DeepSeek's thinking blocks.

Result: 99%+ prefix-cache hit rates on DeepSeek V4, ~$1 for a full work session vs ~$70 without optimization.

Install

# From git
pi install git:github.com/hermitokatt/pi-deepseek-cache

# Or try without installing
pi -e git:github.com/hermitokatt/pi-deepseek-cache

Run /reload in pi after install.

What it does

Feature	How	Cache impact
Byte-stable system prompt	Strips dynamic date/cwd, injects as trailing note	Protects prefix cache across sessions
Storm detection	Blocks identical tool calls repeated 3+ times	Stops token-wasting loops
Tool result shrinking	Caps at ~3000 tokens per result	Keeps context lean
Scavenge repair	Recovers tool calls lost in DeepSeek thinking blocks	Fixes V4's #1 failure mode
Compaction visibility	Logs compaction events + estimates cache-miss cost	Makes cache events visible
Budget gate	Warns at 80%, refuses at 100% of configurable cap	Cost control
Cache-hit stats	Session-level cumulative cost, cache%, context% in footer	Visibility
Model gating	DeepSeek-specific features disable for other providers	Safe to switch models

SYSTEM.md

The extension works best with a byte-stable system prompt. A template SYSTEM.md is included — place it at ~/.pi/agent/SYSTEM.md (global) or <project>/.pi/SYSTEM.md (project-local). The template removes dynamic fields that break the prefix cache and adds formatting rules that keep DeepSeek's output well-structured.

Without SYSTEM.md, the extension still strips the dynamic date/cwd lines from pi's default system prompt. The template provides additional cache-stable formatting rules.

Commands

Command	What it shows
/scavenge-stats	Scavenge recoveries, compaction events, recent session history

Composes with pi-cache-optimizer

pi-cache-optimizer handles system prompt reordering (stable content before dynamic), skill listing compression, and PI_CACHE_RETENTION=long. pi-deepseek-cache handles tool execution invariants (storm, shrinking, scavenge, budget). They layer cleanly — install both for comprehensive cache optimization.

How it works

DeepSeek's prefix cache bills cached input at ~$0.0036/M tokens vs $0.435/M for uncached — a 120x difference. The cache activates when the EXACT byte prefix of a request matches the previous request. This extension enforces invariants that keep the prefix stable:

1. System prompt is made byte-stable by stripping dynamic date/cwd (which change daily/per-directory) and injecting them as a trailing user-note message that sits after the append-only log.

2. Tool calls are monitored for storms (identical calls repeated in a loop). Blocked after threshold to stop token waste.

3. Tool results are capped at ~3000 tokens. The model had the full text for the turn that read it; subsequent turns don't need to drag 50KB through every prompt.

4. Scavenge repair recovers tool calls that DeepSeek V4 emitted inside <think> blocks but forgot to include in the explicit tool_calls list. This is V4's most common failure mode.

5. Compaction (pi's context management) is tracked and its cache-miss cost is estimated and logged.

Configuration

Edit the constants at the top of index.ts:

Constant	Default	Description
MAX_TOOL_RESULT_CHARS	12_000	Max chars per tool result (~3000 tokens)
STORM_THRESHOLD	3	Identical calls before blocking
BUDGET_USD	20	Session budget cap (0 to disable)

Verify

1. Select a DeepSeek model: /model deepseek/deepseek-v4-pro
2. Send several prompts in the same session
3. Watch the footer status line: $0.42 Sitzung | Cache 99.1% (36.7M/333K) | Kontext 23%
4. Run /scavenge-stats to see recovery and compaction stats

License

MIT