Skip to content

perf(agent): keep the prompt-cache prefix stable across harness updates, date flips, and tool refreshes #26

Description

@rynfar

Part of #22 (Meridian reliability umbrella). P1.

Problem

Anthropic prompt caching requires a byte-stable prefix (system prompt + tool definitions + message history). Prime Agent's system prompt embeds volatile state, so unrelated actions invalidate the whole cache for a long-running session:

  • _rebuildSystemPrompt (packages/coding-agent/src/core/agent-session.ts) serializes harnessState: this._loadMergedHarnessState() — mutable continual-harness state (memories, skills, subagent specs) — directly into the system prompt. Any rlm.harness.create_* call by the model changes the cached prefix for every subsequent request.
  • Current date: ${date} (packages/coding-agent/src/core/system-prompt.ts) flips the prefix daily — long-running/scheduled sessions cold-cache at midnight.
  • Tool-set changes (_refreshToolRegistry) reorder/replace context.tools, and cache_control is placed on the last tool definition, so any reorder invalidates the tool-block cache.
  • The extension context hook (extensions/runner.ts emitContext) lets any extension rebuild the message array per call — a silent prefix-breaker worth a debug-level stability check.

Through Meridian this compounds: a prefix change on one turn forces the Claude Code SDK session to re-cache the full conversation (5-min-TTL cache), which is exactly the "expensive cache churn" measured in #22 (65.8% avg hit rate vs 95.1% for OpenCode).

Proposed shape

  • Move volatile content (harness state, current date) out of the cached prefix — e.g. into a trailing uncached system block or the first user message — so refining the harness doesn't cold-cache the session.
  • Stabilize tool ordering (stable sort by name) so the cache_control-marked last tool doesn't move when the registry refreshes.
  • Optional: a debug assertion that logs when a rebuilt system prompt/tool block differs from the previous turn's, with a diff summary — makes future prefix regressions visible instead of silently expensive.

Acceptance

  • A rlm.harness.create_memory call mid-session does not change the system-prompt bytes of the next request's cached prefix region (test at the buildParams/payload seam).
  • Tool registry refresh with an unchanged tool set produces byte-identical tool definitions in the payload.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions