pipeline: management-role system prompts sit below Anthropic's minimum cacheable prefix — every management call reads 0% cache

[macanderson]

Problem

Every management role (triage/verdict/guidance/plan/conversational) dispatches through metered_raw_call with tools: Vec::new() and a role-specific instruction block as the system message (crates/stella-pipeline/src/pipeline/raw_usage.rs, management_prompt.rs). The Anthropic adapter stamps cache_control on that system block, but e.g. TRIAGE_INSTRUCTIONS is ~600 tokens — below Anthropic's ≥1024-token minimum cacheable prefix (management_prompt.rs:18-23 admits this) — so no cache entry is ever created. With the deck defaulting to pipeline mode, several such calls ride every turn, adding input tokens with 0% cache read. This is likely the largest single contributor to Stella's 76% aggregate cache ratio vs Claude Code's 93%: it moves the denominator without ever moving the numerator.

Fix direction

Options, roughly in order of leverage:

1. Fold the fixed instruction blocks into ONE shared ≥1024-token system prefix reused byte-identically across all management roles (role-specific instruction rides as the first user message), so all roles share one cache entry.
2. Route management calls to a cheap model tier by default (their tokens then stop dominating the cost side even uncached) — the routing plumbing exists (role router).
3. Both.

Verify

Measure before/after on a recorded pipeline run (the cache_insight surface); adapter-level witness that all management roles serialize the identical system prefix bytes.

Constraints

Prompt changes affect triage/verdict quality — keep the per-role output contracts (PASS/FAIL token etc.) intact and diff behavior on the replay fixtures (crates/stella-pipeline/tests). Related: PR #1811 (output side), #1840 (conversational input side).

[macanderson]

Triage 2026-08-06: P1 — several sub-minimum system prefixes ride every turn on the default deck path, each guaranteed 0% cache read, and this plausibly explains most of the 76%-vs-93% cache-ratio gap against Claude Code (moves the denominator, never the numerator). Plan: as specced; next step is option 1 — one shared ≥1024-token byte-identical management prefix with the role-specific instruction as the first user message — pinned by the adapter-level byte-equality witness, then a before/after measurement on a recorded run via cache_insight. Keep the per-role output contracts (PASS/FAIL tokens) intact against the replay fixtures; cheap-tier routing (option 2) compounds and can follow separately.

[macanderson]

Assessment (not started) — option 1 is a prompt-authoring job, not a refactor

management_prompt.rs's own header already measures the problem and agrees with the diagnosis, which is worth quoting because it means no investigation is needed to start:

Measured (#1786, 2026-08, estimator scale): no fixed block clears the Anthropic minimum on its own — the verdict instructions estimate ~520 tokens, guidance and triage less, and the witness author's system prompt ~620. So for the RAW calls (triage/verdict/guidance) the split's cache win is real only with an agents.<role>.prompt override padding the prefix past the minimum; stability across calls is what the split guarantees, not a hit.

So the structure (#1434) is already correct — system block stable, payload volatile. What is missing is length, and that is the part that cannot be refactored into existence.

Why I stopped rather than shipping it. Option 1 needs a shared preamble of roughly 1024 tokens (~4000 characters) placed byte-identically at the head of every management role's system block. Two things follow:

1. It changes model behaviour on triage, verdict, guidance and plan simultaneously. The issue's own Verify says to measure before/after on a recorded run, and its constraint says to keep the per-role output contracts intact and diff against the replay fixtures. Landing it unmeasured is precisely what that constraint forbids.
2. Padding to reach the minimum would be dishonest — tokens the model must read on every management call, written to satisfy a cache threshold rather than to help it decide. If the shared preamble is not independently worth its length, option 1 is not worth taking.

There is genuinely shared content worth writing (what the pipeline is, how to treat evidence, the discipline the output contracts share), so this is a real authoring task with a real payoff — it just is not a drive-by.

Option 2 is the cheaper half and is independent. Routing management roles to a cheap tier by default needs no prompt change and no behaviour diff on the contracts; the role router already exists. It does not fix the 0% cache read, but it removes management tokens from dominating cost while option 1 is being written and measured. Worth splitting into its own issue so the measurement work does not block it.

One adapter-level witness is cheap and useful now, whichever option lands: assert that all management roles serialize an identical system prefix. That pins the structural half (#1434) against regression and can be written before the preamble exists — it will simply assert over a short shared prefix today and a long one later.

Related work that has since landed: #1848 (schema ordering — the tools tier of the same prefix) and #1856 (per-server schema budget).