Skip to content

Witness author prompt is cache-hostile: the fixed instruction block rides the volatile user message on every author and repair call #1786

Description

@macanderson

Problem

witness_prompt (crates/stella-pipeline/src/witness.rs) prepends a ~450-word fixed hard-requirements block to a per-call user message; the system message beside it (WITNESS_SYSTEM_PROMPT, pipeline.rs) is two sentences (~35 tokens). So the one verification role that runs a multi-step tool loop (author turn + repair turn, each with tool round-trips) pays uncached input for its entire instruction block on every call — exactly the failure the #1434 ManagementPrompt split fixed for triage/verdict/guidance.

Second half (#1434's known caveat, management_prompt.rs): the split only pays off past provider cache minimums (Anthropic ≥1024 tokens). VERIFIER_INSTRUCTIONS measures ~522 tokens post-#1782-era additions, still below the minimum. Nothing measures or warns.

Fix direction

  1. Move the fixed hard-requirements block into the witness author's system message (byte-stable prefix; invariant 7), leaving runners/structure/recall/goal in the user half. The engine path already cache-marks stable system prefixes (Management calls ship a byte-stable system prefix the adapters can cache-mark #1474).
  2. Measure: a small test or bench note recording each management role's stable-prefix token count vs the provider minimums, so "the split buys nothing here" is a stated fact instead of a hope. Consider whether padding via agents.<role>.prompt guidance belongs in docs.

Verify

Witness for 1: prompt-shape test asserting the hard-requirements text arrives as the system message and the user message starts with the volatile sections. Fails today.

Related

#1434, #1474; docs/spec/witness-protocol.md §7.3 (authoring is demand-driven — the block is paid only when a witness is warranted, which bounds the win but does not zero it).

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1Important — next in linearea:pipelinestella-pipeline — triage→…→judge orchestration

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions