Problem
Currently, LanguageModelRequest::builder().system() accepts a plain String, which gets converted to a single system message. There's no way to attach cache_control metadata to content blocks in the system prompt or messages.
This means providers that require explicit cache breakpoints (notably Anthropic via cache_control: {"type": "ephemeral"}) can't leverage prompt caching through aisdk, even though the SDK already reads cached token counts from responses (AnthropicCacheCreation).
Impact
For multi-turn tool-use agents, the system prompt + tool declarations are large (30-60K chars) and identical across turns within a conversation. Without cache_control breakpoints:
- Anthropic models: 0% cache hit rate (confirmed via telemetry)
- Google/Z-AI models: ~35-46% cache hit rate (automatic caching, no explicit markers needed)
Enabling Anthropic caching would reduce input token costs by ~90% on cache reads for the static prefix.
Proposed API
Allow passing structured content blocks with optional cache_control to .system() and within messages:
// Option A: Builder method for cache breakpoint on system prompt
let request = LanguageModelRequest::builder()
.model(model)
.system_with_cache("system prompt text", CacheControl::Ephemeral)
.messages(messages)
.build();
// Option B: Content block array with cache markers
let request = LanguageModelRequest::builder()
.model(model)
.system(vec![
ContentBlock::text("static system prompt").with_cache(CacheControl::Ephemeral),
])
.messages(messages)
.build();
Provider Mapping
The cache_control should be mapped to each provider's native format:
| Provider |
Format |
Min Tokens |
Notes |
| Anthropic |
cache_control: {"type": "ephemeral"} on content blocks |
1,024 (Sonnet/Opus) |
Max 4 breakpoints |
| OpenAI |
Automatic (no explicit control needed) |
1,024 |
Could be a no-op |
| Google |
cachedContent resource |
32,768 |
Different mechanism |
| DeepSeek |
Supports cache_control (Anthropic-compatible) |
1,024 |
Via OpenAI-compatible API |
For providers without explicit caching support, the cache_control metadata can be silently ignored.
Current Workaround
None — the system prompt path (LanguageModelOptions.system: Option<String>) doesn't support structured content blocks, and the conversion in openai_chat_completions/conversions.rs always produces a single ChatMessage with a plain string content.
Context
We're using aisdk 0.5.2 via OpenRouter to power an AI assistant with 65 tools. After optimizing tool declarations (41% size reduction), caching the static prefix is the next biggest cost-saving opportunity.
Problem
Currently,
LanguageModelRequest::builder().system()accepts a plainString, which gets converted to a single system message. There's no way to attachcache_controlmetadata to content blocks in the system prompt or messages.This means providers that require explicit cache breakpoints (notably Anthropic via
cache_control: {"type": "ephemeral"}) can't leverage prompt caching through aisdk, even though the SDK already reads cached token counts from responses (AnthropicCacheCreation).Impact
For multi-turn tool-use agents, the system prompt + tool declarations are large (30-60K chars) and identical across turns within a conversation. Without cache_control breakpoints:
Enabling Anthropic caching would reduce input token costs by ~90% on cache reads for the static prefix.
Proposed API
Allow passing structured content blocks with optional cache_control to
.system()and within messages:Provider Mapping
The cache_control should be mapped to each provider's native format:
cache_control: {"type": "ephemeral"}on content blockscachedContentresourcecache_control(Anthropic-compatible)For providers without explicit caching support, the cache_control metadata can be silently ignored.
Current Workaround
None — the system prompt path (
LanguageModelOptions.system: Option<String>) doesn't support structured content blocks, and the conversion inopenai_chat_completions/conversions.rsalways produces a singleChatMessagewith a plain string content.Context
We're using aisdk 0.5.2 via OpenRouter to power an AI assistant with 65 tools. After optimizing tool declarations (41% size reduction), caching the static prefix is the next biggest cost-saving opportunity.