Kv prefix cache - #23

Merged

caseymcc merged 2 commits into

mainfrom

kv-prefix-cache

Jun 11, 2026

Commits on Jun 11, 2026

Implement cache_prompt KV prefix reuse in the llama runtime

AVA Agent
and
claude
committed
da9f630
Parse cache_prompt in the server chat completions route

AVA Agent
and
claude
committed
1e5d47e