Skip to content

[serve][llm] Enable KV cache offloading & KV routing awareness of CPU KV caches - #65063

Draft
jeffreywang88 wants to merge 2 commits into
kv-atomic-reserve-v1from
kv-offload
Draft

[serve][llm] Enable KV cache offloading & KV routing awareness of CPU KV caches#65063
jeffreywang88 wants to merge 2 commits into
kv-atomic-reserve-v1from
kv-offload

Self review

2b10551
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar