Skip to content

SGLang cache artifact transfer #466

Description

@athreesh

Problem

#451 adds vLLM cache artifact transfer so warm replicas can share runtime-generated caches before engine init. We should make the same path work for SGLang instead of leaving artifact transfer as vLLM-only.

MVP

  • identify SGLang cache artifacts worth publishing/installing,
  • install compatible artifacts before SGLang model init,
  • publish after the source is ready/warm,
  • keep it gated by MX_ARTIFACT_TRANSFER=1 + P2P metadata,
  • add a small SGLang e2e/CI check.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions