Standalone host-memory sources for ModelExpress

[athreesh]

Proposal: Standalone Host-Memory Sources for ModelExpress

Problem

ModelExpress P2P sources are currently tied to runtime engine processes. A vLLM,
SGLang, or TensorRT-LLM worker loads weights into GPU memory, registers those
tensors with NIXL, and publishes itself as a source.

That is the right path for many inference cold-start cases, but it couples two
roles that do not always need to be coupled:

• weight residency: holding and serving model weights,
• execution: allocating runtime tensors and serving inference or rollout
  traffic.

For model loading and RL weight refresh, it would be useful for MX to support
sources that are not runtime workers.

Proposal

Add a standalone MX source-worker role that can hold model weights in host DRAM,
register that memory with NIXL, and publish itself as a source through the
existing MX metadata path.

model artifact / cache / checkpoint
  -> MX host-memory source worker
  -> NIXL transfer
  -> runtime worker VRAM

The runtime still owns the target side: tensor allocation, compatibility checks,
load/refit, and execution. The source worker only owns weight residency,
metadata publication, and memory lifetime.

This should be backend-agnostic. The source worker should use the selected NIXL
backend rather than being tied to a specific fabric or cloud.

Why This Matters

This would make MX usable as a standalone model-weight service, not only as an
engine plugin.

Possible benefits:

• scale runtime workers without requiring another runtime worker as the seed,
• run weight sources as sidecars, DaemonSets, colocated processes, or dedicated
  pools,
• hydrate weights from cache/checkpoints/storage into DRAM before runtime load,
• give RL rollouts a versioned weight relay path,
• keep runtime lifecycle separate from weight-source lifecycle.

The core question is:

Should MX source workers be allowed to exist independently of runtime engine
workers?

MVP

A useful first spike could be intentionally small:

local safetensors or existing MX cache
  -> standalone MX host-memory source
  -> DRAM registered with NIXL
  -> existing runtime target receives into VRAM

MVP requirements:

• standalone source process,
• load one model/version into host memory,
• publish source metadata with an explicit host-memory source role,
• allow an existing runtime target to pull DRAM -> VRAM,
• validate tensor name, dtype, and size matches,
• report source load, registration, transfer, and target apply timings,
• preserve the current GPU-source path.

This does not need to solve full RL resharding, source pooling, or remote
object-store integration in the first step.

Non-Goals

• replacing runtime loaders,
• making MX the RL sample/reward/trajectory data bus,
• requiring direct object-storage-to-VRAM transfer,
• making the design provider-specific,
• solving arbitrary train-layout to inference-layout conversion in the MVP.