Summary
ModelExpress source identity is currently mostly model + backend + TP/PP/EP + dtype/quantization.
That is enough for simple P2P cold-start cases, but it isn't enough for mapping topologies that users now care about: WideEP, TEP/DEP, P/D disagg, shadow/failover engines, and EPLB-style expert layout.
We should explore an engine-agnostic topology metadata contract so MX can tell when two workers are actually compatible sources for each other.
Motivation
Large MoE deployments have new requirements, especially when served with EP. Oftentimes, the EPLB info is essential for ensuring the transfer starts a new worker that matches the source worker.
A worker's compatibility can depend on factors like:
- whether it's a P/D worker, agg worker, or shadow engine,
- sharding layout,
- WideEP rank layout,
- expert placement,
- whether EPLB /redundant experts are enabled,
- whether expert traffic expects NVL/MNNVL, but KV or weight movement can use RDMA,
- engine runtime (SGL, vLLM, TRT-LLM, etc.)
Today MX has no first-class place to represent this. We need to explore what fields affect source compatibility, which fields are per-worker runtime state, and how different engines want to describe the same topology. Especially for MoE, (wide)EP, P/D deployments.
Proposal
Add a topology descriptor to MX metadata that is common across engines. The first version should focus on source compatibility and discovery, not on making MX own runtime scheduling.
Some possible fields:
- serving worker role:
prefill, decode, aggregated, shadow, etc.
- parallelism shape: TP, PP, DP, EP, CP/PCP where available,
- rank identity: global rank, local rank, DP rank, EP rank, engine group id,
- topology domain: node, rack, NVL domain, RDMA fabric, compute domain,
- expert metadata: logical experts, physical slots, redundant experts, placement strategy,
- EPLB metadata: enabled/disabled plus a stable config/layout digest,
- transport expectations: RDMA, NVL/MNNVL, local-only, or mixed paths,
- framework-specific escape hatch for fields that do not normalize cleanly yet.
This ideally should stay backend-agnostic. vLLM DP+EP, SGLang WideEP, and TRT-LLM disagg/WideEP should all be able to populate the same shape, but map to engine specific implementations.
Acceptance
- MX can distinguish incompatible MoE/P/D workers that currently look identical.
- MX can track info such as EPLB and use that to select the right worker/replicate for target new workers.
- The schema isn't engine-specific.
- Exisiting TP/PP/EP deployments continue to work unchanged.
- A new doc that details a WideEP or P/D scaleout example
Summary
ModelExpress source identity is currently mostly model + backend + TP/PP/EP + dtype/quantization.
That is enough for simple P2P cold-start cases, but it isn't enough for mapping topologies that users now care about: WideEP, TEP/DEP, P/D disagg, shadow/failover engines, and EPLB-style expert layout.
We should explore an engine-agnostic topology metadata contract so MX can tell when two workers are actually compatible sources for each other.
Motivation
Large MoE deployments have new requirements, especially when served with EP. Oftentimes, the EPLB info is essential for ensuring the transfer starts a new worker that matches the source worker.
A worker's compatibility can depend on factors like:
Today MX has no first-class place to represent this. We need to explore what fields affect source compatibility, which fields are per-worker runtime state, and how different engines want to describe the same topology. Especially for MoE, (wide)EP, P/D deployments.
Proposal
Add a topology descriptor to MX metadata that is common across engines. The first version should focus on source compatibility and discovery, not on making MX own runtime scheduling.
Some possible fields:
prefill,decode,aggregated,shadow, etc.This ideally should stay backend-agnostic. vLLM DP+EP, SGLang WideEP, and TRT-LLM disagg/WideEP should all be able to populate the same shape, but map to engine specific implementations.
Acceptance