In a Leaf-Spine fabric, every leaf switch has multiple equal-cost uplinks to the spine tier, creating inherent path diversity. How the network distributes traffic across those paths — and how it adapts when conditions change — determines fabric throughput, latency, and resilience. This project covers the full load-balancing stack, from static ECMP hashing through congestion-aware adaptive routing to the transport-level innovations designed for large-scale AI training.
The following documents are structured progressively. The first covers foundational ECMP mechanics, the second builds on that with congestion-aware adaptive routing, the third applies both to the specific constraints of RDMA transport in AI training fabrics, and the fourth is a protocol-level deep dive into MRC.
- Fabric Load Balancing: ECMP fundamentals — path discovery, hash functions, flow pinning, hash polarization, resilient hashing, the elephant flow problem, weighted ECMP, and centralized traffic engineering.
- Adaptive Routing: How modern switch ASICs overcome ECMP's congestion blindness — port grading, quality scores, flowlet switching, packet spraying, global adaptive routing with upstream notifications (ARN), and the interplay with centralized TE.
- Load Balancing for RoCEv2: Why RDMA's single-path Reliable Connection breaks standard load balancing at AI scale — the RC ordering constraint, QP scaling, adaptive routing for RoCE, measuring balance with CoV, and cell-based switching.
- MRC: Multipath Reliable Connection: The OCP multipath transport protocol for AI-scale Ethernet — entropy values and per-packet spraying, multi-plane topology, routing modes (ECMP, Structured EV, SRv6), out-of-order placement, selective retransmission (SACK/NACK), lossy Ethernet with packet trimming, NSCC congestion control, the EV state machine, probe-based failure recovery, the software API, and production impact.