Enhanced ECMP for AI Cluster

Currently, there are two granularities for network load balancing: per-flow ECMP and per-packet forwarding. As illustrated in , the per-flow ECMP method employs flow characteristic-based hashing (typically using the five-tuple) to distribute traffic across multiple ECMP paths. This approach works effectively in environments with numerous small flows and absence of elephant flows. Its primary advantage is the elimination of packet reordering issues. However, this method presents limitations when dealing with either: a limited number of flows, or the presence of elephant flows. In such cases, five-tuple-based hashing may lead to hash collisions, causing disproportionate mapping of oversized flows to the same path. This results in suboptimal load balancing performance.

The other approach is per-packet forwarding. This method applies hashing to each individual packet, distributing traffic across different ECMP paths, as illustrated in . Theoretically, it achieves optimal load-balancing granularity. However, it introduces severe packet reordering within the same flow, necessitating additional mechanisms (e.g., reordering buffers or sequence tracking) to handle out-of-order delivery. This imposes higher demands on network infrastructure.

In AI training scenarios, the current mainstream load balancing technology is per-flow ECMP. However, hash collision issues lead to imbalanced traffic distribution, adversely affecting application performance. To address this problem, this document proposes an enhanced ECMP method that resolves load imbalance caused by hash collisions. The proposed solution effectively improves load balancing efficiency, reduces network congestion, and enhances overall network performance.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Due to the unique traffic patterns in AI training networks - characterized by a limited number of flows - achieving balanced load distribution becomes challenging. Traditional flow-based load balancing strategies often result in uneven traffic distribution, potentially leading to network congestion. While packet-based approaches can mitigate this imbalance to some degree, they introduce packet reordering issues as flow packets may traverse different paths, requiring additional network-level reordering mechanisms. This document proposes two enhanced ECMP methods to address the load imbalance issue in AI training networks and improve the overall network performance.

Group the ingress interfaces for traffic, assign an ECMP number to the interfaces within the same group, and then perform ECMP hashing based on this ECMP number. This method is suitable when the forwarding traffic size for each ingress interface is roughly the same.

group-id 1 -> ECMP Index 1--> |if 1 | +-------+ ingress-interface 2 -> group-id 1 -> ECMP Index 2--> |if 2 | +-------+ ingress-interface 3 -> group-id 1 -> ECMP Index 3--> |if 3 | +-------+ ingress-interface 4 -> group-id 1 -> ECMP Index 4--> |if 4 | +-------+ ingress-interface 5 -> group-id 2 -> ECMP Index 1--> |if 1 | +-------+ ingress-interface 6 -> group-id 2 -> ECMP Index 2--> |if 2 | +-------+ ingress-interface 7 -> group-id 2 -> ECMP Index 3--> |if 3 | +-------+ ingress-interface 8 -> group-id 2 -> ECMP Index 4--> |if 4 | +-------+ ]]> As shown in above, the eight ingress interfaces are divided into two groups, with four interfaces in each group. Within each group:

The ingress interfaces are assigned ECMP numbers 1, 2, 3, and 4 respectively.
ECMP hashing is performed based on these assigned ECMP numbers to select corresponding egress interfaces for forwarding.

For traffic entering through the four ingress interfaces in Group 1: Different egress interfaces are selected for forwarding (four distinct paths). Similarly, for traffic entering through Group 2's four ingress interfaces: Different egress interfaces are selected for forwarding (four distinct paths).

As shown in the figure, the source HOST connects to the source Leaf via an Ingress interface, while the destination HOST connects to the destination Leaf through an Egress interface. Multiple ECMP (Equal-Cost Multi-Path) links exist between the Leaf switches and multiple Spine devices. To improve load-balancing distribution uniformity, the ECMP interfaces connecting to multiple Spine devices are grouped on the source Leaf. This grouping SHOULD be configured on the Leaf device. For example, if there are 128 equal-cost links between the source Leaf and Spine devices, they can be divided into groups of 4 interfaces each (Group 1: interfaces 1-4; Group 2: interfaces 5-8, etc.). For traffic load balancing, flows MUST first be mapped to specific groups based on their location information (flows with the same location information are assigned to the same group), and then hash-based load balancing is performed within each group. The location information used for group mapping can be either the source's Ingress interface or the destination's Egress interface. By implementing fine-grained grouping of ECMP interfaces, this solution achieves more uniform traffic load distribution, thereby addressing current issues of imbalanced load sharing and flow collisions.

First, as shown in , when the Egress Leaf advertises a route, it MUST carry the egress group index, which can be composed of the local device's router-id and group-id (see for details). On the Ingress Leaf, the ECMP egress interfaces toward the Spine devices MUST be grouped (as illustrated in ). When the Ingress Leaf receives a route, it MUST extract the remote egress group index carried in the route. It then maps this remote egress group index to a local ECMP subgroup index, effectively directing traffic to the corresponding subset of interfaces for forwarding. This ensures that flows destined for different remote addresses are load-balanced across different ECMP subgroups, improving distribution granularity. Refer to for details.

To support the enhanced ECMP mechanism based on egress grouping described in , the route advertised by the Egress Leaf MUST carry an egress group index, which is composed of the local device's router-id and group-id. The encoding of this information is defined as a new BGP Extended Community (BGP_EXT-COMM-ECMP-GROUPID) in . On the Ingress Leaf, when a route carrying this extended community is received, the remote egress group index is mapped to a local ECMP subgroup index as described in , directing traffic to the corresponding subset of ECMP interfaces for forwarding.

TBD.

This document has no IANA actions. The registration request for BGP_EXT-COMM-ECMP-GROUPID is defined in .