equal-cost multipath

Suppose four equally short highways all connect your town to the next city. To avoid jamming one road while three sit empty, a sensible traffic system spreads cars across all four. But you also don't want a single carload of family members to keep getting split onto different roads and arriving out of order. The trick: send each carload (each conversation) down one road for the whole trip, but assign different carloads to different roads. That is exactly what equal-cost multipath does inside a datacenter fabric.

ECMP is a router behavior: when the forwarding table shows several next-hops that all reach the destination with the same lowest cost (as it does constantly in a fat-tree or leaf-spine, where every spine offers an equal path), the router spreads traffic across them instead of picking just one. To keep each TCP connection on a single path — so its packets stay in order and don't trigger spurious retransmissions — the router does not split packet by packet. Instead it computes a hash over fields that identify a flow (typically the five-tuple: source IP, destination IP, source port, destination port, protocol) and uses that hash to pick the next-hop. Same flow, same hash, same path; different flows scatter across the available links.

Why it matters: ECMP is the glue that turns a fat-tree's or leaf-spine's many parallel paths into usable aggregate bandwidth — without it, all that wiring would sit idle while one link overloaded. But it is statistical load balancing, not perfect. Two large 'elephant' flows can hash to the same link and collide, leaving other links empty (hash collisions), and ECMP is oblivious to actual load — it spreads by flow count, not by bytes. This is why operators add flowlet switching, adaptive load balancing, or congestion-aware schemes on top, and why a fat-tree's theoretical non-blocking bandwidth is not automatically delivered in practice.