The problem the wiring has to solve
The previous guide left us with a sharp demand. A datacenter network connects tens of thousands of servers, and most of their traffic is east-west — server talking to server inside the building — rather than north-south traffic out to the Internet. The dream is that any server can talk to any other server at full speed at the same time, as if every machine were plugged into one impossibly large switch. The whole topology question is simply: how do you build that one giant switch when no single switch chip is anywhere near big enough?
Two numbers measure how well you have done. The first is bisection bandwidth: imagine slicing the whole datacenter into two equal halves and adding up the bandwidth of every cable that crosses the cut. That is the worst-case capacity for traffic flowing between the halves, and it is the honest yardstick for an all-to-all workload. The second is path diversity: how many different routes exist between two servers, because many short paths mean traffic can spread out and a single failed switch is just one path lost, not a catastrophe.
Hold onto a distinction from the very first rung, because the topology cannot break it. More cables and more paths buy you more bandwidth — more bytes per second between the halves. They do not buy you lower latency: a packet still has to travel through some number of switches, and the speed of light and per-hop forwarding time set a floor that no amount of extra wiring can beat. The goal here is bandwidth and resilience at scale; keeping latency low is a separate fight, fought mostly with fewer hops and the transport tricks of later guides.
The old way: a three-tier tree, and where it chokes
The classic enterprise design is the three-tier topology: a tree. At the bottom, access switches sit at the top of each rack and connect the servers in that rack. Those uplink to a middle aggregation layer, which uplinks to a small core layer at the very top. Every server reaches every other server by climbing up to a common ancestor in the tree and back down — which is fine for north-south traffic heading out to the Internet, the job this shape was originally designed for.
The trouble is the funnel. A rack might hold 40 servers each with a 10 Gbps link, which is 400 Gbps of possible traffic leaving the rack — but the access switch may have only, say, 80 Gbps of uplink to the layer above. That ratio, 400 to 80 or 5 to 1, is called oversubscription: you have sold five times more capacity at the edge than the core can actually carry. As you climb the tree, the links get fatter but rarely fat enough, and the few core switches become a bottleneck that all cross-datacenter traffic must squeeze through.
For east-west traffic this is exactly the wrong shape. Two servers in different racks that want to exchange a terabyte for a data-shuffle job must both haul their bytes all the way up to the contended core and back down, fighting every other cross-rack flow for that thin core capacity. The expensive cure was to buy ever-bigger, ever-more-special core switches — vertical scaling that gets exponentially pricier and still leaves you with a handful of boxes whose failure takes down half the building.
The new way: build a big switch out of small ones
The breakthrough flips the economics. Instead of a few giant, custom core switches, use a large number of small, cheap, identical commodity switches and wire them in a clever pattern so that, taken together, they behave like one enormous non-blocking switch. The pattern is a Clos network, an idea borrowed from 1950s telephone exchanges, and its datacenter form is the fat-tree. The word fat is the whole point: in an ordinary tree the links thin out as you go up, but a fat-tree keeps the total upward capacity at every level equal to the capacity below it, so the funnel disappears.
How do thin links add up to a fat one? Not by making any single cable thicker, but by making many parallel paths. Where a plain tree has one link going up, a fat-tree has several uplinks fanning out to several switches in the layer above. The aggregate of those parallel links carries as much as everything below, so a packet from one rack to another no longer fights for a scarce single path — it picks one of many equally-good routes up and over. Capacity scales by adding more cheap boxes sideways (scaling out), not by buying a bigger box (scaling up).
Two more wins fall out of this shape for free. First, resilience: with dozens of paths between any pair of racks, losing one switch removes one path out of many, and traffic just flows around it — no single failure takes down half the network. Second, cost and uniformity: every switch is the same small commodity model, so they are cheap to buy, easy to stock as spares, and simple to manage. You have replaced a few irreplaceable giants with a swarm of interchangeable ants.
Leaf-spine: the fat-tree everyone actually wires
In practice almost every modern datacenter wires a flattened two-layer fat-tree called a leaf-spine fabric. The leaf switches sit at the top of each rack (one or two per rack, connecting that rack's servers). The spine switches form the upper layer. The defining rule is brutally simple and worth memorizing: every leaf connects to every spine, and leaves never connect to leaves, spines never connect to spines. The result is that any server is exactly one hop away from any spine, and any other server is at most two hops further — leaf, up to a spine, down to the destination leaf.
SPINE 1 SPINE 2 SPINE 3 SPINE 4
| \ \ / | \ / | / / | ...
| (every leaf connects to EVERY spine)
LEAF A LEAF B LEAF C LEAF D
/||\ /||\ /||\ /||\
servers servers servers servers
rack A rack B rack C rack D
A server in rack A -> server in rack C always = 3 hops:
leaf A -> (any one) spine -> leaf C
With 4 spines there are 4 equal-length paths to choose from.This flatness is exactly what east-west traffic wanted. Every cross-rack flow takes the same predictable three hops, so latency between any two servers is uniform — there is no near-and-cheap versus far-and-expensive, the way the old tree had. And the number of spine switches is a dial you turn: more spines means more parallel paths and thus more bisection bandwidth, so you can build the fabric for full non-blocking capacity, or deliberately under-provision the spine for a known oversubscription ratio like 3 to 1 if your workload does not truly need all-to-all line rate. Scaling the building means adding leaves (more racks) and spines (more bandwidth), never replacing the core.
Spreading the load: ECMP and its honest limits
All those parallel paths are useless unless traffic actually uses them. When a leaf has, say, four equally-short paths up to four different spines, it spreads flows across them using equal-cost multipath (ECMP). The router's forwarding table now lists several next-hops of equal cost for the destination, and ECMP picks among them — so different conversations naturally fan out over the whole fabric instead of all stampeding down one link. This is the mechanism that turns 'many paths exist' into 'many paths get used.'
Here is the clever, careful detail. ECMP must not scatter the packets of a single flow across different paths, because paths can have slightly different delays and the packets would arrive out of order — and the transport layer reads reordering as a sign of trouble. So instead of choosing per packet, ECMP chooses per flow. It hashes a few fields from the packet header — typically the source and destination IP addresses and port numbers, the 'who-to-who' identity of the connection — and uses that hash to pick one path. Every packet of the same flow hashes to the same number, so they all follow the same route and stay in order, while different flows land on different routes.
Be honest about what this does and does not guarantee. ECMP spreads flows evenly only on average; it is a hash, not a scheduler, so it knows nothing about how big each flow is. Two enormous 'elephant' flows can hash to the very same spine link and collide there while three other spines sit idle — a hash collision that costs you real bandwidth. This is a genuine, well-known weakness of plain ECMP, and the fixes you will meet later (finer-grained flowlet switching, congestion-aware load balancing, and at the endpoint multipath TCP splitting one connection across paths on purpose) all exist precisely to patch this gap.
Putting it together
Trace one packet end to end and the whole design clicks. A server in rack A sends to a server in rack C. The packet rises to leaf A; leaf A sees several equal-cost paths up to the spines and hashes the flow's five-tuple to choose, say, spine 2; the packet crosses to spine 2; spine 2 forwards it down to leaf C; leaf C delivers it to the destination server. Three hops, one of several interchangeable spines, and a sibling flow from the same rack might take spine 4 at the very same instant. That is scale, resilience, and load-spreading all in a single trip.
- Servers in a rack connect to their leaf (top-of-rack) switch — this is the only place a server plugs in.
- Every leaf connects to every spine; leaves never touch each other and spines never touch each other.
- Any rack-to-rack path is leaf -> spine -> leaf, always the same short hop count, with one candidate path per spine.
- ECMP hashes each flow's header fields to pin it to one of those paths, spreading many flows across the fabric while keeping each flow in order.
- Need more bandwidth? Add spines. Need more servers? Add leaves. You never have to replace a giant core switch.