east-west traffic
Imagine drawing a network diagram on a whiteboard with the outside world (users, the Internet) at the top and the servers stacked in rows below. Traffic that goes up and down between the servers and the outside world is north-south — it enters and leaves the building. Traffic that runs sideways, server-to-server, never leaving the building, is east-west. The names come purely from how the diagram is usually drawn; there is no compass involved.
East-west traffic is communication between machines inside the same datacenter: an application server talking to a database, a web tier fanning a request out to dozens of microservices, storage nodes replicating data to each other, or thousands of GPUs exchanging gradients while training a model. In modern cloud datacenters this internal chatter dwarfs the north-south flow to and from users — east-west often accounts for the large majority of all bytes moved. A single click by a user at the edge can set off a cascade of internal requests that bounce between many racks before one answer comes back.
Why it matters: this single fact reshapes how datacenter networks are built. If most traffic were north-south, you would just need fat pipes to the gateway at the top. But because most traffic is east-west and can flow between any pair of servers, the network needs high bisection bandwidth — roughly, the ability to move data fast even if you cut the datacenter in half and every server on one side talks to a server on the other. That requirement is exactly why operators abandoned the old top-heavy three-tier design in favor of fat-tree and leaf-spine fabrics that give many equal-cost paths between any two servers.
Training one large language model can generate enormous east-west traffic: at each step, thousands of GPUs must exchange their computed gradients with one another. Almost none of this leaves the building — yet it can saturate the internal fabric far more heavily than user traffic ever does.
GPU-to-GPU gradient exchange is pure east-west traffic.
East-west versus north-south is just a description of direction on a diagram, not a protocol or a layer. The important takeaway is that east-west now dominates, which is why bisection bandwidth — not just gateway capacity — is the metric datacenter designers optimize.