From one chip to a synchronous bus: SDRAM and DDR
In the last guide we crawled inside a single DRAM chip: bits parked as charge on tiny capacitors, organized into banks, each bank a grid of rows and columns, with a fast row buffer holding the one row you most recently opened. We watched the RAS/CAS dance — strobe a row address to open a row, then strobe column addresses to read pieces out of the row buffer. Now we step back from the chip and ask the system question: how does the processor actually talk to a wall of these chips fast enough to keep itself fed?
The first big idea is to make memory synchronous. Early DRAM was asynchronous — the controller poked address and control lines and waited a fixed, advertised number of nanoseconds for data to appear. Synchronous DRAM (SDRAM) instead shares a clock with the memory controller and answers in step with the beat. That sounds like a small change, but it is the same trick we saw with synchronous design inside the CPU: once both sides march to one clock, you can pipeline requests, overlap them across banks, and reason about timing in whole cycles instead of fuzzy analog delays. SDRAM turned the memory bus into a disciplined, predictable assembly line.
DDR — Double Data Rate — adds the next twist. An ordinary clock signal goes up, then down, once per cycle; a plain synchronous bus moves one chunk of data on each rising edge. DDR SDRAM moves a chunk on the rising edge and the falling edge, so it transfers data twice per clock cycle without raising the clock at all. Think of a metronome ticking at 1000 beats per second: single data rate places a parcel on each tick (1000 parcels/s), while DDR places one on the tick and one on the tock (2000 parcels/s) from the very same metronome. This is why memory is rated by a 'transfer rate' (megatransfers per second, MT/s) that is twice its clock frequency.
Sticks and lanes: DIMMs, ranks, and channels
A single DRAM chip is narrow — it might hand over just 4 or 8 bits at a time on its data pins. A modern CPU wants a whole cache line (often 64 bytes) and wants it as a fat parallel gulp. The fix is to gang many chips together. A DIMM (dual inline memory module) is the stick you snap into a slot: a little printed-circuit board carrying several DRAM chips wired so their data pins combine into one wide bus — typically 64 bits (8 bytes) of data delivered together on every transfer (72 bits if it carries ECC for error checking, a topic for guide 5).
Within one DIMM, the chips that together form one full 64-bit-wide group are called a rank. A DIMM may hold one, two, or more ranks stacked behind the same data wires; the controller picks which rank to talk to with a chip-select signal. Ranks matter because while one rank is busy finishing a transfer or recovering, the controller can fire a command at another rank — yet another layer of overlap, on top of the bank-level overlap inside each chip. Ranks and banks are both about keeping the data wires busy by always having some part of memory ready to answer.
Now the biggest multiplier: a memory channel. A channel is one complete, independent path between the memory controller and a set of DIMMs — its own 64 data wires, its own command bus, its own controller logic. A two-channel system runs two of these paths side by side, doubling the data delivered per transfer; a four- or eight-channel server multiplies it further. Channels are the true source of headline memory bandwidth. Picture supermarket checkouts: one fast cashier (a channel) can only scan so quickly, but open eight lanes and the whole store empties far faster. The catch — you must spread shoppers across the lanes, which is exactly what the controller's job becomes.
The one formula: bandwidth = width x rate x channels
Everything above collapses into one honest equation for peak memory bandwidth — the maximum bytes per second the memory system can stream. Multiply three things: how wide each transfer is (the bus width in bytes), how fast transfers happen (the transfer rate, already including DDR's double pumping), and how many independent channels run in parallel. That is bandwidth = width x rate x channels. It is the throughput of the memory pipe, and it has nothing to do with how long any single access takes — bandwidth is about volume, not latency.
Worked example: a dual-channel DDR4-3200 desktop bus width per channel = 64 bits = 8 bytes transfer rate = 3200 MT/s (1600 MHz clock, double-pumped) channels = 2 per-channel bandwidth = 8 bytes x 3200 M/s = 25.6 GB/s total bandwidth = 25.6 GB/s x 2 channels = 51.2 GB/s Double the channels (4) or move to DDR5-6400 and the number climbs the same way -- it is just width x rate x channels. Note: this is the PEAK ceiling, never the steady speed you see.
Be honest about that gap between the peak and what you really get. The formula assumes transfers never stop — but they do. Every time an access lands in a closed row, you pay the full RAS-then-CAS open before any data flows; the chip periodically goes deaf for refresh; and two requests aimed at the same bank must wait their turn. Sustained bandwidth on real workloads is often only 60-80% of the printed peak. The peak number is a genuine ceiling — useful for comparing parts and for the roofline model later — but no program ever lives at the ceiling.
The controller's craft: scheduling for parallelism
All those channels, ranks, and banks only help if something keeps them all busy at once — and that something is the memory controller, the busy traffic cop sitting between the last-level cache and the DIMMs. It receives a stream of read and write requests (every one a cache miss that escaped the on-chip caches) and it does not simply serve them in arrival order. Instead it reorders them to win two prizes: hits in an already-open row buffer, and spreading work across independent banks and channels so several accesses are in flight together. This bank-level parallelism is what turns raw width and rate into delivered bandwidth.
- A load misses every cache, so a 64-byte cache-line request arrives at the memory controller carrying a physical address.
- The controller decodes that address into channel, rank, bank, row, and column — choosing the mapping so that consecutive cache lines spray across different channels and banks rather than piling on one.
- If that row is already sitting in the bank's row buffer, it is a row-buffer hit: issue CAS only, and data flows almost immediately.
- If a different row is open, that is a conflict: close the old row, RAS the new one, then CAS — the slow path, the full RAS/CAS latency.
- Meanwhile the controller fires commands at other banks and channels so their accesses overlap this one, and it batches reads and writes to avoid costly turnarounds on the shared data bus.
This is why two programs with identical instruction counts can have wildly different memory speed — the same lesson as cache-friendly code, now one level lower. Code that walks memory with good spatial locality keeps hitting the open row buffer and lets the controller stripe cleanly across banks; code that jumps around randomly forces a row conflict almost every time and serializes onto one bank. The hardware bandwidth is fixed, but how much of it you earn depends on your access pattern. The controller is clever, but it cannot rescue an access pattern that fights the memory's structure.
Where this leaves us: bandwidth grew, latency didn't
Stack up the whole story and a sober picture emerges. Across the DDR generations, bandwidth has multiplied many times over — double pumping, ever-higher transfer rates, more channels, wider buses. But the latency to the first byte of a fresh, closed-row access has improved painfully little: the RAS/CAS open of a DRAM array is still tens of nanoseconds, much as it was generations ago, because it is set by analog physics — charging bit lines and waking sense amplifiers — that simply does not shrink the way digital logic does. We bought a fire hose, but the time for the first drop to arrive barely changed.
That split is the punchline that sets up the rest of the rung. Bandwidth can be bought with parallelism — more channels, more banks, the whole structure of this guide. Latency cannot: it is a physical floor. And meanwhile the processor's appetite has only grown — more cores, wider pipelines, all hungry for data at once. The widening gulf between how fast a CPU can compute and how fast memory can feed it is the memory wall, the subject of the very next guide, where we will meet high-bandwidth memory and 3D stacked memory — radical answers built by attacking the bandwidth side of exactly this equation.