Physics 1877

On the Relationship Between the Second Law and Probability

Ludwig Boltzmann

Entropy is counting: the world drifts toward disorder because disorder has overwhelmingly more ways to happen.

Choose your version

In depth · the introduction

Heat flows one way, smells spread, rooms get messy — and Boltzmann showed it all comes down to counting.

The big idea

Boltzmann asked a startling question: what is entropy actually made of? His answer was that it is a measure of how many ways a thing can be arranged while still looking the same. A neat pile of cards has essentially one tidy arrangement; a shuffled deck has billions. "Disorder" simply means there are vastly more ways to be that way.

From this one insight the whole second law of thermodynamics — "things run down, heat spreads, order decays" — stops being a mysterious rule and becomes a near-certainty of arithmetic. A system drifts toward its most disordered state for the same reason a shuffled deck almost never falls back into order: not because order is forbidden, but because the disordered arrangements outnumber the orderly ones beyond all imagining.

How it came about

In 1877, working in Graz, Ludwig Boltzmann set out to give the second law a deeper foundation. Earlier physicists — Carnot, Clausius — had described entropy and shown it always increases, but no one could say why. Boltzmann's bet was that matter is made of countless tiny atoms, and that the law of heat is really a law of probability for those atoms.

It was a lonely bet. Many leading scientists, including Ernst Mach and Wilhelm Ostwald, did not believe atoms were real at all, and they attacked his work for years. Boltzmann, prone to deep depression, was worn down by the fight; he took his own life in 1906 — just before experiments on the jitter of pollen grains finally proved atoms real and proved him right. His tombstone in Vienna carries the equation S = k log W, in the tidy form his rival-turned-heir Max Planck gave it.

Why it mattered

This is the birth of statistical mechanics: the art of explaining the big, smooth, measurable world — temperature, pressure, heat — from the frantic statistics of unimaginably many atoms. It gave the "arrow of time" a reason. And decades later the very same counting reappeared, almost unchanged, as the formula for information itself, tying the physics of heat to the modern science of data.

A way to picture it

Picture a child's bedroom. There is basically one way for it to be "tidy" — every toy in its exact place. But there are millions of ways for it to be "messy," toys strewn anywhere at all. So if things get bumped around at random, the room overwhelmingly ends up messy, simply because messy has so many more ways to happen. Entropy is that count of ways; the second law is just the room obeying the odds.

Where it sits

Carnot (1824) had measured the efficiency of heat engines and Clausius named the entropy that always grows; Boltzmann told us what that entropy is. His way of counting states was then borrowed by Max Planck to crack the black-body problem in 1900 (see Planck 1900) — the opening move of quantum physics — and borrowed again by Claude Shannon in 1948, whose formula for information is Boltzmann's, letter for letter (see Shannon 1948).

The original document

Original source text

L. Boltzmann · Wien. Ber. 76 (1877): 373–435 · trans. Sharp & Matschinsky, Entropy 17 (2015)

Clausius and Carnot had given the second law a precise but mute form: in any isolated change a certain quantity, the entropy, can only increase. Why it must increase — what entropy is made of — they did not say. Boltzmann's proposal is that the answer is combinatorial: count the molecular arrangements, and the law follows from arithmetic.

Macrostate and microstate

Describe a gas two ways. The macrostate is the coarse, measurable description — so much energy spread over the molecules in such-and-such a distribution. A microstate (Boltzmann's "complexion") is the exact list of which molecule has which share. Many microstates look identical from the outside; the number that realise a given macrostate is its permutability measure, what we now call W.

For a distribution that places N₁ molecules in the first energy cell, N₂ in the second, and so on, that number is the multiplicity W = N! / (N₁! N₂! ⋯). The macrostate with the most complexions is the most probable; maximising W under fixed particle number and fixed total energy yields exactly the Maxwell distribution of molecular speeds.

Entropy is the logarithm of W

Because independent systems multiply their numbers of complexions while their entropies must add, entropy can only be proportional to the logarithm of the permutability measure. The second law then ceases to be a separate axiom: an isolated system drifts toward equilibrium simply because the equilibrium macrostate is realised by so vastly many more microstates that any large departure from it is, in practice, never seen.

[ … ]

Irreversibility, on this view, is not a law of mechanics but a statement of overwhelming odds. The molecules obey time-reversible equations; what makes heat flow one way is only that the disordered outcomes outnumber the ordered ones beyond all imagining. (The compact engraving S = k log W, with the constant k, is Max Planck's later form, c. 1900; Boltzmann wrote the proportionality, not the constant.)

Ludwig Boltzmann · Graz, 1877