the Kolmogorov-Sinai entropy
/ kol-mo-GOR-ov SEE-nigh /
Kolmogorov-Sinai entropy assigns to a measure-preserving system a single non-negative number h(T) that measures its rate of information production — how many bits of genuinely new randomness the dynamics generates per step. It is the master invariant that finally distinguishes systems that all other invariants (ergodicity, mixing, spectrum) fail to tell apart, and it sits at the top of the mixing hierarchy where chaos becomes quantitative.
Construction (in brief). For a finite measurable partition P = {P_1, ..., P_k} of Omega, its Shannon entropy is H(P) = - sum_i P(P_i) log P(P_i), the information in observing which cell you are in. Refining over time, form the join P^n = P or T^(-1) P or ... or T^(-(n-1)) P (the partition recording the cell-itinerary for n steps). The entropy of T relative to P is h(T, P) = lim_n (1/n) H(P^n) (the limit exists by subadditivity of H), and the Kolmogorov-Sinai entropy is h(T) = sup over all finite partitions P of h(T, P). The Kolmogorov-Sinai theorem makes this computable: if P is a generating partition (its iterates generate F mod 0) then h(T) = h(T, P), so one need not take the supremum. Equivalently, by Shannon-McMillan-Breiman, h(T) is the almost-sure exponential decay rate of the probability of a typical length-n orbit-name.
Why h(T) is decisive. It is an isomorphism invariant (isomorphic systems have equal entropy), and Ornstein's deep theorem makes it complete for Bernoulli shifts: two Bernoulli shifts are isomorphic if and only if they have the same entropy. This solved a long-standing problem — for instance the (1/2, 1/2) and (1/3, 1/3, 1/3) shifts are non-isomorphic precisely because their entropies log 2 and log 3 differ. Entropy also links to dynamics broadly: positive entropy is the hallmark of a K-system (chaos), zero entropy characterises predictable systems (rotations, and all systems with discrete spectrum), and the variational principle ties measure entropy to topological entropy. Honest cautions: entropy is a complete invariant for Bernoulli shifts but NOT in general — non-isomorphic systems can share an entropy value (Ornstein-Shields and others built such examples). And entropy is a property of (T, P), not of T alone: the same map has different entropies for different invariant measures, the variational principle picking out the maximiser.
The Bernoulli shift on an alphabet with probabilities (p_1, ..., p_k) has entropy h = - sum_i p_i log p_i exactly (the single-symbol partition is generating). So the fair-coin shift has h = log 2, the fair-die shift h = log 6, and the irrational rotation has h = 0 — quantifying precisely the gulf between coin-flips and a clockwork rotation.
Entropy turns 'how random?' into a number, and (for Bernoulli shifts) that number decides isomorphism.
Entropy is a complete isomorphism invariant only within Bernoulli shifts (Ornstein); in general two systems with equal entropy need not be isomorphic. Also h(T) depends on the invariant measure, not on the map alone.