Applications: Data, Graphs & Dynamics

stationary distribution

A stationary distribution is the resting point of a random process: if the chain is already in this distribution, taking one more step leaves it unchanged. It is the long-run fraction of time the system spends in each state, regardless of where it began (for nice chains).

Formally, for a Markov matrix P it is a probability vector pi with pi = P pi (column convention) and entries summing to 1. So pi is an eigenvector of P for eigenvalue 1. In the row convention it is the left eigenvector: pi^T = pi^T P. Solving (P - I) pi = 0 together with the normalization sum(pi) = 1 pins it down.

Existence is guaranteed because 1 is always an eigenvalue of a stochastic matrix. Uniqueness and convergence from any start come from Perron-Frobenius: if the chain is irreducible (every state reachable from every other) and aperiodic, then pi is unique and P^k p_0 -> pi for any starting p_0.

Why it matters: the stationary distribution is what PageRank computes, what tells you the equilibrium occupancy of a queue, and what makes Markov chain Monte Carlo work. The caveat: a periodic chain has a stationary distribution but never settles into it (it keeps cycling), and a reducible chain can have many stationary distributions depending on which trap you fall into.

P = [0.9, 0.5; 0.1, 0.5] -> solve pi = P pi, sum=1 -> pi = (5/6, 1/6)^T

The stationary vector is the eigenvector for eigenvalue 1, normalized to sum to one.

Stationary is not the same as reversible. A stationary distribution just satisfies pi = P pi; a reversible chain additionally satisfies detailed balance pi_i P_ji = pi_j P_ij, a stronger symmetry that many sampling algorithms exploit.

Also called
equilibrium distributioninvariant distribution