the Chapman-Kolmogorov equations

Suppose you want the probability of getting from city A to city C in exactly five days, but the map only lists single-day hops. The natural idea is to break the journey at some midpoint: go from A to some city B in two days, then from B to C in three days, and add up over every possible stopover B. The Chapman-Kolmogorov equations are exactly this common-sense rule of splitting a long trip through an intermediate stop, made precise.

Write P^(n)(i, j) for the probability of going from i to j in n steps. Then for any split n = m + r, the equation says P^(n)(i, j) = sum over all states k of P^(m)(i, k) times P^(r)(k, j). In words: the chance of getting from i to j in n steps equals, summed over every possible halfway state k, the chance of reaching k in m steps times the chance of finishing from k to j in the remaining r steps. The Markov property is what licenses this: once you are at k after m steps, how you got there is irrelevant to the rest.

In matrix language this becomes beautifully simple: the n-step matrix is just a matrix product, P^(n) = P^m times P^r, and in particular the n-step matrix is the ordinary matrix power P^n. So the equations are the probabilistic reason that 'multiply the transition matrices' computes multi-step probabilities. They are the foundation on which the entire long-run theory of Markov chains is built.