the transition probability matrix

Once you have a Markov chain, you need a tidy way to store every one-step possibility. The transition matrix P is exactly that bookkeeping device: a square table whose entry P(i, j) in row i, column j is the probability of jumping from state i to state j in a single step. Read a row and you see the complete fortune of someone standing in that state — all their next-step odds laid out at once.

Because the chain must go somewhere from any state, each row's entries are non-negative and add up to 1. A matrix with this property is called a stochastic matrix; rows are probability distributions, not arbitrary numbers. (Columns generally do not sum to 1 — that is a different, special kind of matrix.) For a chain on m states, P is an m-by-m matrix, and it is the single object you compute with: multiplying a current distribution row vector by P pushes it forward one step.

The matrix view is what unlocks the whole theory. Multi-step behaviour is matrix powers, equilibrium is an eigenvector, and convergence speed is read from eigenvalues. The transition matrix turns probabilistic questions about a wandering token into clean linear algebra — which is why so much of Markov chain theory is really the study of stochastic matrices.