mean hitting and first-passage times

Beyond 'where will the chain end up?' you often want 'how LONG until something happens?'. How many steps, on average, until the gambler goes broke? Until the queue empties? Until you first reach the goal square? These are first-passage or hitting times: the first time the chain enters a target set of states. Their expected values are mean hitting times, and they are among the most practically useful quantities a chain offers.

Let h(i) be the expected number of steps to first reach a target set A, starting from state i. There is a clean way to compute these all at once: condition on the first step. From a target state, h = 0. From any other state i, you take one step (costing 1) and then face the expected time from wherever you land: h(i) = 1 + sum over all j of P(i, j) h(j). That is one linear equation per non-target state, a small system you solve simultaneously. The same one-step / first-step analysis also gives absorption probabilities and expected costs.

A special case is the mean return time: starting at state i, the expected number of steps to come back to i. For an irreducible positive-recurrent chain this is finite and equals exactly 1 over pi(i), the reciprocal of the stationary probability — so high-equilibrium states are revisited quickly and rare states take long to return to. Hitting-time analysis powers gambler's ruin problems, the expected length of board games, queueing delays, and reliability calculations of when a system first fails.