Continuous-Time Chains & Jump Processes

the embedded jump chain

The embedded jump chain is the discrete-time Markov chain you get by recording only the SEQUENCE of states a continuous-time chain visits, ignoring how long it stays in each. It is the way to reduce a CTMC to the familiar discrete-time theory of Vol I: jump chain plus holding times together rebuild the full continuous-time process. The question it answers is: where does the chain go, separated cleanly from when?

Let Y_0, Y_1, Y_2, ... be the successive distinct states visited by X_t. Then (Y_n) is a discrete-time Markov chain with transition matrix obtained by normalizing the off-diagonal rates: for a non-absorbing state i (q_i > 0), the jump probability is K(i,j) = q_ij / q_i for j not equal to i, and K(i,i) = 0; an absorbing state (q_i = 0) gets K(i,i) = 1. The full CTMC is then reconstructed by: from i, draw a holding time exponential(q_i), then jump to j according to K(i,.), independently. This is the construction direction: given a jump kernel K and holding rates q_i, you build the process. Many qualitative questions about the CTMC reduce to the jump chain: which states are recurrent or transient, communicating classes, and whether absorbing states are reached are properties of (Y_n) alone, because they do not involve time.

The jump chain captures the 'where' but discards the 'when', so it cannot by itself answer timing questions: expected time to absorption, stationary FRACTIONS of time, or whether the chain explodes all need the holding rates too. A sharp warning: positive recurrence of the jump chain is NOT the same as positive recurrence of the CTMC, and the stationary distribution of the jump chain is generally different from the stationary distribution of the CTMC. They are related by re-weighting with the mean holding times: pi_CTMC(i) is proportional to nu(i)/q_i, where nu is the jump-chain stationary distribution. Confusing the two stationary distributions is a classic error.

A pure-death chain on {0,1,2,...} with q_n = death rate n*mu and only jumps n -> n-1. Its jump chain is the deterministic walk n -> n-1 (it always goes down), so 'where' is trivial; all the interest is in the holding times, whose sum (time to reach 0) is a sum of independent exponentials of rates n*mu.

When the 'where' is trivial, the jump chain says little and the holding times carry all the information — and vice versa.

The jump chain's stationary distribution is NOT the CTMC's: pi_CTMC(i) is proportional to nu(i)/q_i, re-weighting the jump-chain stationary nu by mean holding times 1/q_i. Conflating the two is one of the most common mistakes in CTMC analysis.

Also called
jump chainembedded chainskeleton chain嵌入鏈跳躍鏈骨架鏈