a continuous-time Markov chain and its generator
/ MAR-kov /
A discrete-time chain ticks in whole steps, but real events happen at irregular moments in continuous time: customers arrive, atoms decay, machines fail. A continuous-time Markov chain (CTMC) is the memoryless process that lives on a continuous clock. It sits in a state for a random length of time, then jumps to a new state — and the Markov property holds at every instant: given the present state, the future is independent of how long it has already been sitting there or how it arrived.
Memorylessness in continuous time forces the holding time in each state to be exponentially distributed — the exponential is the unique memoryless continuous waiting time. The dynamics are captured by a generator (or rate matrix) Q. Its off-diagonal entry q(i, j) (for i not equal to j) is the rate of jumping from i directly to j; each diagonal entry q(i, i) is negative and makes each row sum to 0, recording the total rate of leaving i. From state i the chain waits an exponential time with rate equal to the total exit rate, then jumps to j with probability proportional to q(i, j). Stripping out the timing leaves the embedded jump chain, an ordinary discrete-time chain that records only the sequence of states visited.
The generator plays the role the transition matrix played before. Transition probabilities over a time t are P(t) = e^(Qt), the matrix exponential — the continuous-time analogue of P^n. A stationary distribution now solves pi Q = 0 (instead of pi P = pi), and detailed balance reads pi(i) q(i, j) = pi(j) q(j, i). CTMCs are the language of queueing theory (the M/M/1 queue), of birth-death and population models, of chemical reaction networks, and of reliability engineering — anywhere events strike a continuous clock at random rates.
An M/M/1 queue: customers arrive at rate lambda and are served at rate mu, the state being the number waiting. From state n (with n at least 1) the chain leaves at total rate lambda + mu, jumping up (an arrival) or down (a departure). Its generator has q(n, n+1) = lambda, q(n, n-1) = mu, and q(n, n) = -(lambda + mu). Holding time in state n is exponential with rate lambda + mu.
A CTMC waits an exponential time set by the exit rate, then jumps; the generator Q records all the rates.
The exponential holding time is forced, not chosen — it is the ONLY memoryless continuous waiting distribution. If holding times were, say, fixed or uniform, the process would not be Markov in continuous time, because how long you have already waited would predict when you leave.