absorbing state

An absorbing state is a place you can check in to but never leave. Once the agent enters it, every action loops straight back to the same state and pays nothing more; the game is simply over. The checkmated board, the crashed car, the reached goal: each is a dead end the agent can never escape and from which no further reward flows.

This little trick is what lets an episodic task be written as a single tidy MDP. By making the terminal state absorbing with zero reward, an episode that ends is really just an infinite stay in a state where nothing happens, so the same infinite-horizon machinery and the same Bellman equations apply without special cases. In practice the bookkeeping marks the step as terminal and resets the environment for the next episode.