episodic MDP

An episodic MDP is one that comes in self-contained chapters. The agent starts, interacts for a while, hits an end, by winning the game, crashing the car, or running out of time, and then the world resets and a fresh episode begins. Each playthrough of a video game level or each hand of cards is one episode, with a clear beginning and a clear stop.

Technically an episode ends when the agent reaches a terminal or absorbing state, after which no further reward accrues. This structure is convenient: returns are finite sums over a bounded chapter, you can compare whole episodes, and Monte Carlo methods that wait for an episode to finish before learning become possible. It contrasts with continuing tasks, which never reset and rely on discounting instead. Many continuing problems are sliced into artificial episodes just to make learning easier.