episode vs continuing task

Some tasks have a natural ending. A chess game ends in win, loss, or draw; a robot run ends when the cup is placed or dropped. Each self-contained run from start to finish is an episode, after which the world resets and a fresh one begins. These are episodic tasks, and we usually care about total reward gathered within one episode.

Other tasks just keep going with no reset in sight—a thermostat regulating a building, a system balancing a power grid, a bot serving traffic indefinitely. These are continuing tasks. Summing reward over infinite time would blow up, so we typically discount future reward, valuing it a little less the further off it is, which keeps the total finite and well-defined.