finite vs infinite horizon

The horizon is how far into the future the problem runs. A finite-horizon task has a known deadline, like play exactly twenty moves then stop, so the best action can depend on how many steps remain. An infinite-horizon task has no fixed end; the agent keeps interacting indefinitely, as a thermostat or a trading bot does, and behaves the same way regardless of the clock.

The distinction changes the maths. With a finite horizon the optimal policy is generally nonstationary, varying with time, and you can solve it by working backwards from the last step. With an infinite horizon you lean on the discount factor to keep returns finite, and the optimal policy is usually stationary, one fixed rule for every state. Many practical problems are episodic but long, and are modelled as infinite-horizon for convenience.