discounted return

Imagine you are an agent collecting rewards over time. The discounted return is the total reward you expect to gather from now until the end, but with a twist: rewards that arrive later count for less. A reward today is worth more than the same reward ten steps from now, the way money in your pocket beats money promised next year. This is the actual thing an agent's choices try to make large — not the immediate reward, but the whole future stream, weighted toward the near term.

Concretely, you sum the rewards but multiply each future reward by the discount factor raised to how many steps away it is, where that factor sits between 0 and 1. With it near 0 the agent is myopic, caring almost only about the next reward; near 1 it weighs the distant future almost equally. Discounting also keeps the sum finite in tasks that never end, and it quietly encodes a preference for getting good outcomes sooner.