discount factor
/ DIS-kownt FAK-tur /
The discount factor is a single dial, usually written gamma (γ), that sets how much the agent cares about the future versus right now. A reward one step away counts in full; a reward two steps away is multiplied by gamma; three steps away, gamma times gamma; and so on, shrinking with every step of delay. It is the mathematical expression of 'a bird in the hand is worth two in the bush' — future rewards are worth a little less than present ones.
Gamma lives between 0 and 1. Set it near 0 and the agent becomes shortsighted, grabbing whatever pays off immediately and ignoring long-term consequences. Set it near 1 (say 0.99) and the agent becomes far-sighted, willing to forgo a small reward now for a bigger one many steps later. Tuning gamma is genuinely choosing the agent's personality — its patience.
There's also a quiet technical reason gamma exists. In tasks that can run forever, simply adding up rewards gives infinity, which is mathematically useless for comparing strategies. Discounting tames that infinite sum into a finite number, so 'better' and 'worse' stay meaningful. The catch is that gamma is a knob you set, not something the world tells you — pick it too low and the agent won't plan ahead enough to ever solve the task; pick it too high and learning can become slow and unstable.
Offered +100 in ten steps, with gamma = 0.9, the agent values it today at 100 × 0.9^10 ≈ 35. With gamma = 0.99 it values the same reward at ≈ 90 — almost full. The same prize, the same delay, but two very different appetites for waiting.
Each step of delay multiplies a reward by gamma. Low gamma = impatient; high gamma = far-sighted.
Gamma is a design choice, not a fact about the task, and it quietly shapes what the agent will even attempt. Too low and it can't 'see' a distant goal, so it never learns to reach it; too high and the far-off, uncertain sums make learning noisier and slower to settle.