n-step returns

An n-step return is the practical middle ground between one-step TD and full Monte Carlo. Instead of bootstrapping after a single step, the agent walks forward, banks the first n real, discounted rewards it actually receives, and only then plugs in its current value estimate of the state it has reached. Set n to one and you recover the TD(0) target; let n grow to the end of the episode and you recover the full Monte Carlo return.

Choosing n trades off two errors. Small n leans heavily on the agent's own value estimates, so it has low variance but inherits whatever bias those estimates carry. Large n uses more real rewards, lowering bias but letting random outcomes accumulate into more variance. Intermediate values of n typically learn fastest, which is why n-step methods sit at the core of modern algorithms.