n-step temporal-difference

N-step temporal-difference learning is simply TD that uses an n-step return as its target. Where TD(0) shifts a state's value toward a one-step estimate, n-step TD shifts it toward the sum of n actual rewards plus the bootstrapped value beyond them. By tuning n you smoothly interpolate the whole family between one-step TD and Monte Carlo, choosing how much to trust real experience versus the agent's own estimates.

The practical catch is timing. The update for the state visited at time t cannot be applied until n more steps have elapsed and those rewards are known, so the algorithm runs with a short delay and a small buffer of recent transitions. In return, the credit for an outcome reaches back several states at once, which usually makes learning markedly faster than waiting for one-step errors to slowly propagate.