TD(λ)

Rather than forcing a choice between a one-step TD update and a full Monte Carlo return, TD(λ) blends all of them at once. It forms the lambda-return, a weighted average of the one-step, two-step, three-step, and longer returns, where each extra step of horizon is discounted by another factor of lambda. The parameter lambda, between zero and one, is a single dial that controls how far the agent effectively looks ahead.

The dial spans the whole spectrum. With lambda at zero the lambda-return collapses to the plain one-step TD(0) target; with lambda at one it becomes the full Monte Carlo return. Intermediate values usually learn fastest, capturing more of the real future than TD(0) while staying far less noisy than Monte Carlo. TD(λ) can be computed efficiently online using eligibility traces rather than storing whole episodes.