TD(0)

TD(0) is the simplest temporal-difference update, and a great place to first meet the idea. Each time the agent moves from one state to the next, it looks one step ahead: it forms a target from the reward it just got plus the discounted value it currently assigns to the new state, then shifts the old state's value a small step alpha toward that target. That is the whole algorithm.

The single step ahead is what the zero refers to — it sits at one end of a family that stretches all the way to full Monte Carlo. TD(0) updates online after every transition, needs only a tiny amount of memory, and provably converges to the correct values for a fixed policy under standard conditions. It is the foundation that SARSA, Q-learning, and the deep RL methods all build on.