backpropagation
Backpropagation is how a layered neural network learns from its mistakes. The network makes a guess, you measure how wrong it was, and then that error is passed backward through the layers, telling every single connection how much it was to blame. Picture a kitchen where a dish comes out too salty: the head chef traces the fault back down the line — a little from the sauce station, a little from the seasoning — and each cook nudges their part. Do this over and over with millions of examples, and the whole network gets better.
Under the hood it is just careful bookkeeping with the chain rule from calculus — a way to figure out, layer by layer, how a tiny change in each weight would change the final error. Working backward from the output is the clever bit: it computes all those blame-shares in one sweep instead of testing each weight separately, which would be hopelessly slow. The weights are then adjusted a notch in the direction that lowers the error, and the cycle repeats.
Its importance is hard to overstate: backpropagation is what made deep networks — stacks of many layers — actually trainable, and it powers nearly every modern AI, from image recognizers to chatbots. One common mix-up: backpropagation only finds the direction to adjust each weight; a separate step (gradient descent) decides how big a step to take. It is the messenger that delivers the error, not the hand that turns the dial.
For each weight w, backprop computes how the total error E would change if that weight changed — the gradient — by chaining the layers together.
The name is short for "backward propagation of errors." The method was popularized by a famous 1986 paper from David Rumelhart, Geoffrey Hinton, and Ronald Williams, though the core idea had been worked out independently by several people in the prior decades.