inductive bias
/ in-DUK-tiv BY-us /
Inductive bias is the set of built-in assumptions a learning method uses to choose between the countless rules that all fit the training data equally well. Here is the problem it solves: any finite batch of examples can be explained by infinitely many different rules. Shown the numbers 2, 4, 6, you might guess "add 2 each time" — but "the even numbers," "double the position," or some bizarre formula all fit too. To pick one, a learner must lean on a preference for some kinds of explanations over others. That leaning is its inductive bias.
Without such a bias, learning from examples would be impossible — you could never reach beyond the data you have already seen. The bias is what lets a model generalize. Different methods bake in different assumptions: a straight-line model assumes the world is roughly linear; a convolutional network for images assumes that a pattern is the same wherever it appears in the picture; a decision tree assumes the answer can be reached by a series of yes/no splits. None of these is universally correct — they are bets about the shape of the world.
The deep point is that bias here is not a flaw; it is the engine. A learner with the right inductive bias for a problem learns fast from few examples; a learner whose bias mismatches the problem learns slowly, or wrongly, no matter how much data you throw at it. This is the same insight the no-free-lunch theorem makes precise: there is no assumption-free learner that is best at everything. Choosing a model is, at heart, choosing which assumptions to make.
Why does a network for images use convolution? Because designers baked in the assumption that a cat is a cat whether it sits in the top-left or bottom-right of the photo. That single inductive bias lets the model learn from far fewer pictures than a network forced to relearn "cat" separately in every position.
The right built-in assumption lets a model learn far more from far less.
No inductive bias, no learning: a method with zero assumptions cannot generalize beyond its examples at all. The skill is matching the bias to the problem — a strength on one task is a weakness on another.