weight
/ wayt /
A weight is a number that says how much one input matters. Think of deciding whether to go for a hike: the weather might count a lot, your free time somewhat, and the brand of your shoes hardly at all. Each of those factors gets a mental weight, and your decision is really a weighted vote. In a neural network, every connection between neurons carries exactly such a number. A large positive weight means "when this input is strong, push the output up"; a large negative weight means "push it down"; a weight near zero means "basically ignore this input."
Weights are the things a network actually learns. When people say a model has "175 billion parameters," the vast majority of those parameters are weights. They start out as small random numbers — the network knows nothing — and during training an algorithm nudges each weight up or down, again and again, so the network's outputs drift closer to the right answers. Learning, in a neural network, is almost entirely the slow tuning of these millions of dials.
It helps to separate weights from two neighbors. A weight scales an input; a bias is an extra standalone number added at the end, independent of any input. And weights are different from activations: a weight is a fixed setting the network has learned and keeps, while an activation is a temporary value that flows through when you feed in a particular example. Weights are the recipe; activations are this evening's dish.
A spam filter learns the weight on the word "viagra" should be strongly positive (it pushes toward "spam"), while the weight on "meeting" should be negative (it pushes toward "not spam"). Words it has learned to ignore, like "the," end up with weights near zero. The filter never saw a rule written by hand — it discovered these weights from labeled examples.
Weights encode learned importance: strongly positive, strongly negative, or near-zero (ignore).
Weights are learned and stored; activations are computed fresh for each input. Confusing the two is one of the most common beginner mix-ups — remember that the weights stay the same whether you feed the network a cat photo or a dog photo.