hyperparameter
/ HY-pur-puh-RAM-uh-tur /
A hyperparameter is a setting you choose before training starts — a knob on the learning process itself, rather than something the model learns. Think of baking bread: the recipe's amounts of flour and water are like parameters the dough "works out" as it rises, but the oven temperature and baking time are choices you make in advance. Set them poorly and even good ingredients give you a brick or a soggy loaf. Hyperparameters are those oven settings for machine learning.
Typical hyperparameters include how big to make the model (how many parameters or layers), how fast it should adjust during training (the learning rate), how long to train, and how strongly to discourage overfitting. The crucial difference from ordinary parameters is who sets them: parameters are discovered automatically by the training algorithm from the data, while hyperparameters are set by the human (or by an outer search) and are held fixed while that training runs.
Choosing them well is part science, part trial and error — a process called hyperparameter tuning. You train the model many times with different settings and keep whichever does best on a validation set, data held aside precisely so you don't fool yourself. There is no universally best choice; good values depend on the data and the task, which is one everyday face of the no-free-lunch idea. The honest caveat: tune against your test set and you will get a flattering number that quietly evaporates in the real world.
Two engineers train the same network on the same data. One uses a learning rate of 0.1, the other 0.001. The first model overshoots and never settles; the second crawls but lands well. Same data, same architecture, same parameters-to-be-learned — only one hyperparameter changed, and it decided everything.
One hand-set knob (the learning rate) can make or break a model.
The "hyper" just means "above": these knobs sit one level above the ordinary parameters and govern how those parameters get learned. Always tune them on a validation set, never on the test set you reserve for the final, honest score.