Gaussian process
/ GOW-see-un PROSS-ess /
A Gaussian process is a way of being uncertain about whole curves, not just numbers. Most methods fit a single best line through your data points; a Gaussian process instead keeps a probability distribution over every function that could plausibly pass through them. Picture a bundle of squiggly curves all threaded through your observed points: where you have data they bunch tightly together, and where data is missing they fan out wide. That fanning is the model honestly admitting it doesn't know.
What ties the curves together is a 'kernel,' a rule for how similar two inputs are. The usual choice says nearby inputs should give nearby outputs — the curve is smooth, not jagged. Feed the Gaussian process your data and the kernel, and out comes, for any new point you ask about, both a prediction and an error bar: the most likely value and a candid measure of how unsure it is, growing automatically as you move away from where you actually observed something.
Why it matters: that built-in error bar makes Gaussian processes a favorite when getting each data point is expensive and knowing your uncertainty is crucial — tuning a costly experiment, optimizing an industrial process, modeling sensors. They work beautifully from very few points and need no hand-set architecture. The honest catch is cost: the standard method's work grows with the cube of the number of data points, so plain Gaussian processes choke on large datasets, and their quality leans heavily on choosing a sensible kernel. They are the tool of choice for small, precious data — not for millions of rows.
Predicting a hilly terrain's height from a handful of survey stakes. Right next to a stake, the Gaussian process is confident and its error bar is tiny. Halfway between two distant stakes, it still guesses a smooth height but draws a wide error band — honestly flagging that the valley between could dip or rise. Add a stake there and the band collapses to nearly nothing.
A Gaussian process is confident near data and visibly unsure far from it — uncertainty you get for free, not as an afterthought.
Gaussian processes give uncertainty almost for free, but their compute cost scales with the cube of the dataset size, so they don't naturally fit big data without approximations. And their predictions are only as sensible as the kernel you choose — a poor kernel gives confident nonsense.