energy-based models (EBM)

An energy-based model assigns every possible configuration a scalar energy, low for plausible data and high for implausible data, and declares the probability to be proportional to the exponential of the negative energy. This is the most flexible way to specify a density, because the energy can be any function you like, an arbitrary neural network, with no architectural constraints of invertibility or tractable sampling. The price is the normalizing constant, the integral of the exponential over all configurations, which is almost always intractable.

Maximum-likelihood training needs the gradient of the log-normalizer, which equals the model's own expectation of the energy gradient, an expectation you cannot compute in closed form. Contrastive divergence approximates it by drawing negative samples from short Markov chain Monte Carlo runs, typically Langevin dynamics that follow the energy gradient plus noise, and then lowering the energy of real data while raising the energy of these synthesized negatives. Score matching and noise-contrastive estimation offer alternative, sampling-light routes to fit the same unnormalized model.

EBMs are conceptually central because diffusion, score matching, and many classical models are all unnormalized-energy methods in disguise, but training stability and slow MCMC sampling have kept pure EBMs less practical than their score-based and flow-based relatives.