stochastic gradient Langevin dynamics (SGLD)

Ordinary SGD walks downhill to a single minimum. SGLD adds a precisely calibrated dose of Gaussian noise to each step so the walk never quite settles; instead it wanders around the posterior, and the sequence of weights it visits becomes a stream of correlated samples from the parameter posterior. It is the bridge between optimization and Markov chain Monte Carlo.

Welling and Teh's insight is that if the injected-noise variance is matched to the step size, with noise standard deviation equal to the square root of twice the step, then as the step size decays the SGLD update transitions from SGD-like optimization into a discretization of Langevin diffusion whose stationary distribution is the posterior, and the Metropolis-Hastings correction becomes negligible. Using minibatch gradients keeps each step cheap, making it a scalable form of MCMC.

SGLD is the foundation of stochastic-gradient MCMC; descendants add momentum (SGHMC), preconditioning, and thermostats. In deep nets the main practical issue is mixing: the chain explores one region slowly, and minibatch-gradient noise contaminates the carefully tuned Langevin noise, so realistic samplers need many steps and careful step-size schedules.