variational continual learning (VCL)

When a network learns a new task it tends to overwrite the old one, a problem called catastrophic forgetting. VCL prevents this with a simple Bayesian recursion: the posterior you obtained after learning task one becomes the prior when you learn task two. Knowledge accumulates instead of being erased, because the influence of old data is carried forward in the prior rather than needing to be revisited.

Bayes' rule on streaming data factorizes: the posterior after t tasks is proportional to the new task's likelihood times the posterior after t minus one tasks, so in principle you never need the old data again. VCL realizes this for deep nets by approximating each posterior with variational inference: the new variational distribution is fit to the new data using the previous one as the prior, a KL anchor that pulls weights that mattered before back toward their earlier values, usually aided by a small coreset of retained examples to combat approximation drift.

Nguyen and colleagues showed this principled recursion outperforms heuristic regularizers like EWC on standard continual-learning benchmarks while needing no task-specific tuning. Its limits mirror those of variational inference: mean-field approximation errors compound over a long task sequence, the running prior gradually loses fidelity, and a coreset, a little memory replay, is usually needed to keep performance from degrading.