novelty search

Novelty search makes a radical bet: ignore the objective entirely and reward only doing something new. Rather than scoring agents by how much task reward they earn, it scores them by how different their behaviour is from everything seen before, measured in a space of behaviour descriptors. The population is pushed relentlessly toward novelty for its own sake, with no direct pressure to actually solve the task.

Counter-intuitively this often beats objective-driven search on deceptive problems, where greedily climbing the reward leads straight into a dead end — chasing novelty keeps generating stepping stones that a reward-blind explorer would never visit, and the solution is frequently found among them. The ideas grew out of evolutionary computation and now blend with RL in quality-diversity methods, which seek a whole collection of behaviours that are both novel and high-performing rather than one single best policy.