prediction-error exploration

This is a broad idea that underlies much of curiosity-driven RL: let the agent be drawn to wherever its own predictions fail. The agent trains a model to forecast something — the next state, a feature of it, the output of another network — and turns the size of the forecasting error into a bonus reward. High error means an unfamiliar, information-rich situation, so chasing error means chasing learning.

The appeal is that it needs no visit counts and scales to high-dimensional inputs; the danger is that not all prediction error is worth seeking. Error caused by genuine novelty is useful, but error caused by inherent randomness — noise the agent can never predict no matter how long it looks — is a trap that can freeze a curious agent in front of pure static. The various curiosity methods are largely distinguished by how they keep the first kind of error and discard the second.