random network distillation (RND)

RND is a strikingly simple way to measure novelty. Fix a randomly initialised neural network and never train it — it maps each state to some arbitrary but fixed vector. Then train a second network to imitate the first, using the states the agent actually visits. On states seen often the imitator matches well; on a fresh, unvisited state it has had no chance to practise, so its error is large. That error is the novelty bonus.

Because the target is a deterministic function of the state, there is no inherent randomness for the predictor to chase — it sidesteps the noisy-TV problem that plagues forward-model curiosity, since prediction error can only come from unfamiliarity, not from stochastic dynamics. Cheap, stable, and easy to add, RND helped drive some of the first strong results on hard-exploration games like Montezuma's Revenge.