noisy networks for exploration

Most DQN agents explore with epsilon-greedy: act greedily, but with some probability flip a coin and do something random. It works, yet it is crude — the randomness is the same everywhere, blind to whether a state is already understood or genuinely novel, and it has to be hand-scheduled to decay over time. Noisy networks replace that bolted-on randomness with exploration baked into the network itself.

The idea is to add learnable noise to the network's weights: each weight gets a mean and a noise-scale, and on each forward pass random perturbations are drawn and injected. Early on the noise is large, so the agent's actions vary and it explores; as learning proceeds, gradient descent can shrink the noise where confidence is high and keep it where the agent is still unsure. Exploration thus becomes state-dependent and self-tuning, learned rather than scheduled. Noisy nets were one of Rainbow's six ingredients and often beat epsilon-greedy on hard-exploration games.