target network

When DQN updates Q(s,a) toward its target, the right-hand side is computed by the same network being trained. So every weight update changes the target too, like trying to hit a dartboard that lurches sideways the instant you throw. This feedback loop makes naive deep Q-learning oscillate or diverge. The target network breaks the loop by holding a frozen copy of the network to compute targets from.

You keep two sets of weights: the online network, updated every step, and a target network, a snapshot of the online weights that is refreshed only occasionally — copied over every ten thousand steps, or blended in slowly with a small step size each update. Targets computed from the slow copy stay stable for a stretch, giving the online network a fixed thing to chase. It is one of the two pillars, with experience replay, that turned Q-learning with neural nets from unstable to reliable.