dueling network architecture

In many states the exact action barely matters — if you are about to crash no matter what, every action is roughly as bad, and the agent wastes capacity learning a separate value for each. The dueling architecture splits the question in two: how good is being in this state at all, and how much better or worse is each action than average here. It learns these as separate streams and recombines them into the Q-values you actually use.

Internally the network shares early layers, then forks into a value stream V(s), a single number for the state, and an advantage stream A(s,a), one number per action. They are merged so that the Q-value is V(s) plus the advantage minus the mean advantage, the subtraction keeping the decomposition identifiable. The benefit is that the state-value stream gets updated on every step regardless of the action taken, so the agent learns which states matter faster, especially where action choice is irrelevant. It is purely architectural and slots into DQN unchanged.