value decomposition networks

Value decomposition networks attack cooperative credit assignment with a clean, almost naive idea: assume the team's joint value is simply the sum of one value per agent. Each agent learns its own local action-value from its own observation, and you add them up to get the team value. Because the team reward trains the sum, every agent's piece is nudged in a direction that helps the whole — the single shared number is gently split into individual responsibility.

The payoff of this additive structure is decentralised execution for free. To act greedily as a team you would want to maximise the joint value, but if the joint value is a sum of per-agent terms, then each agent can just maximise its own term independently and the team is automatically acting greedily overall. So a centralised sum trains the parts, yet each agent decides alone at run time — textbook CTDE.

The honest limitation is that real teams are not always additive: sometimes two agents' values interact, and good joint behaviour cannot be written as a plain sum. VDN cannot represent those couplings and will misjudge such situations. That exact shortcoming is what motivated QMIX, which keeps the run-time decentralisability but allows a richer, monotonic (rather than merely additive) combination of the per-agent values.