cooperative multi-agent RL

Cooperative MARL is the friendly corner of the field: every agent is on the same team and wants the same thing. Think of a fleet of warehouse robots that all win or lose together by how fast the orders ship, or players in a co-op video game chasing one shared score. Because the goal is shared, there is no rivalry to model — the whole challenge is getting individuals to coordinate into behaviour that is good for the group.

The clean way to state it is a Markov game where every agent receives the identical team reward at each step. That sounds easy, but two problems bite immediately. First, agents usually act on their own local observations and cannot see the whole state, so they must coordinate without full information. Second, since one number rewards everyone, an agent cannot tell from that number whether its own action helped or hurt — the multi-agent credit assignment problem.

Most of the well-known cooperative methods exist to crack exactly those two problems: centralised training with decentralised execution lets a shared critic see everything during learning while agents still act locally, and value decomposition methods like VDN and QMIX try to split the team value into per-agent pieces. It is the setting where MARL has worked best in practice, from robot soccer to the StarCraft micromanagement benchmarks.