multi-agent reinforcement learning (MARL)

Multi-agent reinforcement learning puts several learning agents in one world at once. The moment they share an environment, each agent's surroundings become non-stationary, because the others are learning and changing too, so a target that looked optimal yesterday may be wrong today. Cooperation, competition, and even communication all become part of what must be learned.

Its proper foundation is game theory: the setting is a stochastic, or Markov, game, and the right notion of a solution is an equilibrium such as Nash or correlated equilibrium rather than a single optimum. The dominant practical paradigm is centralized training with decentralized execution, where a centralized critic sees all agents' observations and actions during training, as in MADDPG or value-factorization methods like QMIX, while each policy acts on only its own local observation at run time. The hard parts are non-stationarity, assigning credit among agents, and a joint action space that grows exponentially in the number of agents.

Self-play and population-based training drive striking emergent strategies, but convergence guarantees are weak outside special structures such as zero-sum or potential games.