multi-agent reinforcement learning

Picture a busy four-way intersection where every car is its own learner, each trying to get home faster, each reacting to what the others do. That is the world of multi-agent reinforcement learning: instead of one agent facing a fixed environment, several agents share the same world, and the consequences of one agent's action depend on what everyone else chose at the same moment. It is how you model traffic, markets, team sports, swarms of robots, and games like poker or StarCraft.

Formally, each agent has its own observations, its own set of actions, and its own reward. The catch is that the environment's next state depends on the joint action of all agents at once, so from any single agent's point of view the rules of the game keep shifting as the others learn. The objects studied are mostly Markov games (also called stochastic games), and the solution concepts come from game theory rather than from a single optimal policy.

MARL is powerful but genuinely harder than single-agent RL for three reasons that recur throughout this field: the environment is non-stationary because everyone is learning at once, the joint action space grows exponentially with the number of agents, and when a team shares one reward it is hard to tell which agent actually earned it. Almost every technique here is an answer to one of those three problems.