mixed cooperative–competitive settings

Most interesting multi-agent situations are not purely cooperative or purely adversarial — they are both at once. In a five-a-side football match your four teammates share your goal while the other team fights against it; in a marketplace, sellers cooperate to keep a market alive yet compete for each customer. These mixed settings, formally the general-sum Markov games, are where teamwork and rivalry are braided together.

Here an agent has to do two jobs simultaneously: coordinate with allies whose rewards align with its own, and out-think opponents whose rewards conflict. There is no single shared objective to optimise and no neat zero-sum structure to exploit, so the cooperative tricks and the competitive tricks both apply only partway. The relevant solution concepts are general-sum equilibria, which may not be unique and can be hard even to compute.

Because general-sum games lack the strong guarantees of the zero-sum case, this is the most realistic and the most fragile part of MARL. Equilibria can be multiple and inefficient, agents can fall into mutually destructive patterns, and learning may never settle. Benchmarks like the multi-agent particle environments and team-based video games are popular precisely because they force algorithms to handle teamwork and competition in the same breath.