Applications, Environments & Sim-to-Real

AlphaZero (self-play RL)

AlphaZero learned chess, shogi, and Go to superhuman strength starting from nothing but the rules — no human games at all. It improved by playing against itself, each new version becoming a slightly stronger sparring partner, so the system bootstrapped from random play all the way to world-class.

A single neural network outputs both a move policy and a position value. During play, Monte Carlo tree search uses the network to guide its lookahead and produces moves better than the raw network alone; those search results then become the training targets, so the network distills the search while the search keeps getting sharper — a virtuous loop. Self-play supplies an endless stream of fresh data at exactly the right difficulty.

AlphaZero generalized the earlier AlphaGo (which had used human games and handcrafted features) into one clean algorithm; MuZero later removed even the need to know the rules by learning a model of the game itself.

Also called
AlphaZeroself-play with search