Monte Carlo tree search (in RL)

Monte Carlo tree search is decision-time planning done with discipline. Faced with a position, it grows a tree of possible futures, but instead of expanding every branch — hopeless in games like Go — it spends its limited simulations where they matter most, repeatedly diving down the most promising lines while still occasionally checking neglected ones. After many simulations it recommends the action that looks best.

Each simulation has four phases: selection, walking down the tree by a rule that balances exploitation of high-value moves against exploration of under-tried ones (commonly a UCT/PUCT score); expansion, adding a new node at the frontier; evaluation, estimating that node's value, classically by a random rollout and in modern systems by a learned value network; and backup, propagating the result up the visited path so the statistics improve. The action most visited after the budget runs out is chosen.

MCTS is the search engine behind AlphaGo, AlphaZero, and MuZero, where a neural network guides which branches to explore and how to value leaves, fusing learning with lookahead. It shines when a model (given or learned) lets you simulate, and its anytime nature means more thinking simply yields better play.