soft actor-critic (SAC)

Soft actor-critic is an off-policy actor-critic for continuous control that does not merely chase reward; it also wants its policy to stay as random as it can while still acting well. Picture an agent that, all else equal, keeps its options open rather than committing early to one move. Treating that randomness as a bonus makes the agent explore broadly and stay robust when the world shifts slightly.

Concretely it optimizes a maximum-entropy objective, the expected return plus a temperature times the policy's entropy at each step. It learns soft Q-functions by a soft Bellman backup that folds the entropy term into the target, and a stochastic Gaussian policy trained with the reparameterization trick to track the Boltzmann distribution of the soft Q. Twin Q-networks with a clipped minimum curb overestimation, and the temperature is tuned automatically by holding the expected entropy to a target.

It is among the most sample-efficient and stable continuous-control methods because it reuses off-policy data from a replay buffer. The main caveat is sensitivity to reward scaling, since that implicitly sets how strongly entropy is weighted, and to the chosen entropy target.