clipped double-Q (continuous)

Value-based RL has a chronic disease called overestimation: because the learning target uses a max (or an actor that maximises) over noisy value estimates, errors are systematically biased upward, and the actor learns to exploit whichever estimate happens to be too rosy. Clipped double-Q is the blunt, effective cure used by TD3 and SAC. Train two critics with separate parameters, then whenever you need a value for the target, take the smaller of the two.

The logic is statistical. If two independent estimators each have random error, the larger one is biased high, so the max amplifies overestimation; the minimum, by contrast, leans pessimistic and cancels much of the upward bias. Writing the target with min(Q₁, Q₂) instead of either critic alone keeps the agent from chasing phantom value. It is deliberately a little too cautious — slightly underestimating is a price worth paying to stop the runaway over-optimism that wrecks DDPG.

This is the continuous-action sibling of double DQN, adapted from discrete Q-learning to the actor-critic setting where you cannot enumerate actions. It is one of the highest-leverage stabilisers in modern off-policy control, and its presence in both TD3 and SAC is a big part of why both are so much steadier than plain DDPG.