clipped surrogate objective

This is the exact loss that makes PPO behave. Start with the probability ratio between the new and old policy for each action; if the advantage was positive you want to raise that ratio, if negative you want to lower it. The clipped objective lets you do this only up to a band around one, say from 0.8 to 1.2. Push the ratio outside the band in the helpful direction and the gradient flattens to zero — the loss no longer rewards going further, so the update self-limits.

The subtlety is taking the minimum of the unclipped and clipped terms. This makes the bound pessimistic: it caps the gain when the policy moves in the rewarded direction, but it does not clip away the penalty when a bad action's probability has already grown too much, so the optimizer is still pulled back. The result approximates TRPO's KL trust region with a first-order, per-sample loss that needs no constraint solver — cheap to compute and easy to drop into any policy-gradient codebase.