trust-region line search

TRPO computes a search direction from a quadratic model of the KL constraint, then scales it to the largest step the model says fits inside the trust region. But the model is only an approximation, so the full step can quietly violate the real KL bound or even lower the true surrogate. The line search is the safety check that catches this: it tries the full step, and if the actual KL is too large or the surrogate did not improve, it shrinks the step and tries again.

Mechanically it is a backtracking loop. Halve the step length repeatedly, each time measuring the empirical KL and the surrogate at the proposed policy, and accept the first candidate that both satisfies the KL constraint and shows positive improvement. If no shrunken step works, TRPO simply skips the update for that iteration. This guardrail is what converts the approximate natural-gradient direction into a step you can actually trust, and it is a core reason TRPO behaves so stably.