Newton's method
/ NOO-tunz /
Suppose you are standing on a curve at some point and want to reach the spot where it crosses zero, but you cannot see far ahead. A reasonable bet: pretend the curve is a straight line — its tangent — right where you stand, slide down that line to where IT hits zero, and step there. From the new point, do it again. Newton's method is exactly this 'follow the tangent to its x-intercept' rule, and near a simple root it homes in with breathtaking speed.
Here is the derivation in one line. At x_n the tangent line is y = f(x_n) + f'(x_n)(x - x_n). Setting y = 0 and solving for x gives the next guess: x_{n+1} = x_n - f(x_n) / f'(x_n). You evaluate the function and its derivative, take a step of size -f(x_n)/f'(x_n), and repeat until f is tiny. Geometrically each iterate is where the tangent at the current point crosses the axis; a steep curve (large |f'|) takes a small, careful step, a flat curve takes a big leap. Near a SIMPLE root (where f = 0 but f' is nonzero) the error obeys e_{n+1} is approximately C * e_n^2 — the number of correct digits roughly DOUBLES every step, so 3 correct digits become 6, then 12. This is quadratic convergence.
Newton is the workhorse behind countless solvers, from computing square roots and reciprocals in hardware to training neural networks and solving giant nonlinear systems (with the Jacobian replacing f'). But its speed is a LOCAL gift, not a guarantee: it needs a good starting guess, a nonzero derivative, and a well-behaved function. Far from a root it can overshoot wildly, get trapped in a cycle, or wander off when f' is near zero; at a multiple root it slows to merely linear. That is why robust code brackets the root and falls back to bisection when Newton misbehaves.
To find sqrt(2), solve f(x) = x^2 - 2 = 0 with f'(x) = 2x, so x_{n+1} = x_n - (x_n^2 - 2)/(2 x_n) = (x_n + 2/x_n)/2 — the ancient 'average your guess with 2/guess' rule. From x_0 = 1: 1.5, then 1.41667, then 1.414216, then 1.4142135623747 — correct digits roughly double each step, reaching machine precision in four iterations.
x_{n+1} = x_n - f(x_n)/f'(x_n): follow the tangent to the axis; digits double near a simple root.
Quadratic convergence is a LOCAL promise near a simple root with a good start — it is not unconditional. A bad guess, a near-zero f', or a multiple root can make Newton diverge, cycle, or crawl, so production solvers always pair it with a safeguard.