the central difference
To read the slope of a hill at the spot where you stand, you could look only forward, or only backward — but the fairest estimate looks BOTH ways and averages the view. The central difference does exactly that: it measures slope by straddling the point of interest with one sample ahead and one behind, and this small symmetry buys a big jump in accuracy almost for free.
The formula for the first derivative is (f(x + h) - f(x - h)) / (2h). Compare it with the lopsided forward difference (f(x + h) - f(x)) / h. The Taylor expansion shows why centering wins: in the forward difference the leading error is (h/2) f''(x), of order h, but in the central difference the symmetric f''(x) terms CANCEL, leaving a leading error of (h^2/6) f'''(x), of order h^2. Halving h cuts the forward error in half but the central error by a factor of four. The same symmetric trick gives the standard second-derivative stencil (f(x + h) - 2 f(x) + f(x - h)) / h^2, also O(h^2).
Centering is the default first choice whenever you have function values on both sides of the point — it gives second-order accuracy at the cost of just two evaluations. The honest limits: you cannot center at the very edge of a data set (there is no point past the boundary), so one-sided formulas take over there; and like all finite differences it still faces the step-size trade-off, where shrinking h too far surrenders accuracy to cancellation. For a much higher order at the same cost, feed the central difference into Richardson extrapolation.
For f(x) = sin(x) at x = 1 (true f'(1) = cos 1 = 0.5403). Central difference with h = 0.01 gives (sin 1.01 - sin 0.99)/0.02 = 0.54030, error about 9e-6. Forward difference with the same h gives 0.5361, error about 4e-3 — a thousand times worse, purely from being one-sided.
Symmetry cancels the order-h error, so accuracy jumps to order h^2.
Higher order is not the same as exact: the central difference is still wrong by O(h^2), and pushing h too small lets round-off (from subtracting two nearly equal values) overwhelm that gain. Its even-order error expansion is precisely what makes Richardson extrapolation so effective on it.