Computational & Experimental Physics

least-squares fitting

You have a cloud of data points that clearly trend along a straight line, and you want the single best line threading through them. But what does 'best' mean when no line passes through every point? Least squares answers with a specific, workable definition: the best curve is the one for which the data stray the least, in total, measured by the sum of the squared vertical gaps between points and curve.

Given data (x_i, y_i) and a model y = f(x; a, b, ...) with adjustable parameters, least squares chooses the parameters that minimize the sum of squared residuals S = sum of (y_i - f(x_i))^2. When each point carries its own uncertainty sigma_i, you instead minimize the chi-squared, sum of (y_i - f(x_i))^2 / sigma_i^2, which sensibly down-weights the noisier points. For a model linear in its parameters, such as a straight line y = a + b x, setting the derivatives dS/da = dS/db = 0 gives the normal equations, a small linear system with a closed-form solution; for a nonlinear model you minimize numerically by iteration.

This is the default method for pulling physical parameters out of data: a slope that is a spring constant, an intercept that is a rest length, an exponent that is a decay rate. It returns not only the best-fit values but their uncertainties and correlations, read from the curvature of chi-squared around its minimum. The honest caveat: least squares is optimal, coinciding with maximum likelihood, only when the errors are Gaussian; because it squares residuals it is notoriously sensitive to outliers, a single wild point having enormous leverage, and the basic form assumes the x-values are essentially error-free.

Fitting position-versus-time data for a falling object to y = y_0 + v_0 t + (1/2) g t^2 and reading g off the best-fit quadratic coefficient, you obtain g together with an uncertainty set by the scatter of the points about the fitted parabola.

A best-fit parabola turns noisy fall data into a value of g with an error bar.

Minimizing the sum of squared residuals coincides with maximum likelihood only for Gaussian errors; because it squares residuals a single outlier can drag the whole fit off, so always plot the data and the residuals rather than trusting the fitted numbers blindly.

Also called
least squarescurve fittingregression最小二乘法曲線擬合