least squares
Sometimes A*x = b has no exact solution: there are more equations than unknowns, and no single x can satisfy them all at once (think of fitting one straight line through many scattered points). Least squares answers a gentler question: which x comes closest?
Closest here means smallest total squared error. You measure how far A*x lands from b, square that gap, and pick the x that makes the squared miss as small as possible. Squaring keeps everything positive and punishes big misses harder than small ones.
Geometrically it is just a projection. The reachable outputs A*x form a subspace (the column space of A), and the best approximation is the shadow of b onto that subspace, with the leftover error standing perpendicular to it. This is the backbone of line-fitting and regression.
Minimize the squared error; the solution projects b onto A's column space.
The best x solves the normal equations A^T A x = A^T b, which simply demand that the leftover error be orthogonal to the column space.