singular value

Eigenvalues can be negative, complex, or refuse to exist over the reals; they only behave nicely for square matrices. Singular values are the well-mannered cousins that exist for every matrix, of any shape. Geometrically they are the stretch factors of the map A: feed A the unit sphere and it comes out as an ellipsoid, and the lengths of that ellipsoid's semi-axes are exactly the singular values.

Precisely: for any m-by-n matrix A, form the symmetric positive-semidefinite matrix A^T A. Its eigenvalues are all real and nonnegative. The singular values sigma_1 >= sigma_2 >= ... >= 0 are the nonnegative square roots of those eigenvalues, sigma_i = sqrt(lambda_i(A^T A)). The number of nonzero singular values equals the rank of A. By convention we list them in decreasing order.

Because A^T A is symmetric PSD, its eigenvalues never go negative, so the square root is always real — that is why singular values are real and nonnegative no matter how wild A is. The largest, sigma_1, is the biggest amount A can stretch any unit vector (the operator 2-norm); the smallest nonzero one tells you how close A comes to being singular in the directions it actually acts on.

You get the same numbers from A A^T, which is m-by-m: A^T A and A A^T share the same nonzero eigenvalues, just padded with extra zeros to fit the larger dimension. So the singular values are an intrinsic property of the map, blind to whether you look at it through its inputs or its outputs.