the symmetric eigenproblem
When the matrix is symmetric (equal to its own transpose, A = A^T) — and many of the most important matrices in science are, like covariance matrices, stiffness matrices, and graph Laplacians — the eigenvalue problem becomes about as well-behaved as numerical mathematics ever gets. Everything that can go wrong with general matrices (complex eigenvalues, sensitive eigenvalues, non-orthogonal or even missing eigenvectors) simply does not happen. The symmetric case is the friendly country where the theory is clean and the algorithms are fast and trustworthy.
The spectral theorem guarantees three lovely facts. The eigenvalues are all REAL — no complex pairs to chase. The eigenvectors can be chosen ORTHONORMAL, so the matrix factors as A = Q Lambda Q^T with Q orthogonal and Lambda diagonal; this is the eigendecomposition and it is also a rotation to a basis where A acts by simple scaling along perpendicular axes. And — the punchline for numerical work — the eigenvalues are PERFECTLY conditioned: a perturbation of size epsilon to A moves each eigenvalue by at most epsilon (Weyl's inequality). There is no eigenvalue ill-conditioning to fear, unlike the nonsymmetric case. The Rayleigh quotient v^T A v / (v^T v) ranges exactly between the smallest and largest eigenvalues and gives a quadratically accurate eigenvalue estimate from a linearly accurate eigenvector.
Because of this structure, the algorithms specialize and speed up. Reduce to TRIDIAGONAL form (not just Hessenberg) with two-sided Householder, then run symmetric QR with the Wilkinson shift (cubic convergence), or divide-and-conquer, or the MRRR algorithm, to get all eigenpairs in O(n^3) but with smaller constants and guaranteed orthogonal eigenvectors. For a few extreme eigenvalues of a huge sparse symmetric matrix, the Lanczos algorithm is the tool. And the SVD of any matrix A is computed by quietly leaning on the symmetric eigenproblem of A^T A. So the symmetric eigenproblem is not just a special case — it is the foundation that the general and the singular-value machinery are built on.
A covariance matrix is symmetric and positive semidefinite, so all its eigenvalues are real and non-negative and its eigenvectors are orthogonal — these eigenvectors are exactly the principal component directions, and the eigenvalues are the variances along them. Diagonalizing A = Q Lambda Q^T rotates your data to axes along which it varies independently, with Lambda telling you how much variance lives on each axis. That is PCA, and it works cleanly precisely because the matrix is symmetric.
Symmetric matrices have real eigenvalues, orthonormal eigenvectors, and perfectly conditioned eigenvalues — the easy, beautiful case.
Well-conditioned eigenVALUES do not guarantee well-conditioned eigenVECTORS: when two eigenvalues are nearly equal, the individual eigenvectors are sensitive (they can rotate freely within the near-degenerate subspace), even though each eigenvalue itself is stable. The invariant SUBSPACE is well-determined; the individual vectors inside it may not be.