a matrix splitting
A matrix splitting is the trick that turns the unsolvable equation A x = b into a sequence of easy ones. The idea: write A as a difference A = M - N, where M is some part of A that you can solve with cheaply (for instance just its diagonal, or its lower-triangular part), and N is whatever is left over. The hard matrix A has been split into an easy piece M and a correction N.
Substitute the split into A x = b to get M x = N x + b. The right-hand side still contains the unknown x, so we cannot solve it directly — but we can iterate: plug in the current guess on the right and solve the easy system on the left for the next guess, x_{k+1} = M^{-1}(N x_k + b). Equivalently x_{k+1} = G x_k + c with iteration matrix G = M^{-1} N and c = M^{-1} b. A good splitting makes two competing demands: M must be cheap to invert (so each step is fast), yet M must also resemble A closely enough that N is small and G = M^{-1} N has spectral radius well below 1 (so the iteration converges quickly). These two goals pull in opposite directions, and the art is in the balance.
Different choices of M name the classical methods: M = the diagonal of A gives Jacobi; M = the lower-triangular part (diagonal plus everything below) gives Gauss-Seidel; a weighted blend gives SOR. The very same idea — replace A by a nearby, easy-to-invert M — is exactly what a preconditioner does for Krylov methods. So the splitting concept quietly underlies almost all of iterative linear algebra: you never solve the hard problem, you repeatedly solve a cheap stand-in and let the iterations chip away the difference.
Write A = D - L - U (its diagonal D, strictly-lower part L, strictly-upper part U). Jacobi uses M = D, N = L + U. Gauss-Seidel uses M = D - L, N = U. SOR uses M = (1/omega) D - L with a relaxation parameter omega.
One matrix A, three classical splittings — each trading ease of M against fidelity to A.
A splitting is called convergent if the spectral radius of G = M^{-1} N is below 1; this is not automatic. For Jacobi and Gauss-Seidel, convergence is guaranteed for important classes (e.g. strictly diagonally dominant or symmetric positive-definite A), but can fail outside them.