the kernel of an inverse operator

Here is the cleanest way to say what a Green's function really is. A differential operator L turns a function into another function by differentiating. Solving L u = f means undoing that — applying the inverse operator L^(-1) to f to recover u. The beautiful fact is that this inverse is not another differential operator but an integral operator, and its 'matrix entries' form a function of two variables. That function is the Green's function: it is the integral kernel of L^(-1).

Think of it by analogy with linear algebra. If L were a matrix, solving L u = f would be u = L^(-1) f, a sum u_i = sum over j of (L^(-1))_ij f_j. Pass from finitely many indices i, j to continuous positions x, y, and the sum becomes an integral and the matrix entry (L^(-1))_ij becomes a kernel G(x, y): u(x) = integral of G(x, y) f(y) dy. Comparing, G(x, y) plays exactly the role of the inverse matrix entry, the (x, y) entry of L^(-1). This is why L G(., y) = delta(. - y) — that equation is the continuous version of L L^(-1) = I, since the delta is the continuous identity matrix (delta(x - y) is the kernel of the identity operator). Reciprocity G(x, y) = G(y, x) is then the statement that the inverse of a symmetric matrix is symmetric.

Seeing the Green's function this way unifies everything: solving the PDE is matrix inversion in disguise; the representation formula is just u = L^(-1) f written out; the spectral representation is diagonalizing L (eigenfunctions are eigenvectors) and inverting eigenvalue by eigenvalue; the resolvent is the kernel of (L - lambda)^(-1). The caveat, again, is the inverse must exist: if L has a nontrivial null space (a homogeneous solution satisfying the boundary condition, i.e. zero is an eigenvalue), it is not invertible and there is no ordinary Green's function — you need a modified Green's function that inverts L only on the complement of that null space, exactly as a singular matrix has only a pseudo-inverse.