compact operator
Of all the operators on an infinite-dimensional space, compact operators are the ones that behave most like ordinary matrices. A finite-rank matrix squashes everything into a finite-dimensional image, where bounded sets are nicely compact. A compact operator nearly does the same: it takes any bounded set and crushes it into something whose closure is compact. It is the infinite-dimensional way of saying almost finite-rank.
Precisely: a bounded operator T on a Banach space is compact if the image of the unit ball has compact closure — equivalently, every bounded sequence (x_n) has a subsequence with (T x_n) convergent. On a Hilbert space, the compact operators are exactly the norm-limits of finite-rank operators, so a compact operator can be approximated as closely as you like by genuine matrices acting on finite-dimensional pieces.
Why they are so important: compactness is what rescues the eigenvalue theory you knew from finite dimensions. For a compact self-adjoint operator you get a countable set of real eigenvalues that can accumulate only at 0, with an orthonormal eigenbasis — the spectral theorem in its cleanest infinite-dimensional form. Integral operators with continuous kernels are compact, which is why compact operators are central to integral equations and the Fredholm alternative.
A sharp caveat: the identity operator on an infinite-dimensional space is never compact, because the unit ball there is not compact (its closure is not compact). So general bounded operators can be radically un-matrix-like — compactness is a strong, special hypothesis, and most of the well-behaved spectral results require it.
Integral operators with continuous kernels: the canonical family of compact operators.
Rule of thumb: compact operator ~ infinite matrix whose entries decay fast enough that it is essentially finite-rank. Smoothing and integral operators tend to be compact; differentiation (which roughens) tends to be unbounded.