the multivariate normal distribution and its covariance matrix
Push the bell curve into many dimensions and you get the multivariate normal, the joint distribution of a whole vector of variables (X_1, ..., X_n) that are each normal and linked linearly. It is the single most important multivariate distribution in statistics, machine learning, and finance, because sums and linear mixes of normals stay normal and the algebra stays clean.
It is pinned down by just two objects: a mean vector mu, listing the centre of each coordinate, and a covariance matrix Sigma, an n-by-n table whose diagonal entries are the variances Var(X_i) and whose off-diagonal entry in row i, column j is Cov(X_i, X_j). The matrix is symmetric (Cov is symmetric) and positive semidefinite (no combination of the variables can have negative variance). Sigma encodes the entire dependence structure: the ellipsoidal contours of the density are oriented and stretched by its eigenvectors and eigenvalues. The density itself is the n-dimensional bell, e to the power of minus one-half times (x - mu) transpose Sigma-inverse (x - mu).
The multivariate normal has remarkable closure properties: any subset of the variables is again multivariate normal (the marginals), any conditional given some of them is again multivariate normal with a shifted mean and reduced covariance, and ANY linear transformation A x + b of the vector is again multivariate normal. As in two dimensions, zero covariance between jointly normal coordinates means they are actually independent — a property special to the normal family, not true of variables in general.
A portfolio of three assets has returns modelled as multivariate normal with covariance matrix Sigma. The variance of an allocation w_1 X_1 + w_2 X_2 + w_3 X_3 is w transpose Sigma w, which uses every entry of Sigma. Choosing weights to minimise that quadratic form, balancing variances against covariances, is the heart of portfolio optimisation.
The covariance matrix turns the variance of any linear combination into the quadratic form w-transpose Sigma w.
A covariance matrix must be symmetric and positive semidefinite. Within a jointly normal vector, zero covariance does imply independence — but this is special to the normal, not a general rule.