BLAS & LAPACK
BLAS and LAPACK are the bedrock libraries of numerical linear algebra — the highly optimized code that essentially every scientific computing tool, from NumPy and MATLAB to deep-learning frameworks, calls underneath. BLAS supplies the low-level building blocks (vector and matrix operations); LAPACK builds the high-level solvers (linear systems, least squares, eigenvalues, SVD) on top of them. Together they are the reason fast, correct linear algebra is a library call rather than a research project.
BLAS is organized in three levels by how arithmetic relates to data movement. Level 1 is vector-vector (like dot products and saxpy), O(n) work on O(n) data. Level 2 is matrix-vector, O(n^2) on O(n^2) data. Level 3 is matrix-matrix, O(n^3) arithmetic on only O(n^2) data — and that imbalance is the secret. Level 3 reuses each loaded data element many times, so it can run near the machine's peak speed; the other levels are starved by memory bandwidth.
This is why modern algorithms are blocked: they are rewritten to spend as much time as possible in Level 3 matrix-matrix kernels, partitioning the matrix into tiles that fit in cache and operating tile-by-tile. LAPACK's factorizations are all blocked precisely so that the bulk of their O(n^3) flops land in well-tuned Level 3 BLAS, achieving a large fraction of peak performance instead of being throttled by memory traffic.
The practical wisdom is blunt: do not roll your own. Vendor-tuned BLAS implementations (OpenBLAS, Intel MKL, Apple Accelerate) and the algorithms in LAPACK embody decades of work on numerical stability, cache blocking, and parallelism that a hand-written triple loop cannot approach — often by one to two orders of magnitude in speed, and with far better accuracy guarantees. Reach for the library.
Level 3 BLAS does cubic arithmetic on quadratic data, so it reuses cached data heavily and runs near peak speed — the basis of blocked algorithms.
The BLAS is an interface (a specification), not a single program. Many implementations conform to it — reference BLAS, OpenBLAS, MKL, BLIS — and they can differ by orders of magnitude in speed while computing the same results, which is why which BLAS you link against can matter enormously.