Numerical Linear Algebra: Direct Methods

the flop count

Before you run an algorithm you often want to know roughly how much arithmetic it will do, so you can predict whether it finishes in a second or a week. The flop count is exactly that estimate: the number of floating-point operations — additions, subtractions, multiplications, divisions on real numbers — that the algorithm performs. One flop is one such operation. For linear algebra it is the standard back-of-the-envelope measure of cost.

What matters most is how the count grows with the problem size n, written in big-O notation. Solving a dense n-by-n system by LU factorization takes about 2 n^3 / 3 flops to factor, plus about n^2 flops for each forward-and-back substitution to solve. Cholesky on a symmetric positive-definite matrix is half the factoring work, about n^3 / 3. A matrix-vector product A x is about 2 n^2 flops; a matrix-matrix product is about 2 n^3. The cubic terms dominate for large n: doubling n makes an LU factorization take roughly eight times as long, because 2^3 = 8.

Counting flops tells you the asymptotic story and lets you compare algorithms — it is why factor-once-solve-many (one n^3 factorization, then many n^2 solves) is so much better than re-eliminating for every right-hand side. But be honest about its limits: on modern hardware actual run time is often set by how data moves through cache and memory, not by raw flop count, so a memory-bound kernel can be far slower than its flops suggest. That gap is exactly why blocked, cache-aware BLAS and LAPACK routines exist.

Doubling a dense system from n = 1000 to n = 2000 raises the LU factorization cost from about 6.7e8 flops to about 5.3e9 flops — roughly 8x, since the cost scales like n^3.

The n^3 scaling is why size, not constants, decides feasibility for large dense problems.

Flop counts predict asymptotic cost, not wall-clock time: a memory-bound routine is limited by data movement and cache misses, so two algorithms with equal flops can differ by an order of magnitude in speed.

Also called
floating-point operation countoperation countFLOPs運算量