gradient and level sets

On a topographic map, the contour lines connect points of equal elevation. A level set is the higher-dimensional version: the set of all points where a function takes one fixed value, f = c. The clean geometric fact is that the gradient at any point is perpendicular (normal) to the level set passing through that point — the steepest-uphill arrow always crosses the contours at right angles.

Here is why, with no mystery. If you move along a level set, the value of f does not change, so the rate of change of f in any direction tangent to the level set is zero. But that rate is grad f dot (tangent direction), so grad f is orthogonal to every direction lying in the level set — which is exactly what 'normal to the level set' means. In two dimensions grad f is perpendicular to the level curve and points toward higher values; in three dimensions grad f is perpendicular to the level surface and is, in fact, the normal vector you use to write the tangent plane to that surface.

This duality is enormously useful. It gives the tangent plane to any implicitly defined surface F(x, y, z) = c for free: the plane through the point with normal grad F. It is the geometric heart of Lagrange multipliers, where a constrained optimum occurs where the gradient of the objective is parallel to the gradient of the constraint (their level sets are tangent). And it underlies physical pictures everywhere — electric field lines crossing equipotentials at right angles, heat flowing perpendicular to isotherms, fluid pressure gradients normal to surfaces of constant pressure.