Functions & Transformations of Random Variables

the probability integral transform

Here is a small miracle that ties every continuous distribution back to one common reference. Take any continuous random variable X, feed its own value into its own cumulative distribution function F, and the result U = F(X) is uniform on (0, 1) — no matter what X started as. Heavy-tailed, skewed, bell-shaped: run it through its own cdf and out comes a flat, featureless uniform.

Why does this happen? The cdf F maps each value of X to the fraction of probability sitting to its left. The bottom 10 percent of X's outcomes get mapped to values below 0.10, the bottom half to values below 0.50, and so on — that is exactly what it means for the output to be uniform: equal-sized probability slabs map to equal-length intervals. To check it, compute P(U at most u) = P(F(X) at most u) = P(X at most F^{-1}(u)) = F(F^{-1}(u)) = u, the cdf of a uniform. The cdf is precisely the function that ranks outcomes by their cumulative probability, so it always flattens its own variable.

This is the quiet engine behind two big practical things. Reversed, it lets you generate any distribution from a uniform random number (the inverse-transform method). Used forward, it is the foundation of goodness-of-fit testing and of copulas: transform your data through the fitted cdf, and if the fit is good, the transformed values should look uniform. The caveat: the result is exactly uniform only when X is continuous; for a discrete X with jumps, F(X) lands on a discrete set and is not uniform.

Let X be exponential with rate 1, so F(x) = 1 - e^(-x). Then U = F(X) = 1 - e^(-X) is uniform on (0, 1). Check: P(U at most u) = P(1 - e^(-X) at most u) = P(X at most -ln(1 - u)) = 1 - e^(ln(1-u)) = u. The skewed exponential has been flattened to a uniform by its own cdf.

Any continuous variable run through its own cdf comes out uniform on (0, 1).

It holds exactly only for continuous X. For a discrete X, F(X) is not uniform because the cdf jumps and skips values. The result U is uniform; do not confuse it with X being uniform.

Also called
PITuniversality of the uniform機率積分轉換PIT