variance reduction (importance sampling, antithetic variates)
Plain Monte Carlo has one stubborn weakness: its error shrinks only like 1 over sqrt(n), so brute force is expensive. Variance reduction is the art of getting the same accuracy from far fewer samples by being clever about HOW you sample, rather than just sampling more. Since the Monte Carlo error is sigma / sqrt(n), shrinking the effective sigma is worth just as much as growing n — and often far cheaper.
Two workhorse techniques. Importance sampling rescues estimates dominated by rare but important events: instead of sampling from the original distribution p, you sample from a cleverly chosen distribution q that visits the important region more often, then correct the bias by weighting each sample by the likelihood ratio p(x)/q(x). The estimator E_p[g(X)] becomes the q-average of g(X) times p(X)/q(X), which is still unbiased but can have dramatically lower variance when q concentrates effort where g times p is large. Antithetic variates exploit negative correlation: for every sample U you also use its mirror 1 - U (or -X for a symmetric variable), so that when one draw overshoots the truth its partner tends to undershoot. Averaging the negatively correlated pair has variance Var = (1/2)(Var(A) + Cov(A, B)), and the negative covariance shaves it below the independent case for free.
Beyond these, control variates subtract a correlated quantity whose mean you know, and quasi-Monte Carlo replaces random points with cleverly spread-out low-discrepancy sequences that can achieve error close to 1/n instead of 1 over sqrt(n) in nice problems. The honest caveats matter: a poorly chosen importance distribution q can make variance EXPLODE (especially if q has lighter tails than p, so the weights p/q blow up), and antithetic variates only help when g is monotone enough that the mirror really is negatively correlated. Variance reduction is leverage, not magic — applied carelessly it can quietly make your estimate worse, not better.
To estimate P(Z > 4) for a standard normal — a tiny probability around 0.00003 — plain sampling almost never produces a Z above 4, so estimates are pure noise. Importance sampling instead draws from a normal shifted to center near 4, where the rare event is common, then reweights by the likelihood ratio; now most samples are informative and the estimate is accurate with a few thousand draws.
Shrinking variance is worth as much as growing n — but a bad scheme can make variance explode.
Importance sampling can backfire: if the sampling distribution q has lighter tails than the target, the weights p/q blow up and the variance can become infinite — sample where the action is, but keep the tails heavy enough.