Regularization (L1/L2)

Regularization is a technique to prevent overfitting by adding a penalty for large coefficients to the model's fitting objective. Instead of only minimizing prediction error, the model also pays a price for complexity, which shrinks the coefficients toward zero and keeps it from contorting to chase noise.

The two common flavors differ in flavor of penalty: L2 (ridge) shrinks all coefficients smoothly toward but not exactly to zero, taming multicollinearity; L1 (lasso) can push some coefficients all the way to zero, effectively selecting a smaller set of features. A dial called lambda controls how hard you penalize — too little and you still overfit, too much and you underfit — and it is usually tuned with cross-validation.