Regression & Predictive Modeling

Multicollinearity

Multicollinearity happens when two or more predictors in a regression are strongly correlated with each other, so they carry overlapping information. The model then struggles to tell apart each variable's separate contribution, like trying to credit two singers who always sing the exact same notes.

The symptoms are unstable, wildly swinging coefficients with large standard errors (sometimes even flipped signs) — yet the model's overall predictions can still be fine. It hurts interpretation, not necessarily prediction. Analysts detect it with the variance inflation factor (VIF) and respond by dropping or combining redundant variables, or using regularization.

A common rule of thumb flags a variance inflation factor (VIF) above about 5 to 10 as concerning.

Also called
collinearity