sampling and resampling
/ SAM-pling and ree-SAM-pling /
Sampling is choosing which examples make it into your dataset — like a pollster picking a few thousand people to stand in for a whole country. Do it well and a small sample faithfully mirrors the larger population; do it carelessly (only phoning landlines, say) and you bake in a bias that no amount of clever modeling can later remove. Resampling is the related practice of drawing new selections from data you already have, to rebalance it or to measure uncertainty.
Good sampling aims to be representative. Random sampling gives everyone an equal chance; stratified sampling deliberately keeps important subgroups in proportion (so a survey of a country keeps its age mix). Resampling techniques reuse existing data for specific jobs: oversampling duplicates or generates more of a rare class, undersampling drops some of an over-common class (both used to fight class imbalance), and the bootstrap repeatedly draws random samples with replacement to estimate how much a result would wobble if you'd collected slightly different data.
Why it matters: every model silently assumes its training sample reflects the world it will be used in; when that assumption breaks, the model is confidently wrong about exactly the cases it never saw. Resampling to rebalance can help a model pay attention to a rare class, but it is a double-edged tool — undersampling throws away real data, and oversampling can lead a model to memorize the duplicated rare examples. None of it adds genuinely new information; it only redistributes what you already have.
A 1936 magazine predicted a US election from millions of replies — but it mailed only its own well-off subscribers and car owners, so its huge sample was badly skewed and called the result wrong. A pollster using a few thousand carefully balanced respondents got it right. Size lost to representativeness.
Millions of biased replies lost to a few thousand representative ones.
When rebalancing for class imbalance, resample only the training set — never the test set. Oversampling first and splitting afterward lets copies of the same example land on both sides, a textbook case of data leakage.