holdout vs cross-validation

You can't fairly grade a model on the same examples it studied — that's like testing students on the exact questions they were given the answers to. So you hold some data back. The holdout method does this once: split your data, say 80 percent to train on and 20 percent set aside, then score the model only on that untouched 20 percent. Simple, fast, and the right move when you have plenty of data.

Cross-validation is the more thorough cousin, for when data is scarce and a single split might just get lucky or unlucky. In k-fold cross-validation you slice the data into k equal parts — say 5. You train on 4 parts and test on the 5th, then rotate so each part takes its turn as the test set. Every example gets to be a test case exactly once, and you average the five scores. The payoff is a more stable, trustworthy estimate, plus a sense of how much the score wobbles depending on which data you held out.

The trade-off is plain. Holdout is cheap but its verdict can hinge on the luck of one split; cross-validation is far more reliable but costs k times the computation, since you train the model k times over. For very large datasets or expensive models, a single big holdout is usually fine. The deeper rule beneath both: whatever data you use to make decisions — picking models, tuning settings — is contaminated, so a truly final, untouched test set should be reserved for the very end.