train/validation/test split
/ TRAYN val-ih-DAY-shun TEST split /
Imagine a teacher who wants to know whether students truly learned, not just memorized. The trick is to teach from one set of problems but grade with problems nobody has seen. Splitting a dataset works the same way: we carve it into separate piles so we can tell real learning from rote memorizing.
There are usually three piles. The training set is what the model learns from — it adjusts itself to fit these examples. The validation set is a practice exam used while developing the model, to compare choices and tune knobs (hyperparameters) without peeking at the final exam. The test set is locked away and touched only once, at the very end, to give an honest estimate of how the model will do on data it has never met. A common rough split is 70 percent train, 15 percent validation, 15 percent test, though the exact proportions vary.
Why it matters: if you grade a model on the very examples it studied, it can score nearly perfectly by memorizing — a trap called overfitting — while being useless on anything new. The split is the basic discipline that keeps us honest about a model's real-world skill. The cardinal rule: the test set must stay sealed. Tune on it even once, and your reported score becomes optimistic fiction.
You have 10,000 labeled X-rays. You train on 7,000, watch how each model variant does on a held-out 1,500 while you tweak it, and only after picking your final model do you run it on the last 1,500 — reporting that score as your real accuracy.
Three sealed piles: learn on train, choose on validation, judge once on test.
The split must respect structure in the data. With time series, split by date (past to train, future to test); with grouped data (many photos of the same patient), keep a group entirely on one side — otherwise the test set leaks information and flatters the model.