linear probing

Linear probing is a protocol for evaluating how good a frozen feature representation is. You take a pretrained network, freeze all of its weights, attach a single new linear classifier on top of its penultimate features, and train only that linear layer on the target dataset. Because the only trainable component is one matrix of weights and biases, the accuracy you get is a direct measure of how linearly separable the classes already are in the representation the backbone produces.

It is the workhorse evaluation in self-supervised and representation learning, where the whole point is to learn features without labels and then ask how useful those features are. A high linear-probe accuracy means the representation has organised the relevant semantic structure into directions a linear boundary can exploit. Because the backbone is frozen, linear probing is fast, cheap, and reproducible, and it isolates representation quality from the confounding effect of also tuning the features.

Linear probing sits at one end of a spectrum whose other end is full fine-tuning, which updates all weights. Probing usually scores lower in absolute accuracy but is a fairer, lower-variance comparison of representations, is far less prone to overfitting on small datasets, and does not let a strong optimiser paper over a weak backbone. The gap between linear-probe and fine-tuning accuracy is itself informative about how much task-specific adaptation the features still need.