svm image classifier

A support vector machine image classifier first converts each image into a fixed-length vector of hand-crafted features — for example a HOG descriptor, a bag-of-visual-words histogram, GIST, or color histograms — and then learns a boundary that separates one class of these vectors from another. The defining idea of an SVM is the maximum-margin boundary: among all the hyperplanes that could separate the positive examples from the negatives, it picks the one that leaves the widest possible empty street between the two classes. The training examples sitting right on the edges of that street are the support vectors; they alone determine the boundary, and the rest of the data could be removed without changing it.

Made precise, a linear SVM finds a weight vector w and offset b so that the signed distance of every training point to the plane w·x + b = 0 is as large as possible, subject to points being on the correct side. Real data are rarely perfectly separable, so the soft-margin formulation adds slack variables and a penalty C that trades off a wider margin against a few misclassifications; large C punishes errors hard (risking overfitting), small C tolerates more errors for a smoother boundary. The kernel trick then lets the same machinery draw curved boundaries: by replacing every dot product x·x′ with a kernel function k(x, x′) — such as the RBF (Gaussian) kernel or the histogram-intersection and chi-squared kernels favored in vision — the SVM behaves as if it had mapped the features into a vastly higher-dimensional space and found a flat boundary there, without ever computing that mapping explicitly.

For most of the 2000s, HOG features plus a linear SVM was the workhorse of object recognition and detection (it is the core of the Dalal–Triggs pedestrian detector and the deformable part model), because it is fast at test time (one dot product per window), needs relatively little training data, and generalizes well. SVMs are inherently binary, so multi-class recognition is built by training many one-versus-rest or one-versus-one classifiers. The hard limit is that an SVM only ever sees the features you hand it: if your HOG or bag-of-words representation throws away the distinction you need, no kernel can recover it. That dependence on hand-crafted features is exactly what convolutional networks removed by learning the features and the classifier jointly — though a linear SVM or softmax layer on top of frozen CNN or CLIP embeddings remains a strong, common baseline today.