small language models

Not every task needs the biggest, most expensive model. Small language models — roughly a few hundred million to a few billion parameters — are deliberately compact models that run cheaply and fast, often on a single GPU or even a phone, while still handling many real jobs well. The recent surprise is how capable they have become: careful training on high-quality data lets a small model match a much larger one from a couple of years ago on focused tasks.

Small models reach useful quality through better data and training rather than sheer size: heavy data curation and deduplication, training on more tokens than scaling laws once recommended, distilling knowledge from a larger teacher model, and fine-tuning narrowly on the target task. For a specific, well-defined job — classification, extraction, routing, summarising in a known format — a small fine-tuned model can equal or beat a giant general one, at a fraction of the cost and latency.

Small models trade breadth for efficiency. They have weaker general reasoning, less world knowledge, and a smaller margin for messy open-ended requests than frontier models, so they suit narrow, high-volume tasks rather than do-anything assistants. Their rise reshapes application design: instead of routing everything to one expensive model, teams increasingly use a fleet of small specialists and reserve the flagship for the genuinely hard cases.