meta-learning
/ MET-uh-LER-ning /
An experienced cook who has worked through hundreds of recipes can glance at a brand-new dish and improvise it well, because they have learned how cooking works, not just one recipe. Meta-learning — "learning to learn" — tries to give machines that knack: rather than mastering a single task, the system trains across many tasks so it gets good at picking up new tasks quickly, from very little data.
The setup is a layer above ordinary training. You feed the system not one dataset but a whole distribution of small tasks. On the "inner" level it practices solving each little task; on the "outer" level it adjusts itself so that this practice becomes faster and easier next time. One famous approach learns a starting point for the model's parameters that is perfectly poised to adapt — a few training steps on any new task and it is already competent. The product of meta-learning isn't a finished model for one job; it is a model that is good at becoming good at new jobs.
This is the main engine behind few-shot learning, where a system must learn a new category from a handful of examples. The candid limits: meta-learning only transfers to new tasks that resemble the training tasks — face a truly unfamiliar kind of problem and the head start vanishes. It is also finicky to train (the nested loops are unstable and compute-hungry), and for many practical problems, simply fine-tuning a large pre-trained model turns out to work just as well with less fuss.
Train a system on thousands of tiny tasks, each "learn to tell two new animal species apart from five photos each." After all that practice the system has learned how to adapt. Now show it five photos of a tapir and five of an okapi — animals it never saw in training — and it separates them correctly almost immediately.
The output is not one trained model but the ability to adapt fast.
Meta-learning only buys a head start on tasks that look like its training tasks; a genuinely novel problem resets it to square one. "Learning to learn" is not a magic universal learner.