soft prompt tuning

A normal prompt is a sequence of real tokens, each forced to be one of the vocabulary's discrete embeddings. Soft prompt tuning relaxes that constraint: it prepends a handful of free continuous vectors to the input embeddings and learns them by gradient descent, letting the prompt occupy points in embedding space that no actual word maps to. The model stays entirely frozen; the only thing that trains is this small bundle of virtual tokens at the very input.

This is the lightest member of the additive PEFT family. Unlike prefix tuning, the learned vectors are inserted only at the embedding layer rather than into every layer's key-value pairs, so it touches the fewest parameters of all — often a few thousand floats per task. A striking empirical finding is that as the base model grows, soft prompt tuning closes the gap with full fine-tuning, so at large scale a frozen backbone plus a learned prompt becomes genuinely competitive.

Practically this means a single served model can multiplex across tasks by swapping prompts, with near-zero storage per task and trivial deployment. The downsides are real: at small model sizes it underperforms, optimization can be finicky and initialization-sensitive, and the learned vectors are uninterpretable, so debugging a misbehaving soft prompt is closer to art than inspection.