adapter modules

Adapters are the original parameter-efficient trick: instead of rewriting a model, you slot tiny new layers into it and train only those. Picture installing small, swappable cartridges between the model's existing blocks — the big machine stays fixed, and the cartridges carry the new skill.

A typical adapter is a small bottleneck inserted inside each transformer layer: it projects the hidden state down to a low dimension, applies a nonlinearity, projects back up, and adds the result through a residual connection. Only these inserted layers train; everything else is frozen. Different tasks get different adapter sets, swapped in as needed. LoRA can be seen as a leaner cousin that injects its update into the existing weights rather than adding new layers.