model merging

Suppose you have separately fine-tuned the same base model into a coder, a translator, and a poet. Model merging combines them into a single model that does all three — with no extra training — by mathematically blending their weights.

Because the variants share an origin, their weights live in compatible space, so techniques range from simple weight averaging to task arithmetic, where you treat each fine-tune as a direction (a task vector) and add or subtract these directions to compose or remove skills. Methods like TIES and DARE resolve conflicts when edits disagree. Merging is nearly free compared with training and surprisingly effective, which is why it powers a long tail of community models.

It works best when the models share the same base and architecture; merging unrelated models, or piling on too many at once, tends to muddy their abilities rather than combine them.