Model Merging
Model Merging is the technique of combining the weights of multiple fine-tuned models into a single model without additional training — by interpolating, averaging, or selectively combining parameters from models that share the same base architecture but were fine-tuned on different tasks or data, enabling multi-task capability, improved robustness, or novel capability combinations at zero additional training cost.
Why Model Merging Works
Fine-tuned models from the same pretrained base occupy a connected low-loss basin in the loss landscape. The key insight: linear interpolation between fine-tuned checkpoints often produces models that perform well on ALL constituent tasks, not just the average.
Base model → Fine-tune on Task A → Model_A (weights θ_A)
→ Fine-tune on Task B → Model_B (weights θ_B)
→ Fine-tune on Task C → Model_C (weights θ_C)
Merged: θ_merged = f(θ_A, θ_B, θ_C) → Good at A, B, AND CMerging Methods
| Method | Formula | Key Idea |
|---|---|---|
| Linear/SLERP | θ = α·θ_A + (1-α)·θ_B | Simple interpolation |
| Model Soup | θ = (1/N)·Σθ_i | Average multiple fine-tunes of same task |
| Task Arithmetic | θ = θ_base + Σλ_i·τ_i where τ_i = θ_i - θ_base | Add task vectors to base |
| TIES-Merging | Trim + Elect Sign + Merge | Resolve sign conflicts in task vectors |
| DARE | Random drop + rescale task vectors | Sparsify before merging |
| Frankenmerge | Layer-wise selection from different models | Pick best layers from each |
Task Arithmetic
The most influential framework defines a task vector τ = θ_fine-tuned - θ_base:
# Task vectors capture what fine-tuning learned
task_vector_A = model_A.state_dict() - base.state_dict()
task_vector_B = model_B.state_dict() - base.state_dict()
# Addition: combine capabilities
merged = base + 0.7 * task_vector_A + 0.5 * task_vector_B
# Negation: remove capabilities (e.g., remove toxicity)
detoxified = base - 0.5 * task_vector_toxicTIES-Merging (Trim, Elect Sign, & Merge)
Addresses interference when naively adding task vectors:
1. Trim: Zero out low-magnitude values (keep top-k% of each task vector)
2. Elect Sign: For each parameter, take majority vote on sign across task vectors
3. Disjoint Merge: Average only values that agree with the elected sign
DARE (Drop And REscale)
Randomly drops 90-99% of task vector values and rescales the rest — extremely sparse task vectors merge with less interference. Works especially well for LLMs where fine-tuning changes are highly redundant.
Practical Applications
- Open-source LLM community: Merging specialized LoRA adapters (code + chat + reasoning) is widespread on Hugging Face, creating models that outperform individual fine-tunes.
- Model soups: Averaging multiple training runs reduces variance and improves OOD robustness (Wortsman et al., 2022).
- Evolutionary merging: CMA-ES or genetic algorithms to search optimal merging coefficients per layer (Sakana AI's evolutionary model merge).
Model merging has become a fundamental technique in the open-source AI ecosystem — enabling the creation of capable multi-task models through simple weight arithmetic, democratizing model customization without the computational cost of multi-task training or the data requirements of comprehensive fine-tuning.