Model Merging

Model Merging and Weight Averaging — Combining Neural Networks Without Retraining

Model merging combines the parameters of multiple trained neural networks into a single model without additional training, offering a remarkably efficient approach to improving performance, combining capabilities, and creating multi-task models. This family of techniques has gained significant attention as a cost-effective alternative to ensemble methods and multi-task fine-tuning.

— Weight Averaging Fundamentals —

The simplest merging approaches directly average model parameters under specific conditions that ensure effectiveness:

• Uniform averaging computes the element-wise mean of corresponding parameters across multiple models
• Linear mode connectivity is the property that interpolated weights between two models maintain low loss along the path
• Shared initialization from a common pretrained checkpoint is typically required for successful weight averaging
• Stochastic Weight Averaging (SWA) averages checkpoints from a single training run to find flatter, more generalizable minima
• Exponential Moving Average (EMA) maintains a running average of model weights during training for improved final performance

— Advanced Merging Strategies —

Sophisticated merging methods go beyond simple averaging to handle diverse model combinations more effectively:

• Model soups average multiple fine-tuned variants of the same base model, selecting ingredients that improve held-out performance
• Task arithmetic computes task vectors as the difference between fine-tuned and pretrained weights, then adds or subtracts them
• TIES merging resolves sign conflicts and trims small-magnitude parameters before averaging for cleaner task combination
• DARE randomly drops delta parameters and rescales the remainder before merging to reduce interference between tasks
• Fisher merging weights each model's parameters by their Fisher information to prioritize task-critical parameters

— Applications and Use Cases —

Model merging enables practical workflows that would be expensive or impractical with traditional training approaches:

• Multi-task combination merges separately fine-tuned single-task models into one model handling all tasks simultaneously
• Domain adaptation blends domain-specific fine-tuned models to create models effective across multiple domains
• Federated learning averages locally trained models from distributed clients to produce a global model without sharing data
• Reward model combination merges reward models trained on different preference aspects for balanced alignment
• Continual learning merges models trained on sequential tasks to mitigate catastrophic forgetting without replay

— Theoretical Understanding and Limitations —

Understanding when and why merging works guides practitioners in applying these techniques effectively:

• Loss basin geometry explains that models fine-tuned from the same initialization often reside in the same loss basin
• Permutation symmetry means that networks with shuffled neuron orderings are functionally equivalent but cannot be naively averaged
• Git Re-Basin aligns neuron permutations between independently trained models to enable meaningful weight averaging
• Interference patterns arise when merged task vectors conflict, degrading performance on one or more constituent tasks
• Scaling behavior shows that merging effectiveness can change with model size, with larger models often merging more successfully

Model merging has emerged as a surprisingly powerful technique that challenges the assumption that combining model capabilities requires joint training, offering a practical and computationally efficient pathway to building versatile multi-capability models from independently trained specialists.