Model Merging

Model Merging is a technique that combines multiple fine-tuned LLMs into a single model by interpolating or adding their weight spaces — creating models with combined capabilities without any additional training.

Why Model Merging?

• Fine-tuning a base model for task A and task B separately produces specialized models.
• Naively combining capabilities requires multi-task fine-tuning (expensive, data needed).
• Model merging: Average the weights directly — surprisingly effective in the weight space.

Merging Methods

Linear Merging (Model Soup):
- $\theta_{merged} = \frac{1}{n}\sum_i \theta_i$
- Simple average of fine-tuned model weights.
- Works well for models fine-tuned from the same base.

SLERP (Spherical Linear Interpolation):
- $SLERP(\theta_A, \theta_B, t) = \frac{\sin((1-t)\Omega)}{\sin\Omega}\theta_A + \frac{\sin(t\Omega)}{\sin\Omega}\theta_B$
- Interpolates along the geodesic on a sphere — better for large weight differences.

TIES-Merging:
- Trims redundant parameters, resolves sign conflicts, then merges.
- Handles conflicting updates between multiple models more robustly.

DARE:
- Randomly drops and rescales delta weights before merging.
- Reduces parameter interference.

Task Arithmetic:
- Compute "task vectors": $\tau_A = \theta_{fine-tuned} - \theta_{base}$
- Add/subtract task vectors: $\theta_{merged} = \theta_{base} + \lambda_A \tau_A + \lambda_B \tau_B$
- Can "unlearn" a capability by subtracting its task vector.

Practical Impact

• WizardMath, WizardCoder, OpenHermes and many top open-source models use model merging.
• No training cost: Merge two 70B models in minutes on CPU.
• Competitive with multi-task fine-tuning in many settings.

Model merging is a powerful, zero-cost technique for combining LLM capabilities — it democratizes capability combination for practitioners without large compute budgets.