Editing Models via Task Vectors

Editing Models via Task Vectors is a model modification framework that decomposes fine-tuned model knowledge into portable, composable vectors — enabling transfer, removal, and combination of learned behaviors by manipulating these vectors in weight space.

Key Operations

• Extraction: $\tau = \theta_{fine} - \theta_{pre}$ (extract what fine-tuning learned).
• Transfer: Apply $\tau$ from model $A$ to model $B$: $\theta_B' = \theta_B + \tau_A$.
• Forgetting: $\theta' = \theta_{fine} - \lambda \tau$ (partially undo fine-tuning for selective forgetting).
• Analogy: If $\tau_{EN\rightarrow FR}$ maps English→French, apply it to other models for similar translation ability.

Why It Matters

• Modular ML: Neural network capabilities become modular, composable units.
• Efficient Transfer: Transfer specific capabilities without full fine-tuning.
• Debiasing: Remove biased behavior by subtracting the corresponding task vector.

Editing via Task Vectors is modular surgery for neural networks — extracting, transplanting, and removing capabilities as portable weight-space operations.