Model Editing

Model editing directly updates specific weights to fix factual errors or modify behaviors without full retraining. Motivation: Models contain factual errors, knowledge becomes outdated, want to fix specific behaviors. Full retraining expensive and may lose capabilities. Approaches: Locate-then-edit: Find neurons/parameters responsible for fact, update those weights. Hypernetwork: Train network to predict weight updates for edits. ROME/MEMIT: Rank-one model editing in MLP layers where factual associations stored. Edit types: Factual updates ("The president of X is now Y"), behavior changes, bias corrections. Evaluation criteria: Efficacy: Does edit work? Generalization: Does it work for rephrasings? Specificity: Are unrelated facts preserved? Challenges: Edits may break model coherence, ripple effects on related knowledge, scalability to many edits. Tools: EasyEdit, PMET, custom implementations. Alternatives: RAG with updated knowledge base (avoids editing model), fine-tuning on corrections. Use cases: Recent news updates, correcting misinformation, personalizing responses. Active research area for maintaining LLM accuracy.