Catastrophic Interference

Catastrophic interference (also called catastrophic forgetting) is the phenomenon where a neural network trained on a new task abruptly and severely forgets previously learned knowledge. It is the central challenge of continual learning — standard neural networks are fundamentally poor at accumulating knowledge across sequential tasks.

Why It Happens

• Shared Weights: Neural networks store all knowledge in the same set of weights. When weights are updated for a new task, the changes overwrite information stored for previous tasks.
• Gradient Descent: Optimization moves weights in whatever direction minimizes loss on the current task, with no constraint to preserve performance on old tasks.
• No Explicit Memory: Unlike human brains, standard neural networks have no mechanism to consolidate and protect important memories.

Examples

• A model trained on Task A (classifying animals) then trained on Task B (classifying vehicles) may lose the ability to classify animals entirely.
• Fine-tuning a pre-trained LLM for one specific task can degrade its general capabilities.
• An AI agent learning new skills may suddenly lose previously mastered skills.

Mitigation Strategies

• Regularization-Based: EWC (Elastic Weight Consolidation) identifies weights important for previous tasks and penalizes changes to them. Other methods: SI (Synaptic Intelligence), MAS (Memory Aware Synapses).
• Replay-Based: Experience replay stores examples from old tasks and replays them during new task training to maintain old knowledge.
• Architecture-Based: Progressive neural networks add new capacity for each task rather than reusing existing weights. PackNet uses weight pruning to allocate subnetworks per task.
• Knowledge Distillation: Use the model's own outputs on old tasks as soft targets (teacher) while learning new tasks.

Relevance to LLMs

• Fine-tuning LLMs can cause catastrophic forgetting of general knowledge — mitigated by LoRA (which modifies only a small subset of parameters) and careful learning rate selection.
• RLHF can cause forgetting of pre-training knowledge — known as the alignment tax.

Catastrophic interference is the fundamental barrier to building AI systems that learn continuously — overcoming it is essential for lifelong learning systems.