Continual Learning
Continual Learning is the family of training methodologies that enable a neural network to learn new tasks or absorb new data distributions sequentially without destroying the knowledge it acquired from earlier tasks — directly combating the fundamental failure mode known as catastrophic forgetting.
Why Catastrophic Forgetting Happens
Standard gradient descent treats parameter space as a blank slate. When a model trained on Task A is fine-tuned on Task B, the gradients for Task B freely overwrite the weights that encoded Task A. After just a few epochs, performance on Task A can drop to random chance even though the model excels on Task B.
Major Strategy Families
- Regularization Methods (EWC, SI): Elastic Weight Consolidation computes the Fisher Information Matrix to identify which weights are most important for prior tasks, then adds a quadratic penalty discouraging large updates to those weights during new-task training. Synaptic Intelligence achieves similar protection by tracking cumulative gradient contributions online, avoiding the expensive Fisher computation.
- Replay Methods: The model maintains a fixed-size memory buffer of representative examples from prior tasks and interleaves them into new-task training batches. Generative replay replaces real stored samples with synthetic examples produced by a generative model trained alongside the main classifier.
- Architecture Methods (Progressive Networks): Each new task receives a fresh set of parameters (a new column), while lateral connections allow it to leverage features learned in frozen prior-task columns. Forgetting is eliminated entirely because prior weights are never modified.
Engineering Tradeoffs
| Method | Forgetting Risk | Memory Cost | Compute Overhead |
|---|---|---|---|
| EWC | Moderate (approximate protection) | Low (Fisher diagonal only) | Moderate (Fisher computation per task) |
| Replay Buffer | Low (direct rehearsal) | Grows with tasks | Low per step (small buffer samples) |
| Progressive Nets | Zero (frozen columns) | High (parameters grow linearly) | Forward pass cost grows per task |
When Each Approach Fits
EWC and SI work well when the task sequence is short (5-10 tasks) and memory is constrained. Replay dominates when data storage is feasible and the number of tasks is large. Progressive networks suit hardware-constrained pipelines (such as robotics) where guaranteed zero-forgetting outweighs the parameter growth.
Continual Learning is the engineering bridge between static model training and real-world deployment — where data never stops arriving and retraining from scratch on every distribution shift is economically impossible.