Continual Learning

Continual Learning

What is Continual Learning?
Learning new tasks sequentially without forgetting previously learned tasks, enabling models to accumulate knowledge over time.

The Forgetting Problem
When training on new tasks, models tend to overwrite weights for old tasks:

Task 1: Learn A, B, C  -->  Model knows A, B, C
Task 2: Learn D, E     -->  Model knows D, E, forgets A, B, C

This is called "catastrophic forgetting."

Approaches to Prevent Forgetting

Regularization Methods
Penalize changes to important weights:

# Elastic Weight Consolidation (EWC)
def ewc_loss(model, importance, old_params, lambda_):
    loss = 0
    for name, param in model.named_parameters():
        loss += (importance[name] * (param - old_params[name])**2).sum()
    return lambda_ * loss

# Add to training loss
total_loss = task_loss + ewc_loss(model, fisher, prev_params, 1000)

Replay Methods
Store and replay old examples:

class ReplayBuffer:
    def __init__(self, size_per_task=100):
        self.buffer = []
        self.size_per_task = size_per_task

    def add_task(self, task_data):
        samples = random.sample(task_data, self.size_per_task)
        self.buffer.extend(samples)

    def get_replay_batch(self, size):
        return random.sample(self.buffer, size)

Architecture Methods
Add new capacity for new tasks:

# Progressive networks: Add new column per task
# PackNet: Prune and freeze for each task
# Modular networks: Route to task-specific experts

Comparison
| Method | Memory | Compute | Performance |
|--------|--------|---------|-------------|
| EWC | Low | Medium | Medium |
| Replay | Medium | Low | High |
| Progressive | High | Low | High |
| PackNet | Low | Low | Medium |

Metrics
| Metric | Definition |
|--------|------------|
| Accuracy | Performance on current task |
| Backward transfer | Effect on old tasks |
| Forward transfer | Effect on learning new tasks |
| Forgetting | Accuracy drop on old tasks |

Use Cases
- Chatbots learning from conversations
- Robots adapting to new environments
- Recommendation systems evolving with trends
- Any scenario with sequential data streams

Best Practices
- Evaluate on all tasks, not just current
- Use replay buffers when storage allows
- Consider task similarity for transfer
- Monitor for catastrophic forgetting