Recency Bias

Recency Bias in neural network training is the tendency for models to be disproportionately influenced by recently seen training examples — especially in online or sequential training settings, the model's predictions are biased toward the data distribution of recent mini-batches, potentially forgetting earlier patterns.

Recency Bias Manifestations

• Catastrophic Forgetting: In continual learning, the model overwrites knowledge from earlier tasks with recent data.
• Order Sensitivity: The order of training data affects the final model — later data has more influence.
• Streaming Data: In online learning, the model tracks recent trends but may forget older patterns.
• Batch Composition: The last few batches disproportionately affect predictions — temporal proximity matters.

Why It Matters

• Data Ordering: Shuffling training data mitigates recency bias — standard practice in SGD.
• Continual Learning: Recency bias is the core challenge in continual learning — preventing it requires replay, regularization, or isolation.
• Process Monitoring: Models deployed for drift detection must balance recency (adapting to new conditions) with memory (remembering rare events).

Recency Bias is the tyranny of the latest data — the model's tendency to overweight recent examples at the expense of earlier knowledge.