Self-Paced Learning

Self-Paced Learning is a curriculum learning variant where the model itself decides which training examples to include — the model's own loss on each example determines difficulty, and a pace parameter controls how many "hard" examples are included as training progresses.

Self-Paced Formulation

• Loss Threshold: Include example $i$ if $L(x_i) < \lambda$ — low-loss examples are "easy" and included first.
• Pace Parameter ($\lambda$): Increases over training — starts with only easy examples, gradually includes harder ones.
• Binary Variable: $v_i \in {0,1}$ indicates whether example $i$ is included in the current training set.
• Joint Optimization: Alternate between optimizing model parameters $\theta$ and sample weights $v$.

Why It Matters

• No External Teacher: Unlike standard curriculum learning, self-paced learning doesn't need a difficulty oracle — the model defines its own curriculum.
• Robust to Noise: Noisy/mislabeled examples have high loss — they are naturally excluded until late in training.
• Autonomous: The model autonomously manages its own learning pace.

Self-Paced Learning is the model teaches itself — automatically selecting training examples by difficulty based on its own evolving understanding.