Model Training

An epoch is one complete pass through the entire training dataset, a fundamental unit of training progress. Definition: Every example seen exactly once = one epoch. Multiple epochs means multiple passes. Typical training: Vision models often train 90-300 epochs. NLP models may train 1-3 epochs (large datasets) or more (small datasets). LLM pre-training: Often less than 1 epoch on massive web data. Chinchilla optimal suggests about 1 epoch is ideal. Multi-epoch considerations: Later epochs see same data, risk of overfitting. Learning rate schedules often tied to epochs. Shuffling: Shuffle data each epoch for better optimization. Different order prevents memorizing sequence. Steps per epoch: dataset size / batch size. Common way to measure training progress. Why multiple epochs: Limited data requires multiple passes to fully learn patterns. Each pass with different optimization state. Epoch vs iteration: Epoch is dataset-level, iteration/step is batch-level. May need thousands of iterations per epoch. Monitoring: Track loss per epoch to monitor progress. Compare train vs validation across epochs for overfitting detection.