Supervised

Supervised Fine-Tuning (SFT)

What is SFT?
Supervised Fine-Tuning trains a pretrained LLM on curated (instruction, response) pairs to follow instructions and produce helpful outputs. It is typically the first step after pretraining.

Data Format

{
  "instruction": "Write a haiku about programming",
  "input": "",
  "output": "Lines of code flow down
Debugging through the night hours
Compiler agrees"
}

Or in conversation format:

{
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Write a haiku about programming"},
    {"role": "assistant", "content": "Lines of code flow down
Debugging through the night hours
Compiler agrees"}
  ]
}

Dataset Recommendations

Dataset Sizes
| Use Case | Recommended Size |
|----------|------------------|
| Domain adaptation | 1K-10K examples |
| Instruction following | 10K-50K examples |
| Full capability tuning | 50K-500K examples |

Popular Open Datasets
| Dataset | Size | Focus |
|---------|------|-------|
| OpenAssistant/oasst1 | 161K | Multi-turn conversations |
| Dolly-15K | 15K | Diverse instructions |
| Alpaca-52K | 52K | GPT-generated instructions |
| WizardLM | 196K | Complex instruction evolution |
| CodeAlpaca | 20K | Coding tasks |

SFT Best Practices
1. Quality over quantity: 1K excellent examples > 100K mediocre ones
2. Diversity: Cover wide range of tasks and formats
3. Formatting consistency: Same structure across examples
4. Response length: Match desired output length distribution
5. Human review: Verify a sample of training data manually

Training Considerations
- Epochs: 1-3 (avoid overfitting)
- Learning rate: 1e-5 to 5e-5 for full fine-tuning
- Use LoRA/QLoRA for parameter-efficient training
- Validate on held-out set to monitor overfitting