Federated Learning Basics
Federated Learning — a distributed training approach where models are trained across many decentralized devices (phones, hospitals, banks) without sharing raw data, preserving privacy.
How It Works
1. Server sends global model to N client devices
2. Each device trains on its local data for a few epochs
3. Devices send only model updates (gradients/weights) back to server — NOT the raw data
4. Server aggregates updates (FedAvg: weighted average) → new global model
5. Repeat for many rounds
Why Federated Learning?
- Privacy: Raw data never leaves the device (medical records, financial data, personal messages)
- Regulation: GDPR, HIPAA compliance — data can't be centralized
- Scale: Billions of mobile devices as training nodes (Google Keyboard predictions trained this way)
Challenges
- Non-IID data: Each device has different data distribution (heterogeneous)
- Communication cost: Sending model updates is expensive over mobile networks
- Stragglers: Some devices are slow or drop out
- Privacy attacks: Gradient inversion can partially reconstruct training data
Real Applications
- Google Gboard: Next-word prediction trained on-device
- Apple: Siri improvements without collecting voice data
- Healthcare: Multi-hospital medical imaging models
Federated learning makes it possible to train AI on sensitive data that could never be collected into a single dataset.