Federated Learning

Federated Learning

What is Federated Learning?
Training ML models across decentralized data sources without sharing raw data, preserving privacy while enabling collaborative learning.

How It Works

Central Server
    |
    | Model weights
    v
[Device 1] [Device 2] [Device 3]
    |           |           |
    | Local     | Local     | Local
    | training  | training  | training
    |           |           |
    v           v           v
Local updates aggregated by server

FedAvg Algorithm

def federated_averaging(server_model, clients, rounds=100):
    for round in range(rounds):
        client_weights = []

        # Each client trains locally
        for client in clients:
            local_model = copy(server_model)
            local_model.train(client.data)
            client_weights.append(local_model.state_dict())

        # Aggregate weights (simple average)
        averaged_weights = {}
        for key in server_model.state_dict():
            averaged_weights[key] = sum(
                w[key] for w in client_weights
            ) / len(clients)

        server_model.load_state_dict(averaged_weights)

    return server_model

Challenges
| Challenge | Description |
|-----------|-------------|
| Non-IID data | Clients have different data distributions |
| System heterogeneity | Different compute/network capabilities |
| Communication cost | Sending model updates is expensive |
| Privacy attacks | Gradients can leak information |

Privacy Enhancements
| Technique | Protection |
|-----------|------------|
| Differential privacy | Add noise to updates |
| Secure aggregation | Encrypt updates |
| Local differential privacy | Noise at client |
| Compression | Reduce communication |

Differential Privacy in FL

def dp_sgd_update(gradients, clip_norm, noise_scale):
    # Clip gradient norm
    grad_norm = torch.norm(gradients)
    gradients = gradients * min(1, clip_norm / grad_norm)

    # Add noise
    noise = torch.randn_like(gradients) * noise_scale
    return gradients + noise

Frameworks
| Framework | Features |
|-----------|----------|
| Flower | Flexible, framework-agnostic |
| PySyft | Privacy-focused |
| TensorFlow Federated | Google, production-ready |
| FATE | Enterprise FL |

Use Cases
| Domain | Application |
|--------|-------------|
| Healthcare | Train on hospital data without sharing |
| Mobile | Keyboard prediction with user data |
| Finance | Fraud detection across institutions |
| IoT | Edge device collaborative learning |

Best Practices
- Handle non-IID data with appropriate algorithms
- Compress updates for communication efficiency
- Add differential privacy for strong guarantees
- Validate federated models carefully