Splitfed Learning

SplitFed Learning is a hybrid approach combining Split Learning and Federated Learning — like split learning, the model is split between clients and server, but like federated learning, multiple clients' lower-model updates are aggregated to train a shared lower model.

How SplitFed Works

• Split: Model is split at layer $k$ — clients have layers 1-$k$, server has layers $k+1$-$L$.
• Parallel Clients: Multiple clients simultaneously process their data through their local lower models.
• Server Aggregation (Top): Server receives activations from all clients, processes through top model.
• Client Aggregation (Bottom): After backward pass, clients' lower models are aggregated (FedAvg style).

Why It Matters

• Scalability: Unlike vanilla split learning (sequential), SplitFed supports parallel client training.
• Communication: Only intermediate activations (not full model) are communicated — reduced communication.
• Flexibility: Combines the compute-sharing of split learning with the parallelism of federated learning.

SplitFed is the best of both worlds — combining split learning's model partitioning with federated learning's parallel aggregation.