Early Exit Networks

Early Exit Networks are neural networks with intermediate classifiers at multiple layers that allow easy inputs to exit early — if an intermediate classifier is confident enough, the remaining layers are skipped, saving computation for simple inputs while using the full network for difficult ones.

How Early Exit Works

• Exit Branches: Attach classifiers (small heads) at intermediate layers of the network.
• Confidence Threshold: If an exit branch's confidence exceeds a threshold $\tau$, output that prediction.
• Skip Remaining: All subsequent layers and exits are skipped — computation savings proportional to exit position.
• Training: Train exit branches jointly with the main network, balancing all exit losses.

Why It Matters

• Adaptive Compute: Easy inputs use less computation — average FLOPs per sample decreases significantly.
• Latency: In real-time systems, early exits guarantee latency bounds — hard cases are truncated.
• Edge Deployment: Enables deploying large models on edge by averaging less computation.

Early Exit Networks are fast-tracking the easy cases — letting confident intermediate predictions bypass the remaining computation.