Branchynet

BranchyNet is one of the pioneering early exit network architectures — introducing side branch classifiers at intermediate layers of a deep neural network, enabling fast inference for easy samples while maintaining accuracy for difficult samples through the full network.

BranchyNet Architecture

• Main Network: Standard deep CNN (VGG, ResNet, etc.) as the backbone.
• Branches: Lightweight classifier branches attached at selected intermediate layers.
• Entropy Criterion: Exit at a branch if the prediction entropy is below a threshold — low entropy = high confidence.
• Joint Training: All branches and the main network are trained end-to-end with a combined loss.

Why It Matters

• Foundational: One of the first works to formalize early exit in deep networks for adaptive inference.
• Speedup: 2-5× inference speedup for easy samples with minimal accuracy loss.
• Influence: Inspired MSDNet, SCAN, and many subsequent adaptive inference architectures.

BranchyNet is the original early exit network — pioneering the idea of attaching intermediate classifiers for input-adaptive, efficient inference.