Neural Network Surgery

Neural Network Surgery is the practice of directly modifying a trained neural network's internal structure — adding, removing, or reconnecting layers and neurons post-training to improve performance, efficiency, or adapt to new tasks.

What Is Neural Network Surgery?

• Definition: Direct manipulation of network topology or weights after initial training.
• Operations:
• Pruning: Remove unnecessary neurons or connections.
• Grafting: Insert pre-trained modules from another network.
• Splicing: Connect two networks or sub-networks together.
• Layer Removal: Delete redundant layers (e.g., in over-deep ResNets).

Why It Matters

• Efficiency: Surgery can remove 90% of parameters with < 1% accuracy loss.
• Adaptation: Quickly customize a general model for a specific deployment target.
• Debugging: Remove or replace layers that cause specific failure modes.

Neural Network Surgery is precision engineering for AI — treating trained models as modular systems that can be optimized and reconfigured post-hoc.