Layer Skipping
Layer Skipping is a transformer inference optimization technique that bypasses intermediate layers for tokens or sequences that do not require full-depth processing, using learned skip connections, router-based decisions, or progressive training strategies that build skip-robust representations — exploiting the empirical observation that many transformer layers perform incremental refinements rather than critical transformations, and that later layers often contribute marginally for straightforward inputs.
What Is Layer Skipping?
- Definition: Layer skipping modifies the standard sequential layer-by-layer processing of transformers by allowing tokens to jump directly from layer N to layer N+K via the residual connection, bypassing the self-attention and feed-forward computation of the intervening layers. The decision of which layers to skip can be static (predetermined), learned (router-based), or stochastic (random during training for robustness).
- Residual Bypass: The skip mechanism leverages the residual connections already present in transformer architectures. When a layer is skipped, the token's hidden state passes unchanged through the residual stream to the next active layer — meaning skipping is computationally free and does not require special architectural modifications beyond the routing decision.
- Distinction from Early Exit: Early exit terminates all computation at an intermediate layer and produces a final output. Layer skipping selectively bypasses specific layers while continuing processing at deeper layers — allowing the network to access the final layers' representations even when intermediate layers are bypassed.
Why Layer Skipping Matters
- Inference Speedup: Bypassing 20–40% of layers reduces inference FLOP count proportionally. For autoregressive generation where the forward pass is the bottleneck, this translates directly to tokens-per-second improvement. Implementations report 20–40% latency reduction with less than 1% quality degradation on standard benchmarks.
- Layer Redundancy: Empirical analysis of trained transformers reveals significant redundancy in intermediate layers. CKA (Centered Kernel Alignment) similarity between consecutive layer representations is often >0.95, indicating that adjacent layers make only minor refinements. Layer skipping exploits this redundancy by bypassing near-duplicate layers.
- Training Robustness: Progressive layer dropping during training (randomly skipping layers with increasing probability) forces the network to build representations that are robust to missing intermediate computation. This creates a model that can tolerate layer skipping at inference without the quality collapse that would occur in a conventionally trained model.
- Complementary to Quantization: Layer skipping and weight quantization are orthogonal optimization axes that can be combined. A model with 50% layer skip and 4-bit quantization achieves compound efficiency gains — reducing both arithmetic intensity (fewer layers) and memory bandwidth (smaller weights per layer).
Layer Skipping Approaches
| Technique | Mechanism | Key Benefit |
|---|---|---|
| Stochastic Depth | Random layer dropping during training | Builds skip-robust representations |
| Learned Routing | Per-token router decides skip/execute at each layer | Adaptive to input difficulty |
| Static Pruning | Remove least-important layers post-training based on importance metrics | Simple deployment, no routing overhead |
| Block Skipping | Skip groups of consecutive layers rather than individual layers | Reduces routing decisions |
Layer Skipping is selective depth processing — the inference optimization that recognizes not every transformer layer contributes equally to every prediction, enabling models to bypass redundant computation while preserving the critical processing pathways that determine output quality.