Per-Channel Quantization
Per-channel quantization applies different quantization parameters (scale and zero-point) to each output channel (filter) in a convolutional or linear layer, rather than using a single set of parameters for the entire tensor.
How It Works
- Per-Tensor: One scale $s$ and zero-point $z$ for the entire weight tensor. All channels share the same quantization range.
- Per-Channel: Each output channel $c$ has its own scale $s_c$ and zero-point $z_c$. Channels with larger weight magnitudes get larger scales.
Formula
For a weight tensor $W$ with shape [out_channels, in_channels, height, width]:
$$q_{c,i,h,w} = \text{round}(W_{c,i,h,w} / s_c + z_c)$$
Where $c$ is the output channel index.
Why Per-Channel Matters
- Channel Variance: Different filters in a layer often have very different weight magnitude distributions. Some channels may have weights in [-0.1, 0.1], others in [-2.0, 2.0].
- Better Utilization: Per-channel quantization allows each channel to use the full quantization range optimally, reducing quantization error.
- Accuracy Improvement: Typically provides 1-3% accuracy improvement over per-tensor quantization with minimal overhead.
Trade-offs
- Storage: Requires storing one scale (and optionally zero-point) per output channel. For a layer with 256 channels, this adds 256 floats (~1KB) — negligible compared to the weight tensor itself.
- Computation: Slightly more complex dequantization (each channel uses its own scale), but modern hardware handles this efficiently.
- Compatibility: Widely supported in quantization frameworks (TensorFlow Lite, PyTorch, ONNX Runtime).
Example
Consider a Conv2D layer with 64 output channels:
- Per-Tensor: All 64 channels share one scale. If channel 0 has weights in [-0.05, 0.05] and channel 63 has weights in [-1.5, 1.5], the shared scale must accommodate [-1.5, 1.5], wasting precision for channel 0.
- Per-Channel: Channel 0 gets scale $s_0 = 0.05/127$, channel 63 gets scale $s_{63} = 1.5/127$. Both channels use their quantization range optimally.
Standard Practice
- Weights: Almost always use per-channel quantization (standard in TensorFlow Lite, PyTorch).
- Activations: Typically use per-tensor quantization (per-channel activations are less common due to runtime overhead).
Per-channel quantization is a best practice for weight quantization, providing significant accuracy benefits with minimal cost.