Roofline Model
Roofline Model — a visual framework for understanding whether a computation is limited by compute throughput or memory bandwidth, guiding optimization efforts.
The Model
$$Performance = min(Peak\_FLOPS, Peak\_BW \times OI)$$
Where:
- OI (Operational Intensity) = FLOPs / Bytes transferred from memory
- Peak FLOPS: Maximum compute throughput (e.g., 10 TFLOPS)
- Peak BW: Maximum memory bandwidth (e.g., 900 GB/s for HBM)
Two Regimes
- Memory-Bound (low OI): Performance limited by how fast data can be fed to compute units. Most deep learning inference, sparse computations
- Compute-Bound (high OI): Performance limited by arithmetic throughput. Dense matrix multiply, convolutions with large batch sizes
Example (NVIDIA A100)
- Peak: 19.5 TFLOPS (FP32), 2 TB/s (HBM2e)
- Ridge point: 19.5T / 2T = ~10 FLOP/Byte
- If your kernel does < 10 FLOP per byte loaded → memory-bound
- If > 10 → compute-bound
Optimization Strategy
- Memory-bound → reduce data movement (tiling, caching, compression, data reuse)
- Compute-bound → use tensor cores, vectorization, reduce wasted compute
The roofline model quickly tells you what's limiting performance and where to focus optimization — essential for HPC and GPU programming.