Roofline Model Analysis

Roofline Model Analysis is the visual performance modeling framework that plots achievable performance (FLOP/s) against arithmetic intensity (FLOP/byte) to determine whether a computation is memory-bound or compute-bound — providing immediate insight into the performance bottleneck and the maximum achievable speedup, making it the most practical first-step analysis tool for understanding and optimizing the performance of any computational kernel on any hardware.

Roofline Construction

X-axis: Arithmetic Intensity (AI) = FLOPs / Bytes transferred (operational intensity).
Y-axis: Attainable Performance (GFLOP/s or TFLOP/s).
Memory ceiling: Diagonal line with slope = memory bandwidth. Performance = AI × BW.
Compute ceiling: Horizontal line at peak compute rate.
Performance = min(Peak_Compute, AI × Peak_Bandwidth).

Roofline for NVIDIA A100

Peak FP32: 19.5 TFLOPS
HBM Bandwidth: 2.0 TB/s
Ridge Point: 19,500 / 2,000 = 9.75 FLOP/byte

  TFLOP/s
    19.5 |__________________________ (compute ceiling)
         |                  /
         |                /
         |              /    ← memory ceiling (slope = 2 TB/s)
         |            /
         |          /
         |        /
         |      /
         |    /
         |  /
         |/__________________________ AI (FLOP/byte)
                9.75
              (ridge point)

Left of ridge: Memory-bound → optimize memory access (coalescing, caching, reuse).
Right of ridge: Compute-bound → optimize computation (SIMD, FMA, algorithm efficiency).

Computing Arithmetic Intensity

Kernel	FLOPs/element	Bytes/element	AI	Bound
Vector add (a+b→c)	1	12 (3×4B)	0.08	Memory
Dot product	2N	8N+4	~0.25	Memory
Dense GEMM (NxN)	2N³	3×4N²	N/6	Compute (for large N)
1D stencil (3-point)	2	4 (with reuse)	0.5	Memory
SpMV (sparse)	2×NNZ	12×NNZ	0.17	Memory

Roofline Extensions

Ceiling	Description
L1 bandwidth ceiling	Performance bound by L1 cache bandwidth
L2 bandwidth ceiling	Performance bound by L2 cache bandwidth
SIMD ceiling	Penalty for non-vectorized code
FMA ceiling	Penalty for not using fused multiply-add
Tensor Core ceiling	Peak when using tensor cores (mixed precision)

Using Roofline for Optimization

1. Profile kernel: Measure actual FLOP/s and bytes transferred.
2. Plot on roofline: Where does the kernel sit relative to ceilings?
3. If below memory ceiling: Memory access inefficiency → fix coalescing, add caching.
4. If at memory ceiling: Memory-bound → increase AI (algorithm change, tiling, reuse).
5. If at compute ceiling: Compute-bound → use wider SIMD, tensor cores, better algorithm.

Tools

Intel Advisor: Automated roofline analysis for CPU.
NVIDIA Nsight Compute: Roofline chart for GPU kernels.
Empirical Roofline Toolkit (ERT): Measures actual machine ceilings.

The roofline model is the most effective framework for understanding computational performance — by instantly revealing whether a kernel is memory-bound or compute-bound and quantifying the gap to peak performance, it guides optimization effort toward the actual bottleneck rather than wasting time on non-limiting factors.

Roofline Model Analysis

Take roofline model analysis further