Roofline Model Performance

The Roofline Model is the visual performance analysis framework that plots achievable computation throughput (FLOPS) against arithmetic intensity (FLOPS/byte) — creating a "roofline" ceiling defined by peak compute capacity (horizontal) and peak memory bandwidth (diagonal slope) that immediately reveals whether a kernel is compute-bound or memory-bound and quantifies the gap between achieved and theoretically achievable performance.

The Model

For a given hardware platform:
- Peak Compute (P): Maximum floating-point operations per second (e.g., 100 TFLOPS for an NVIDIA A100 at FP32).
- Peak Memory Bandwidth (B): Maximum bytes per second from main memory (e.g., 2 TB/s for HBM2e).
- Arithmetic Intensity (AI): FLOPS performed per byte loaded from memory for a specific kernel. AI = Total FLOPS / Total Bytes Transferred.

The roofline ceiling for a kernel with arithmetic intensity AI is: Achievable FLOPS = min(P, B × AI).

• If B × AI < P: the kernel is memory-bound — performance is limited by how fast data arrives, not how fast the ALUs compute. The kernel rides the diagonal (bandwidth-limited) slope.
• If B × AI ≥ P: the kernel is compute-bound — the ALUs are the bottleneck, and the kernel hits the horizontal (compute) ceiling.

Reading the Roofline Plot

Performance  |        _______________  (Peak Compute)
(GFLOPS)     |       /
             |      /  (Bandwidth Ceiling)
             |     /
             |    / * Kernel A (memory-bound, 70% of roof)
             |   /
             |  /         * Kernel B (compute-bound, 45% of roof)
             | /
             |/______________________________
                    Arithmetic Intensity (FLOP/Byte)

Kernel A is memory-bound at 70% of the bandwidth roof — optimizing should focus on data reuse (tiling, caching) to increase AI or reducing unnecessary loads.
Kernel B is compute-bound at 45% of the compute roof — optimizing should focus on vectorization, ILP, and instruction mix.

Extended Roofline

The basic model can be extended with additional ceilings:
- L1/L2 Cache Bandwidth: Separate diagonal ceilings for each cache level, showing whether a kernel is bound by main memory, L2, or L1 bandwidth.
- Mixed Precision: Different horizontal ceilings for FP64, FP32, FP16, INT8 — reflecting the different peak throughputs of each data type.
- Special Function: Separate ceilings for transcendental functions (sin, exp) which have lower throughput than FMA operations.

Practical Application

• GEMM (matrix multiply) has AI = O(N) — deep in the compute-bound region. Achieved performance should approach 90%+ of peak FLOPS.
• SpMV (sparse matrix-vector multiply) has AI = O(1) — firmly memory-bound. Performance is limited to 5-10% of peak FLOPS regardless of optimization.
• Convolution AI depends on filter size, channel count, and batch size — can be either compute-bound or memory-bound depending on configuration.

The Roofline Model is the performance engineer's X-ray machine — instantly diagnosing whether a kernel is starved for data or saturated with computation, and quantifying exactly how much performance headroom remains before hitting the hardware's fundamental limits.