Roofline Model
The roofline model is a one-picture performance framework: it plots attainable compute throughput against arithmetic intensity, so you can see at a glance whether a kernel is limited by the chip's math units or by its memory bandwidth.
Two ceilings, one plot. The y-axis is performance (FLOP/s); the x-axis is arithmetic intensity (FLOPs done per byte moved from memory). A sloped line — the memory roof — rises at the machine's peak bandwidth, and a flat line — the compute roof — caps out at peak FLOP/s. Every kernel sits under whichever roof is lower at its intensity.
The ridge point splits the world. Where the two roofs meet is the ridge, at arithmetic intensity = peak FLOPs / peak bandwidth. Left of it a kernel is memory-bound: it starves the math units, and only faster memory (HBM) helps. Right of it a kernel is compute-bound: the pipes are full, and only more or faster FLOPs help. On an H100-class GPU the FP16 ridge sits near a few hundred FLOP per byte — which is exactly why so much LLM inference lands on the memory-bound side.
| Regime | Where | Limited by | Lever that helps | Example |
|---|---|---|---|---|
| Memory-bound | left of ridge | HBM bandwidth | faster memory, more reuse | attention, GEMV, decode |
| Balanced | at the ridge | both | matched tiling | tuned GEMM |
| Compute-bound | right of ridge | peak FLOP/s | more/faster tensor cores | large GEMM, big-batch training |
You move a kernel by changing its intensity. Tiling, kernel fusion, keeping data resident in registers or SRAM, and larger batch sizes all raise arithmetic intensity, sliding a kernel rightward toward the compute roof. This is why FlashAttention is such a large win: by fusing the attention kernel so it never re-reads the big score matrix from HBM, it raises intensity and lifts the kernel off the memory roof.
Read the roofline through a quant lens rather than a tuning-tips lens: it is the single diagram that ties together every other number on this site. HBM bandwidth sets the slope, the tensor core sets the ceiling, and arithmetic intensity — fixed by the algorithm and the memory hierarchy — decides which one you actually hit. Optimizing hardware or software without knowing which side of the ridge you are on is guessing.