GPU Warp Scheduling

GPU Warp Scheduling — the mechanism by which a GPU's streaming multiprocessor (SM) manages and interleaves execution of warps (groups of 32 threads) to hide memory latency.

SIMT Execution

• SIMT (Single Instruction Multiple Threads): All 32 threads in a warp execute the same instruction simultaneously on different data
• If threads take different branches → warp divergence — some threads are masked off, executed serially
• Divergence can halve (or worse) performance

Latency Hiding

• GPU hides memory latency (hundreds of cycles) by switching to another warp
• While warp A waits for data, warp B, C, D execute
• Need enough active warps to keep the SM busy → occupancy

Occupancy

• $Occupancy = \frac{\text{active warps}}{\text{maximum warps per SM}}$
• Limited by: registers per thread, shared memory per block, threads per block
• Higher occupancy = better latency hiding (usually)
• But: Sometimes lower occupancy with more registers per thread is faster

Warp Scheduling Policies

• Round-Robin: Each ready warp gets a turn
• Greedy-Then-Oldest (GTO): Execute same warp until it stalls, then switch
• Two-Level: Group warps into fetch groups

Understanding warp behavior is essential for writing efficient GPU code — the difference between naive and optimized kernels can be 10-100x.