Memory Coalescing

Memory Coalescing — organizing GPU global memory access patterns so that threads in a warp access consecutive memory addresses, allowing the hardware to combine individual requests into efficient bulk transactions.

How It Works

• GPU warp = 32 threads executing in lockstep
• If all 32 threads access consecutive 4-byte addresses → single 128-byte memory transaction (coalesced)
• If threads access scattered addresses → up to 32 separate transactions (uncoalesced, 10-30x slower)

Coalesced vs Uncoalesced

Coalesced (fast):              Uncoalesced (slow):
Thread 0 → addr[0]             Thread 0 → addr[0]
Thread 1 → addr[1]             Thread 1 → addr[100]
Thread 2 → addr[2]             Thread 2 → addr[37]
...                            ...
Thread 31 → addr[31]           Thread 31 → addr[999]
1 transaction (128 bytes)      Up to 32 transactions!

Common Patterns

• Array of Structures (AoS): Bad! Adjacent threads access fields of different structs → strided access
• Structure of Arrays (SoA): Good! Adjacent threads access consecutive elements of same array → coalesced

AoS (bad): struct { float x,y,z; } particles[N];  // thread i reads particles[i].x
SoA (good): float x[N], y[N], z[N];               // thread i reads x[i] ← coalesced!

Rules for Coalescing

• Thread i should access address base + i (or base + i*sizeof(element))
• Alignment to 128 bytes helps
• Avoid strided access patterns in inner loops

Memory coalescing is the most impactful GPU optimization after shared memory — an uncoalesced kernel can run 10-30x slower than a coalesced one.