Global Memory

Global Memory in GPU architecture refers to the main off-chip DRAM accessible by all threads across all streaming multiprocessors (SMs).

## What Is Global Memory?

• Capacity: 4GB to 80GB+ on modern GPUs (HBM2/GDDR6)
• Bandwidth: 500GB/s to 3TB/s depending on memory type
• Latency: 400-800 clock cycles (much slower than shared memory)
• Scope: Accessible by all threads in all blocks

## Why Global Memory Matters

Global memory is where large datasets, model weights, and results reside. Despite high bandwidth, poor access patterns cause performance bottlenecks.

// Global memory access example
__global__ void kernel(float *globalData) {
    int idx = blockIdx.x * blockDim.x + threadIdx.x;
    
    // Coalesced access - threads read consecutive addresses
    float val = globalData[idx];  // Good pattern
    
    // Strided access - inefficient, multiple transactions
    float val2 = globalData[idx * 32];  // Bad pattern
}

Optimization Tips:
- Coalesce memory accesses (consecutive threads → consecutive addresses)
- Use shared memory as cache for repeated accesses
- Align data structures to 128-byte boundaries