Numa Aware Programming

NUMA-Aware Programming is the performance optimization discipline for multi-socket and chiplet-based systems where memory access latency and bandwidth depend on the physical location of the memory relative to the processor — where NUMA-oblivious code can suffer 2-4x performance degradation because remote memory accesses (cross-socket or cross-chiplet) take 1.5-3x longer than local accesses, making data placement and thread affinity the dominant factors in memory-bound application performance.

NUMA Architecture

In a NUMA system, each processor (socket/chiplet) has its own local memory controller and DRAM. Accessing local memory: ~80-100 ns. Accessing remote memory (through the interconnect — Intel UPI, AMD Infinity Fabric): ~130-200 ns. The latency asymmetry is the "non-uniform" in NUMA.

Example: 2-Socket AMD EPYC

Each socket has 4 CCDs (chiplet core dies), each with its own L3 cache and a local slice of the memory channels. Memory access hierarchy:
1. Same CCD L3: ~10 ns
2. Same socket, different CCD: ~30-50 ns
3. Same socket, different memory controller: ~80-100 ns
4. Remote socket: ~130-200 ns

NUMA Optimization Techniques

• First-Touch Allocation: Linux NUMA default policy. Memory pages are allocated on the NUMA node of the first thread that touches (writes to) them. If the initializing thread is on node 0 but the computing thread is on node 1, all accesses are remote. Fix: initialize data on the same threads that will process it.

• Thread-Memory Affinity: Bind threads to specific cores/NUMA nodes using `numactl --cpunodebind=0 --membind=0`, `sched_setaffinity()`, or OpenMP `OMP_PLACES=cores OMP_PROC_BIND=close`. Ensures threads access local memory.

• Interleaved Allocation: `numactl --interleave=all` distributes pages round-robin across all nodes. Provides uniform average latency at the cost of no locality optimization. Useful for shared data accessed by all nodes equally.

• NUMA-Aware Data Structures: Allocate per-node copies of frequently-read data (replication). For producer-consumer patterns, place the buffer on the consumer's node (reads are more latency-sensitive than writes due to store buffers).

Detecting NUMA Issues

• `numastat -p <pid>`: Shows per-node memory allocation and remote access counts.
• `perf stat -e node-load-misses,node-store-misses`: Hardware counters for remote memory accesses.
• Intel VTune / AMD uProf: NUMA-specific analysis modes visualize memory access locality.

NUMA in Practice

• Databases: PostgreSQL, MySQL allocate buffer pools NUMA-aware. Connection threads are pinned to the same node as their buffer pages.
• HPC: MPI rank placement matches NUMA topology. One rank per NUMA node, with OpenMP threads within each rank placed on the same node.
• Cloud/VMs: VM placement must respect NUMA boundaries. A VM spanning two NUMA nodes suffers remote access penalties on half its memory.

NUMA-Aware Programming is the essential optimization for modern multi-socket and chiplet servers — ensuring that data lives close to the processor that uses it, because in a NUMA system, WHERE you allocate memory matters as much as HOW you access it.