Numa Aware Programming Optimization

NUMA-Aware Programming is the practice of structuring parallel applications to account for Non-Uniform Memory Access architecture — where memory access latency and bandwidth depend on the physical distance between the processor core and the memory controller, with local access being 1.5-3× faster than remote access across interconnect links.

NUMA Architecture:
- NUMA Nodes: each processor socket (or chiplet cluster) has a local memory controller and attached DRAM — accessing local memory takes ~80 ns while remote memory access through interconnect (QPI, UPI, Infinity Fabric) takes ~120-250 ns
- Topology Discovery: operating systems expose NUMA topology through sysfs (/sys/devices/system/node/) or hwloc library — applications query topology to determine which cores belong to which NUMA nodes and the distance matrix between nodes
- Interconnect Bandwidth: inter-socket links provide 50-200 GB/s depending on generation — saturating remote bandwidth with memory-intensive workloads causes severe contention and performance degradation
- Multi-Socket Servers: 2-socket and 4-socket servers are common in HPC and enterprise — 4-socket systems have 2-hop remote access adding additional latency; 8-socket systems (rare) have even deeper NUMA hierarchies

Memory Allocation Policies:
- First-Touch Policy: default Linux policy — memory pages allocated on the NUMA node where the first accessing thread runs; initialization pattern determines permanent placement
- Interleave Policy: pages round-robin across all NUMA nodes — provides average performance across all cores but optimal for no specific core; useful for shared data accessed equally by all threads
- NUMA-Bind Policy: explicitly bind allocation to a specific node — ensures data stays local to the threads that access it; implemented via numactl --membind or numa_alloc_onnode()
- Migration: transparent page migration moves pages closer to their most frequent accessor — enabled via AutoNUMA/NUMA balancing in Linux kernel; adds overhead but automatically corrects poor initial placement

Thread Affinity and Binding:
- Thread Pinning: bind threads to specific cores using pthread_setaffinity or OMP_PROC_BIND — prevents migration that would separate a thread from its local memory, catastrophically increasing access latency
- Core Binding Strategies: close binding (fill one socket first) maximizes cache sharing; spread binding (distribute across sockets) maximizes total bandwidth — optimal strategy depends on workload characteristics
- Hyper-Threading Considerations: binding compute-intensive threads to physical cores (not HT siblings) avoids resource contention — memory-intensive threads may benefit from HT by overlapping computation with memory stalls

NUMA-aware programming is essential for achieving scalable performance on modern multi-socket servers — applications that ignore NUMA topology commonly lose 30-50% of theoretical performance due to remote memory access penalties and interconnect contention.