Thread Affinity
Thread Affinity and CPU Pinning is the operating system and runtime technique of binding specific threads or processes to designated CPU cores — preventing the OS scheduler from migrating threads between cores, which eliminates cache migration penalties, reduces NUMA cross-socket traffic, and provides deterministic performance for latency-sensitive and throughput-critical workloads, with proper affinity configuration improving HPC and ML training performance by 10-30% on multi-socket servers.
Why Thread Affinity Matters
- Default OS scheduler: Migrates threads to balance load across cores.
- Migration cost: L1/L2 cache is per-core → migrated thread starts with cold cache.
- Cross-NUMA migration: Thread moves to core on different socket → memory accesses become remote (2-3× latency).
- Jitter: Unpredictable migration causes latency spikes → problematic for real-time and HPC.
Setting Affinity
# Linux: Pin process to cores 0-3
taskset -c 0-3 ./my_application
# Linux: Pin to specific NUMA node
numactl --cpunodebind=0 --membind=0 ./my_application
# OpenMP: Set affinity
export OMP_PROC_BIND=close
export OMP_PLACES=cores// Programmatic affinity (Linux)
#include <sched.h>
cpu_set_t cpuset;
CPU_ZERO(&cpuset);
CPU_SET(0, &cpuset); // Pin to core 0
pthread_setaffinity_np(thread, sizeof(cpu_set_t), &cpuset);Affinity Strategies
| Strategy | Binding | Best For |
|---|---|---|
| Compact | Pack threads onto fewest cores | Cache sharing, low-latency |
| Scatter | Spread across all sockets/cores | Memory bandwidth bound |
| Close | Adjacent cores, same socket | Balanced locality + bandwidth |
| Explicit | Manual core-to-thread mapping | Custom tuned for specific workload |
NUMA-Aware Placement
- 2-socket server: Socket 0 (cores 0-31), Socket 1 (cores 32-63).
- Memory attached to socket 0: Local to cores 0-31, remote to 32-63.
- Rule: Pin threads to same socket as their allocated memory.
- MPI + OpenMP hybrid: Rank 0 on socket 0 with 16 OMP threads on cores 0-15.
GPU Affinity
- Multi-GPU systems: Each GPU has a preferred NUMA node and PCIe topology.
- Pin training process to NUMA node closest to its GPU → minimize PCIe latency.
- CUDA: cudaSetDevice(gpu_id) + CPU affinity to matching NUMA node.
- Frameworks: PyTorch DataLoader workers should be pinned to same NUMA node as GPU.
Performance Impact
| Scenario | Without Affinity | With Affinity | Improvement |
|---|---|---|---|
| MPI latency benchmark | 2.1 µs | 1.4 µs | 33% |
| STREAM bandwidth | 180 GB/s | 240 GB/s | 33% |
| ML training throughput | 850 img/s | 1020 img/s | 20% |
| HPC CFD simulation | 45 min | 38 min | 16% |
Thread affinity is the first-order performance optimization for any multi-socket server workload — while modern OS schedulers are generally good, the cache and NUMA locality benefits of explicit thread pinning are too significant to leave on the table for HPC, ML training, and latency-critical serving, making affinity configuration a standard part of production deployment tuning.