Cache Coherence Protocol

Cache Coherence Protocols are the hardware mechanisms that maintain a consistent view of shared memory across multiple processor caches — ensuring that when one core writes to a memory location, all other cores see the updated value rather than stale cached copies, which is the fundamental requirement for correct shared-memory parallel programming and the source of significant performance overhead in multi-core and multi-socket systems.

The Coherence Problem

Without coherence, Core 0 could write X=5 to its L1 cache while Core 1 still reads the old value X=0 from its L1 cache — violating program semantics. Coherence protocols ensure that the memory system behaves as if there is a single shared memory, even though data is physically replicated across multiple private caches.

MESI Protocol (Baseline)

Each cache line is in one of four states:
- Modified (M): This cache has the only copy, and it has been written (dirty). Memory is stale.
- Exclusive (E): This cache has the only copy, and it matches memory (clean). Can transition to M without bus traffic.
- Shared (S): Multiple caches may hold clean copies. Writes require invalidating other copies first.
- Invalid (I): Cache line is not present or has been invalidated. Access requires fetching from memory or another cache.

MOESI Extension

Adds Owned (O) state: This cache has a modified copy AND other caches have Shared copies. The Owned cache is responsible for supplying data on requests (not memory). Avoids writing dirty data back to memory when sharing — reduces memory bandwidth. Used by AMD processors.

Coherence Implementation

• Snooping (Bus-Based): Every cache monitors (snoops) the shared bus. When a core requests a line, all other caches check their tags simultaneously. Fast for small core counts (2-8) but does not scale — bus bandwidth limits the number of snooping caches.

• Directory-Based: A central directory (distributed across memory controllers) tracks which caches hold each line. On a write, the directory sends invalidation messages only to caches that hold the line. Scales to hundreds of cores (used in NUMA systems and large multi-socket servers). Higher latency than snooping (requires directory lookup) but avoids broadcast.

• Hybrid: Modern processors (Intel, AMD) use snooping within a small cluster (4-8 cores sharing an L2/L3) and directory-based coherence between clusters and sockets.

Performance Impact

• False Sharing: Two cores access different variables that happen to occupy the same 64-byte cache line. Each write invalidates the other core's copy, causing cache line bouncing at hundreds of cycles per ping-pong — devastating performance. Fix: pad data structures to ensure per-core data occupies separate cache lines.
• Coherence Traffic: In a 64-core system, coherence traffic can consume 30-50% of the memory system's bandwidth. Protocols with Shared→Modified transition optimization (silent upgrades) and selective invalidation reduce overhead.

Cache Coherence is the invisible hardware protocol that makes shared-memory programming possible — maintaining the illusion of a single coherent memory while physically distributing data across dozens of private caches, at a performance cost that programmers must understand to write efficient parallel software.