Cache Coherence
Cache coherence is the hardware protocol that ensures every CPU core in a multi-core processor sees a consistent view of shared memory — guaranteeing that when one core writes a value, all other cores that read the same address see the updated value, not a stale copy from their own cache. Without coherence, multi-threaded programs would silently produce wrong results because different cores would disagree on the contents of memory. Every multi-core chip from a phone SoC to a 100+ core server processor implements a coherence protocol in hardware, consuming 10–30% of on-chip interconnect bandwidth.
Why coherence is necessary. Each core has private L1/L2 caches for speed. When core 0 writes address X to its L1, core 1's L1 might still hold the old value of X. Without coherence, core 1 reads stale data — a silent data corruption. The coherence protocol ensures either: (1) core 1's copy is invalidated before core 0 writes, or (2) core 0's write is propagated to core 1's cache. This must happen automatically in hardware (software-managed coherence is too slow and error-prone for general-purpose code).
The two dominant protocol families:
| Protocol | Mechanism | Scalability | Bandwidth | Used in |
|---|---|---|---|---|
| Snoopy (bus-based) | Every cache snoops a shared bus; broadcast invalidations | 2–8 cores | High (every write broadcasts) | Small multi-core (phone SoC, embedded) |
| Directory-based | A directory tracks which caches hold each line; point-to-point messages | 8–1000+ cores | Lower (unicast, not broadcast) | Server CPUs, GPU, AI SoC |
MESI — the classic snoopy protocol. Each cache line is in one of four states:
- M (Modified): This cache has the only valid copy; it's been written. Must write back before another cache can read.
- E (Exclusive): This cache has the only copy, but it matches memory. Can transition to M on write without bus traffic.
- S (Shared): Multiple caches may hold this line (read-only). Must invalidate others before writing.
- I (Invalid): Line is not in this cache (or has been invalidated).
Variants: MOESI (adds Owned state — dirty shared), MESIF (adds Forward — one cache supplies data to requesters instead of memory).
Directory-based coherence — scaling to many cores. A snoopy protocol broadcasts every invalidation to all caches — O(N) traffic per write. At 64+ cores this saturates the interconnect. Directory protocols maintain a bit-vector (or pointer list) per cache line in a distributed directory, recording which cores hold copies. Invalidation messages are sent only to the sharers — O(1) messages per write, scaling to hundreds of cores. AMD's MOESI/Probe-Filter and Intel's MESIF/snoop-filter are production directory variants.
Coherence overhead and performance impact:
- Latency: A write to a shared line requires invalidating remote copies (20–100 ns round-trip to remote cache/directory vs 1–4 ns for a local L1 hit).
- Bandwidth: Coherence messages (requests, invalidations, data responses) consume 10–30% of NoC/interconnect bandwidth.
- False sharing: When two cores write to different variables that happen to share a cache line (64 bytes), the line ping-pongs between cores — catastrophic performance. Padding data structures to cache-line boundaries avoids this.
- Scalability wall: Beyond ~100–200 cores with coherence, the overhead grows faster than the compute benefit — which is why GPUs use a simpler memory model (no hardware coherence across SMs, software-managed shared memory).
Coherence in AI chips — a design choice. AI accelerators face a coherence trade-off:
- CPUs (Grace, Zen): full hardware coherence — required for running OS, general-purpose code, pointer-rich data structures.
- GPUs (H100 SM clusters): L1 is per-SM non-coherent; L2 is shared and coherent within the GPU; cross-GPU coherence via NVLink (limited, software-managed).
- Custom AI ASICs (TPU, Cerebras): often no coherence at all — the dataflow is statically scheduled, so software knows where data lives and moves it explicitly via DMA. This eliminates coherence overhead entirely but requires compiler support.
Cache coherence and the CFS platform. Coherence is the hardware mechanism that makes multi-core programming possible — it's what lets two threads on different cores share a data structure without explicit message passing. The CFS NoC keyword covers the on-chip network that carries coherence messages. The memory-controller keyword covers the DRAM interface that coherence protocol ultimately reads from on a miss. The computer-architecture keyword covers the full system context where coherence operates.