Cache Coherence Hardware Design
Cache Coherence Protocol Hardware Design is the digital logic implementation of the snooping or directory-based protocols that maintain memory consistency across multiple processor cores' private caches — where the coherence controller in each cache must track line states (MESI/MOESI), process snoop requests from other cores, generate invalidations, handle data forwarding, and manage race conditions, all within the tight latency budget of 1-3 clock cycles to avoid becoming the critical path in multi-core processor performance.
Cache Controller State Machine
Each cache line has a coherence state tag (2-3 bits) managed by a state machine that responds to local processor requests (load, store) and external snoop requests (other cores' reads/writes):
MESI State Transitions (simplified):
- I → E: Local read miss, no other cache has the line. Fetch from memory. Exclusive — can silently upgrade to M on write.
- I → S: Local read miss, another cache has the line in S or E. Fetch from memory or peer cache. Shared.
- S → I: External write detected (snoop invalidation). Another core is writing — invalidate local copy.
- S → M: Local write hit. Must send invalidation to all sharers before writing. This is the critical "upgrade" transaction.
- M → I: External read detected. Must write back dirty data to memory and transition to I (or transition to S if using MOESI/O state).
- E → M: Local write hit. Silent upgrade — no bus transaction needed (only this cache has the line). This optimization is why E state exists.
Snoop Filter / Directory Design
For systems with >8 cores, broadcasting snoops to all caches wastes bandwidth. Solutions:
- Snoop Filter: A structure at the shared cache (L3) or interconnect that tracks which L2/L1 caches hold each line. Snoops are sent only to caches that actually hold the line. Inclusive L3 naturally serves as a snoop filter — every line in L1/L2 is also in L3.
- Directory: Distributed or centralized structure storing a bit-vector per cache line indicating which caches have a copy. Enables point-to-point invalidation instead of broadcast. Essential for NUMA systems and multi-socket servers.
- Scalability: Directory storage = cache_lines × core_count bits. For a 64 MB L3 with 128 cores at 64-byte lines: 1M lines × 128 bits = 16 MB of directory — significant overhead. Coarse-grained directories (per-cluster instead of per-core) reduce storage at the cost of precision.
Race Condition Handling
Coherence races occur when multiple cores simultaneously access the same line:
- Write-Write Race: Core A and Core B both try to write line X in S state. Both send invalidation requests. The arbiter serializes: one wins, the other retries. The loser's invalidation is NACKed or queued.
- Read-Write Race: Core A reads while Core B writes. If A's snoop arrives at B before B's write completes, B must stall or forward the old data. Ordering is determined by the point of serialization (L3 slice or home agent).
- Intervention: When Core A reads a line held in M state by Core B, Core B must "intervene" — forwarding the dirty data directly to A (and to memory) without waiting for memory to respond. This cache-to-cache transfer takes 40-80 ns, much faster than memory access.
Performance Impact
Coherence traffic directly affects multi-core scalability. False sharing (two variables on the same cache line written by different cores) causes the line to bounce between caches — potentially 100× performance degradation. Coherence protocol optimizations (silent evictions, speculative forwarding, merged writebacks) are critical for server-class processors.
Cache Coherence Protocol Hardware is the invisible arbiter that makes shared-memory multiprocessing possible — the distributed state machine that ensures every core sees a consistent view of memory, at a performance cost that determines whether adding more cores actually improves throughput.