Memory Architecture Design
On-Chip Memory Architecture is the design discipline that organizes the hierarchy of registers, SRAM caches, and embedded memories within a processor or SoC — where memory access latency and bandwidth determine 50-80% of overall chip performance, making the capacity, organization, and placement of on-chip memory the most impactful architectural decision after the compute pipeline itself.
The Memory Hierarchy
| Level | Size | Latency | Bandwidth | Technology |
|---|---|---|---|---|
| Register File | 1-32 KB | 1 cycle | ~TB/s | Custom flip-flops |
| L1 Cache (I/D) | 32-64 KB | 3-5 cycles | 200+ GB/s per core | 6T/8T SRAM |
| L2 Cache | 256 KB-2 MB | 10-20 cycles | 100+ GB/s | 6T/8T SRAM |
| L3 Cache (LLC) | 4-256 MB | 30-60 cycles | 50-200 GB/s | SRAM or eDRAM |
| HBM/DDR (off-chip) | 16-192 GB | 100-300 cycles | 50-8000 GB/s | DRAM |
SRAM Bitcell Design
- 6T SRAM: Standard bitcell with 6 transistors — two cross-coupled inverters for storage, two access transistors gated by the word line. Provides single-cycle read/write. Area: 0.020-0.030 μm² at 5nm node.
- 8T SRAM: Adds a separate read port (2 transistors) to eliminate read disturb, improving read stability at low voltage. Enables operation at lower Vdd (0.5-0.6V) for power savings.
- Bitcell vs. Periphery Area: At advanced nodes, SRAM bitcell area stops scaling (limited by read/write stability margins), while periphery circuits (sense amplifiers, drivers, address decoders) contribute 30-50% of total memory area. Assist circuits (write-assist negative bitline voltage, read-assist positive word line underdrive) enable bitcell scaling at the cost of peripheral complexity.
Cache Organization Architecture
- Associativity: Higher associativity (8-way, 16-way) reduces conflict misses but increases tag comparison logic, area, and access latency. L1 caches typically use 4-8 way; L3 caches use 8-16 way.
- Line Size: 64 bytes is standard. Larger lines improve spatial locality exploitation but waste bandwidth on sparse access patterns.
- Replacement Policy: LRU (Least Recently Used) approximations (pseudo-LRU, RRIP — Re-Reference Interval Prediction) balance hit rate against hardware complexity.
- Inclusive vs. Exclusive: Inclusive L3 guarantees that L3 contains a superset of L1/L2 data (simplifies coherence). Exclusive L3 maximizes effective capacity (L1+L2+L3) but complicates coherence protocol.
Embedded Memory Compilers
Compilers (tools from ARM, Synopsys, foundry PDKs) generate optimized SRAM/ROM instances from parameterized specifications (word count, bit width, ports, muxing ratio). The compiler produces the layout (GDS), timing model (.lib), netlist, and verification views — enabling rapid integration of custom memory blocks into SoC designs.
On-Chip Memory Architecture is the performance multiplier that determines whether a chip's compute units are fed or starved — because even the most powerful ALU is useless if it spends 90% of its cycles waiting for data from a memory hierarchy that was designed with insufficient capacity, bandwidth, or proximity.