Network on Chip Noc
Network-on-Chip (NoC) Architecture is a scalable on-chip communication framework that replaces traditional bus-based interconnects with packet-switched networks, enabling efficient data movement in many-core and AI accelerator chips.
NoC Topology and Routing
- Mesh Topology: Regular 2D grid arrangement of routers (most common). Scales well to moderate core counts (~100s cores) with predictable performance.
- Torus Topology: Mesh with wrap-around connections on edges. Reduces diameter and improves bisection bandwidth compared to mesh.
- Ring Topology: Linear ordering of nodes. Lower area overhead but higher latency for distant cores.
- Routing Algorithms: XY routing (dimension-ordered), adaptive routing selects alternate paths based on congestion. Deadlock-free routing using virtual channels.
NoC Router Microarchitecture
- Input/Output Port Design: Each router port includes input buffers (FIFO), crossbar switch, and arbitration logic.
- Virtual Channels: Multiple independent channels per physical link prevent HOL (head-of-line) blocking and enable deadlock avoidance. Typically 4-8 VCs per port.
- Crossbar Switch: Handles simultaneous transfers between input and output ports. Area and power scale as O(n²) where n is radix.
- Arbiter Implementations: Round-robin, priority-based, or weighted arbitration for port conflicts. Critical for throughput and fairness.
Flow Control and QoS
- Wormhole Switching: Packet travels in flits. Low latency, low buffer overhead but entire packet remains in-flight during routing.
- Virtual Cut-Through: Buffers entire packet at intermediate nodes. Higher latency but enables better path optimization.
- QoS Mechanisms: Traffic class assignment, priority levels, bandwidth reservation for real-time tasks (critical for SoC interconnects).
Real-World Usage and Performance
- Many-Core CPUs: 64+ core designs require NoC for intra-cluster and inter-cluster communication.
- AI Accelerators: Tensor cores demand low-latency, high-bandwidth communication. TPU, Cerebras, and Graphcore use custom NoC designs.
- Typical Performance: 5-10 cycle latency per hop in modern implementations. Throughput limited by virtual channel bandwidth and arbitration efficiency.