Fat-Tree Topology

Fat-tree topology is the network architecture with increasing uplink capacity toward the core to maintain high aggregate throughput - it is commonly used in HPC and AI clusters where many nodes require simultaneous high-bandwidth communication.

What Is Fat-tree topology?

• Definition: Hierarchical switched fabric where higher tree levels are provisioned with wider bandwidth links.
• Design Goal: Prevent core bottlenecks and support near non-blocking communication patterns.
• AI Cluster Fit: Works well for collective-heavy workloads needing strong all-to-all communication behavior.
• Deployment Variables: Oversubscription ratio, switch radix, cable plan, and expansion strategy.

Why Fat-tree topology Matters

• High Throughput: Sufficient core capacity preserves performance under concurrent multi-job traffic.
• Predictable Latency: Balanced tree design reduces congestion hot spots and queueing spikes.
• Scalability: Supports structured growth while retaining known performance properties.
• Collective Performance: Strong bisection capacity benefits all-reduce and parameter exchange phases.
• Operational Visibility: Hierarchical layout simplifies monitoring and fault-domain isolation.

How It Is Used in Practice

• Capacity Planning: Size spine and aggregation links for expected worst-case east-west traffic.
• Oversubscription Policy: Set target oversubscription ratio based on workload sensitivity and budget.
• Validation: Benchmark bisection and collective behavior after deployment and each expansion phase.

Fat-tree topology is a proven network pattern for communication-intensive AI infrastructure - adequate uplink width at higher tiers is essential to avoid hidden scaling bottlenecks.