GPU Power Management
GPU Power Management and DVFS is the dynamic adjustment of GPU clock frequency and voltage to balance performance, power consumption, and thermal limits — where modern GPUs continuously modulate their operating point hundreds of times per second based on workload demand, power budget, and temperature, with the GPU's actual clock speed often differing significantly from its advertised "boost" frequency.
GPU Power States
| State | Frequency | Voltage | Power | Usage |
|---|---|---|---|---|
| Idle | 210 MHz | 0.65V | 10-30W | Desktop/idle |
| Light Load | 800-1200 MHz | 0.75V | 50-100W | Video, light compute |
| Base Clock | 1200-1800 MHz | 0.85V | 150-250W | Sustained all-core |
| Boost Clock | 1800-2500 MHz | 0.95-1.1V | 250-400W | Thermal/power headroom |
| Max Boost | 2500-3000 MHz | 1.05-1.1V | 400-700W | Transient, single SM |
DVFS on GPUs
- Dynamic Voltage and Frequency Scaling: GPU firmware continuously adjusts V and F.
- $P_{dynamic} \propto C \times V^2 \times F$ — reducing voltage provides quadratic power savings.
- GPU firmware reads: Temperature sensors, power sensors, workload monitors.
- Decision every: ~1 ms — adjusts clock speed in real time.
Power Limiting Mechanisms
1. TDP (Thermal Design Power): Maximum sustained power the cooling solution can handle.
- RTX 4090: 450W TDP. A100 SXM: 400W TDP. H100 SXM: 700W TDP.
2. Power Limit: Software-configurable cap. If GPU hits limit → reduce frequency.
- `nvidia-smi -pl 300` — set power limit to 300W.
3. Thermal Throttling: If junction temperature exceeds limit (83-95°C) → reduce clock.
4. Voltage Limit: Maximum safe voltage for the silicon → caps max boost.
Undervolting and Overclocking
- Undervolting: Reduce voltage at given frequency → less power, same performance.
- Risk: Instability if voltage too low for the specific silicon sample.
- Overclocking: Increase power limit + frequency offset.
- Diminishing returns: 10% more power → 3-5% more performance (voltage scaling).
Data Center GPU Power Management
- NVIDIA MIG Power Isolation: Each MIG instance has proportional power budget.
- Power Capping for TCO: Data centers cap GPU power at 70-80% of max → significantly reduces cooling cost with only 5-10% performance loss.
- nvidia-smi queries:
- `nvidia-smi --query-gpu=power.draw,clocks.gr,temperature.gpu --format=csv`
GPU Power Efficiency Trend
| Generation | Performance/Watt Improvement |
|---|---|
| Kepler → Maxwell | ~2x |
| Maxwell → Pascal | ~1.5x |
| Pascal → Turing | ~1.5x |
| Turing → Ampere | ~1.5x |
| Ampere → Hopper | ~2x (FP8 ops) |
GPU power management is the invisible governor of GPU performance — understanding how DVFS, power limits, and thermal throttling interact is essential for anyone benchmarking, deploying, or optimizing GPU workloads, as the actual sustained performance can be 20-30% below peak specifications.