GPU Virtualization
GPU Virtualization and Multi-Instance GPU (MIG) is the technology enabling a single physical GPU to be partitioned into multiple isolated instances — each with dedicated compute resources, memory, and memory bandwidth, allowing multiple users or workloads to share one GPU safely without interference, maximizing GPU utilization in cloud and enterprise environments.
Why GPU Virtualization?
- Many workloads don't need a full GPU: Inference serving, Jupyter notebooks, small training jobs.
- Without sharing: A user occupying an A100 at 10% utilization wastes 90% of a $10,000+ GPU.
- With MIG: Split one A100 into 7 isolated instances → 7 users, each with guaranteed resources.
NVIDIA MIG (Multi-Instance GPU)
- Available on: A100, A30, H100, H200 GPUs.
- Partitions GPU into up to 7 instances (on A100/H100).
- Each instance gets:
- Dedicated SM (streaming multiprocessor) slices.
- Dedicated memory and memory bandwidth.
- Dedicated L2 cache partition.
- Fault isolation (one instance's error doesn't crash others).
MIG Partition Profiles (A100 80GB)
| Profile | GPU Memory | SMs | Use Case |
|---|---|---|---|
| 1g.10gb | 10 GB | 14 SMs | Small inference |
| 2g.20gb | 20 GB | 28 SMs | Medium inference/training |
| 3g.40gb | 40 GB | 42 SMs | Large inference |
| 4g.40gb | 40 GB | 56 SMs | Medium training |
| 7g.80gb | 80 GB | 98 SMs | Full GPU (no partition) |
Other GPU Sharing Approaches
| Approach | Isolation | Overhead | Flexibility |
|---|---|---|---|
| MIG | Hardware-enforced | Near zero | Fixed profiles |
| vGPU (NVIDIA GRID) | Driver-level | 5-15% | Time-slicing |
| MPS (Multi-Process Service) | Software | Low | Concurrent kernels |
| Time-Slicing | Context switching | 10-30% | Any workload |
| Kubernetes GPU Sharing | Orchestration | Varies | Pod-level |
vGPU (Virtual GPU)
- NVIDIA GRID/vGPU: Hypervisor-based GPU virtualization.
- GPU time-sliced between VMs — each VM sees a virtual GPU.
- Used in: VDI (virtual desktops), cloud gaming, VMware/Citrix environments.
- Overhead: 5-15% per VM due to context switching.
MPS (Multi-Process Service)
- Allows multiple CUDA processes to share a single GPU simultaneously.
- Processes run concurrently (not time-sliced) — better utilization than context switching.
- No memory isolation — one process can potentially access another's memory.
- Used when: trusted workloads need to share GPU without MIG overhead.
Cloud GPU Sharing
- AWS: `p4d.24xlarge` with 8 A100s, or MIG-backed instances.
- GCP: Multi-instance GPU support for A100/H100.
- Azure: MIG available on ND-series VMs.
GPU virtualization is essential for economic GPU utilization in data centers — without partitioning and sharing, the high cost of modern GPU accelerators would be wasted on workloads that use only a fraction of available compute and memory resources.