27,049 technical terms and definitions
shared memory gpu tiling, global memory coalescing, texture cache gpu, register spilling gpu performance
GPU Memory Hierarchy Optimization — GPU performance is fundamentally constrained by memory bandwidth and latency, making effective utilization of the multi-level memory hierarchy…
shared memory cuda, l1 l2 cache gpu, memory bandwidth optimization, global memory access patterns
GPU Memory Hierarchy Optimization is the practice of strategically utilizing the multi-level memory system of modern GPUs — from fast but small shared memory and L1 cache (20…
cuda memory types, gpu cache optimization, shared memory optimization, gpu memory bandwidth
GPU Memory Hierarchy Optimization is the systematic tuning of data placement and access patterns across GPU's multi-level memory system to maximize bandwidth utilization…
unified memory, cuda memory, device memory
GPU Memory Management — understanding the GPU memory hierarchy and managing data transfers between host (CPU) and device (GPU) memory to avoid bottlenecks that dominate…
unified virtual memory, cuda managed memory, gpu memory allocation, pinned memory transfer
GPU Memory Management is the system-level discipline that governs how data is allocated, transferred, and accessed across the discrete address spaces of CPU (host) and GPU…
unified memory cuda, pinned memory allocation, cuda memory types, gpu memory optimization
GPU Memory Management is the systematic allocation, transfer, and optimization of data across CPU and GPU memory spaces to maximize performance and minimize overhead — where…
virtual memory gpu, cuda managed memory, gpu page fault, memory oversubscription gpu
GPU Virtual Memory and Memory Management is the system software and hardware infrastructure that provides address translation, demand paging, and memory protection for GPU…
unified virtual addressing, gpu page fault, gpu memory oversubscription
GPU Virtual Memory Management is the system of hardware and software mechanisms that provide GPUs with virtual address spaces, demand paging, memory oversubscription, and unified…
memory allocator gpu, cuda memory pool, caching allocator, pytorch memory
GPU Memory Pool Allocators are the caching memory management systems that maintain pre-allocated pools of GPU memory to eliminate the overhead of frequent cudaMalloc/cudaFree…
gpu memory pressure and host-device swapping overhead, gpu memory pressure, memory pressure host-device, pressure host-device swapping, host-device swapping overhead, gpu memory
GPU Memory Pressure and Host-Device Swapping Overhead Introduction & Motivation GPU Memory Pressure and Host-Device Swapping Overhead addresses a central problem in heterogeneous…
optimization
GPU memory utilization is the fraction of available accelerator memory actively consumed by model state, activations, and runtime buffers - it guides batch sizing and memory…
multi process service, cuda mps, gpu sharing processes, mps nvidia
GPU Multi-Process Service (MPS) is the NVIDIA runtime service that enables multiple CUDA processes to share a single GPU concurrently with improved efficiency — replacing…
nvidia mig partitioning, gpu isolation mig slices, mig compute instance profile, a100 mig configuration gpu
GPU Multi-Instance GPU (MIG) is a hardware partitioning feature introduced with NVIDIA's A100 (Ampere) architecture that divides a single physical GPU into up to seven independent…
Service MPS, sharing, virtualization
GPU Multi-Process Service MPS is an NVIDIA GPU feature enabling multiple CPU processes to concurrently utilize GPU resources through time-slicing and context management — enabling…
gpu sharing, gpu virtualization multi user, time slicing gpu
GPU Multi-Tenancy is the sharing of a single physical GPU among multiple applications, users, or virtual machines, providing isolation, fairness, and efficient utilization…
nvswitch all to all, nvlink 4.0 bandwidth, nvlink c2c chip to chip, nvidia dgx h100 nvlink
NVLink and NVSwitch GPU Interconnects: High-Bandwidth All-to-All GPU Networking — specialized interconnect technology enabling 900 GB/s per-GPU communication for tightly-coupled…
occupancy calculator, warp occupancy, thread block size, sm utilization
GPU Occupancy Optimization is the process of maximizing the ratio of active warps to the maximum possible warps per Streaming Multiprocessor (SM) — achieved by carefully choosing…
sm occupancy, register pressure gpu, thread block size selection, occupancy calculator
GPU Occupancy Optimization is the performance tuning discipline that maximizes the number of active warps per Streaming Multiprocessor (SM) — measured as the ratio of active warps…
register pressure gpu, occupancy limiter, latency hiding gpu, active warps per sm
GPU Occupancy is the ratio of active warps on a Streaming Multiprocessor (SM) to the maximum number of warps the SM can support — a key performance metric that determines…
cuda occupancy calculator, register pressure gpu, shared memory occupancy, thread block sizing
GPU Occupancy Optimization is the practice of maximizing the number of active warps per Streaming Multiprocessor (SM) relative to the hardware maximum — balancing register usage…
out of memory, cuda error, gradient checkpointing, quantization, memory optimization, batch size
GPU Out of Memory (OOM) errors occur when model weights, activations, or intermediate computations exceed available GPU VRAM — a common issue in ML development that requires…
device plugin, nvidia
GPU Management in Kubernetes NVIDIA GPU Operator Automates GPU driver installation, container toolkit, and device plugins in Kubernetes. Installation Components Installed…
nvtx annotation, roofline model gpu, achieved bandwidth occupancy, gpu bottleneck analysis
GPU Performance Profiling encompasses systematic measurement and analysis of kernel execution, memory access patterns, and hardware utilization using Nsight tools, roofline…
Threads, pattern, kernel design
GPU Persistent Threads Pattern is an advanced GPU kernel design pattern where single kernel launch creates threads that persist across multiple iterations of input data processing…
thermal management, cooling efficiency
GPU Power and Thermal Management is a critical GPU system design and programming discipline ensuring that GPU power consumption and heat dissipation remain within system thermal…
dvfs gpu, gpu power limit, gpu frequency scaling, gpu thermal throttle
GPU Power Management and DVFS is the dynamic adjustment of GPU clock frequency and voltage to balance performance, power consumption, and thermal limits — where modern GPUs…
gpu energy efficiency, power capping, gpu tdp, thermal design power gpu
GPU Energy Efficiency and Power Management is the set of hardware and software mechanisms that dynamically control GPU power consumption to maximize performance within thermal…
Nsight, performance analysis, optimization
GPU Profiling Nsight Performance Analysis is a comprehensive GPU performance profiling and analysis toolkit enabling detailed measurement and visualization of kernel execution…
nsight compute profiling, nsight systems timeline, cuda profiling tools, gpu performance analysis
GPU Profiling and Debugging is the systematic analysis of GPU application performance and correctness using specialized tools that provide detailed metrics, timeline…
kernel performance profiling, gpu occupancy analysis, memory throughput profiling, instruction throughput analysis
GPU Profiling with Nsight Compute is the systematic analysis of GPU kernel performance characteristics — including compute throughput, memory throughput, occupancy, stall reasons…
nsight profiler, occupancy analysis, kernel optimization gpu, performance bottleneck gpu
GPU Performance Profiling and Optimization is the systematic analysis methodology that identifies and eliminates performance bottlenecks in GPU kernels — using hardware…
cuda thread block, warp execution, thread hierarchy gpu, cooperative groups
GPU Programming Model and Thread Hierarchy is the software abstraction that organizes millions of GPU threads into a hierarchical structure — grids of thread blocks (each…
cuda ray tracing optimization, optix ray tracing, gpu graphics compute, rtx ray tracing performance
GPU Ray Tracing and Graphics Compute is the parallel implementation of ray tracing algorithms and graphics computations on GPUs — where dedicated RT Cores on NVIDIA RTX GPUs…
rt core bvh traversal, hardware ray triangle intersection, real time ray tracing, bvh acceleration structure
GPU Hardware Ray Tracing is the dedicated fixed-function hardware (NVIDIA RT Cores, AMD Ray Accelerators, Intel Ray Tracing Units) that accelerates the computationally intensive…
parallel reduction cuda, warp reduction optimization, cuda reduce performance, hierarchical reduction gpu
GPU Reduction Patterns are the parallel algorithms for combining array elements into single value through associative operations — where hierarchical reduction using warp…
optimization, allocation, spilling
GPU Register File Optimization is a critical low-level GPU optimization technique managing allocation and utilization of per-thread register storage — preventing register spilling…
register spilling local memory, occupancy register tradeoff, ptx register allocation, kernel register count
GPU Register Optimization addresses the critical trade-off between register availability for instruction-level parallelism and kernel occupancy, directly impacting throughput…
register allocation gpu, register spill, occupancy register, gpu register file
GPU Register Pressure is the conflict between a kernel's per-thread register demand and the GPU's fixed register file capacity — where each additional register per thread reduces…
parallel scan cuda, cuda prefix sum optimization, inclusive exclusive scan, scan algorithm gpu
GPU Scan (Prefix Sum) is the parallel algorithm that computes cumulative sums or other associative operations across array elements — where inclusive scan produces [a0, a0+a1…
heterogeneous task scheduling, cpu gpu co-scheduling, device affinity
Heterogeneous Task Scheduling is the algorithmic and runtime discipline of assigning computational tasks to the most appropriate processing element — CPU cores, GPU compute units…
shared memory optimization, smem, scratchpad
GPU Shared Memory Optimization — using the fast, programmer-managed on-chip memory (shared memory / SMEM) within each GPU Streaming Multiprocessor (SM) to drastically reduce…
shared memory optimization, bank conflict resolution, shared memory access pattern
GPU Shared Memory Bank Conflicts are the performance penalties that occur when multiple threads in a warp simultaneously access different addresses that map to the same shared…
shared memory optimization cuda, bank conflict avoidance, shared memory padding, warp shared memory access
GPU Shared Memory Bank Conflicts represent the performance hazard that occurs when multiple threads within a warp simultaneously access different addresses mapped to the same…
shared memory bank conflict, shared memory tiling, scratchpad memory gpu, shared memory programming
GPU Shared Memory Optimization is the performance-critical programming technique that uses the GPU's fast, software-managed on-chip memory (shared memory / scratchpad) to cache…
shared memory bank conflict, tiling gpu kernel, shared memory usage cuda, local data share
GPU Shared Memory Optimization is the critical CUDA/GPU programming technique of using on-chip shared memory (32-228 KB per Streaming Multiprocessor) as a programmer-managed cache…
streaming multiprocessor, cuda core, gpu compute unit, sm design
GPU Streaming Multiprocessor (SM) Architecture is the fundamental compute building block of NVIDIA GPUs, where each SM contains a set of CUDA cores, warp schedulers, register…
streaming multiprocessor, warp occupancy, gpu occupancy tuning
GPU SM Occupancy Optimization is the tuning of GPU kernel resource usage (registers, shared memory, block size) to maximize the number of concurrent warps executing on each…
cuda radix sort, parallel sorting gpu, gpu sort performance, cuda sort optimization
GPU Sorting Algorithms are the parallel implementations of sorting that leverage thousands of GPU threads to achieve 100-300 GB/s throughput — where radix sort (optimal…
cuda sparse linear algebra, cusparse optimization, sparse matrix gpu performance, csr coo format gpu
GPU Sparse Matrix Operations are the specialized algorithms for matrices where most elements are zero, exploiting sparsity to reduce memory and computation — where Compressed…
gpu event synchronization, asynchronous gpu execution, cuda stream event ordering, gpu stream dependency management
GPU stream-event synchronization is the practice of expressing the minimum ordering relationships required among asynchronous GPU kernels, memory transfers, and host actions.…