1,016 technical terms and definitions
tensor core matrix multiply, wmma api cuda, tensor core mixed precision, tensor core performance optimization
GPU Tensor Core Programming is the technique of leveraging specialized matrix-multiply-and-accumulate hardware units in modern GPUs to achieve dramatic speedups for linear algebra…
constant memory, read only cache, cuda texture, cuda cache
GPU Texture and Constant Memory are specialized GPU memory spaces with dedicated caches — optimized for specific access patterns that offer higher effective bandwidth than global…
gpu boost clock, thermal design power, gpu temperature, tdp throttle, gpu power limit
GPU Thermal Throttling and Power Management is the hardware and firmware mechanism that dynamically reduces GPU clock frequency and voltage when the chip temperature or power…
optimization
GPU utilization measures the percentage of a GPU's computational resources that are actively being used at any given moment. In the context of AI and LLM workloads, achieving high…
mig multi instance, gpu sharing, vgpu, gpu partitioning
GPU Virtualization and Multi-Instance GPU (MIG) is the technology enabling a single physical GPU to be partitioned into multiple isolated instances — each with dedicated compute…
difference between gpu and cpu, gpu vs cpu chip, what is the difference between a cpu and gpu, cpu vs gpu explained, why gpu for ai
A CPU and a GPU are both chips built by the same lithography and fabrication process, but they are laid out around opposite bets on what kind of work matters most: a CPU devotes…
divergence, mitigation, branching
GPU Warp Divergence Mitigation is a critical CUDA optimization technique addressing the performance penalty incurred when different threads in the same warp execute different code…
sm streaming multiprocessor, cuda core execution, register file gpu, warp scheduler hardware design
GPU Streaming Multiprocessor (SM) Architecture is the fundamental execution unit of GPU chips, containing dozens of CUDA cores, tensor cores, warp schedulers, and hierarchical…
thread divergence, simt divergence, branch divergence gpu, warp efficiency
GPU Warp Divergence is the performance penalty that occurs when threads within the same warp (typically 32 threads executing in lockstep) take different paths at a branch…
branch divergence simt, thread divergence penalty, predication gpu, warp execution efficiency
GPU Warp Divergence is the performance penalty that occurs when threads within the same warp (NVIDIA, 32 threads) or wavefront (AMD, 64 threads) take different execution paths…
thread divergence cuda, branch divergence penalty, predicated execution gpu, control flow efficiency
GPU Warp Divergence is the performance degradation that occurs when threads within a warp take different execution paths at a branch — forcing the hardware to serialize both paths…
simt execution, warp divergence
GPU Warp Scheduling — the mechanism by which a GPU's streaming multiprocessor (SM) manages and interleaves execution of warps (groups of 32 threads) to hide memory latency. SIMT…
warp divergence, cuda thread branching, simt single instruction multiple thread, warp execution
GPU Warp Scheduling and Divergence represents the critical, uncompromising hardware execution mechanic within NVIDIA GPUs where 32 loosely independent software threads are…
warp scheduler hardware, instruction level parallelism gpu, dual issue gpu, warp stall reason
GPU Warp Scheduling is the hardware mechanism that selects which ready warp to execute each clock cycle on a Streaming Multiprocessor (SM) — where the warp scheduler's ability…
warp execution model cuda, thread divergence penalty, warp scheduler hardware, simt divergence handling
GPU Warp Scheduling and Divergence is the hardware mechanism by which a GPU streaming multiprocessor (SM) selects warps of 32 threads for execution each cycle and handles…
simt warp divergence, warp occupancy optimization, gpu thread scheduling, streaming multiprocessor warps
GPU Warp Scheduling and Execution Model — GPU architectures organize threads into warps (typically 32 threads) that execute instructions in lockstep using the Single Instruction…
gpudirect rdma gdr, gpudirect storage gds, gpu direct peer access, gpudirect async
GPUDirect Technology is NVIDIA's suite of technologies that enable direct data paths between GPUs and other system components (other GPUs, network adapters, storage) — bypassing…
gqa, general question answering, evaluation
GQA (General Question Answering) is a dataset for compositional visual reasoning — focusing on real-world images but using procedurally generated questions to rigorously test…
reliability
Graceful Degradation is the system design principle ensuring that applications maintain core functionality when components fail, resources become constrained, or dependencies…
optimization
Graceful Degradation is a resilience strategy that serves reduced functionality when full capability is unavailable - It is a core method in modern semiconductor AI serving…
graceful degradation fallback, graceful degradation, degradation fallback
Graceful Degradation Fallback Introduction & Motivation Graceful Degradation Fallback addresses a central problem in cloud process control and degradation-mode-service-quality…
graph neural networks
Graclus Pooling is a fast graph-clustering based pooling method for multilevel graph coarsening. - It greedily matches nodes to form compact clusters used in graph CNN…
Grad-CAM Class Activation Mapping Introduction & Motivation Grad-CAM Class Activation Mapping addresses a central problem in interpretability and explanation of xai predictions…
grad-cam class activation mapping prediction, grad-cam class activation, class activation mapping, activation mapping prediction, grad-cam class, class activation
Grad-CAM Class Activation Mapping Prediction with Machine Learning Introduction & Motivation Grad-CAM Class Activation Mapping Prediction with Machine Learning addresses a central…
Grad-CAM Class Activation Mapping (Extended Study) Introduction & Motivation Grad-CAM Class Activation Mapping (Extended Study) addresses a central problem in interpretability…
Grad-CAM Class Activation Mapping (Extended Study) Introduction & Motivation Grad-CAM Class Activation Mapping (Extended Study) addresses a central problem in interpretability…
Grad-CAM Class Activation Mapping (Extended Study) Introduction & Motivation Grad-CAM Class Activation Mapping (Extended Study) addresses a central problem in interpretability…
Grad-CAM++ for Multi-Instance Visual Feature Attribution Introduction & Motivation Grad-CAM++ for Multi-Instance Visual Feature Attribution addresses a central problem…
Grad-CAM Gradient-Weighted Class Activation Mapping Introduction & Motivation Grad-CAM Gradient-Weighted Class Activation Mapping addresses a central problem in explainability…
explainable ai
Grad-CAM (Gradient-weighted Class Activation Mapping) is a visual explanation technique that produces a coarse localization map highlighting the important regions in an image…
explainable ai
Grad-CAM++ is an improved version of Grad-CAM that uses higher-order gradients (second and third derivatives) — providing better localization for multiple instances of the same…
interpretability
GradCAM is a class-discriminative localization method using gradients of target outputs over feature maps - It identifies image regions most associated with model class…
backprop, backward pass
Gradients and Backpropagation What is Backpropagation? Backpropagation computes gradients of the loss with respect to each parameter, enabling gradient-based optimization.…
compression, distributed, training, communication
Gradient Compression Distributed Training is a technique reducing communication volume during distributed training by compressing gradient updates before transmission, minimizing…
effective batch
Gradient Accumulation What is Gradient Accumulation? Accumulate gradients over multiple mini-batches before updating weights, simulating a larger batch size without requiring more…
microbatch
Gradient accumulation simulates larger batch sizes by summing gradients over multiple forward/backward passes (micro-batches) before performing a single optimizer step, enabling…
model training
Gradient accumulation simulates larger batch sizes by accumulating gradients over multiple forward-backward passes before updating. How it works: Run forward and backward multiple…
large batch, vit training
Gradient Accumulation is a critical memory optimization technique universally employed in large-scale Vision Transformer and LLM training that mathematically simulates the effect…
effective batch size, gradient accumulation steps, large batch training, memory efficient training, micro-batch training
Gradient checkpointing and gradient accumulation are the two techniques that let you train a model that does not fit in memory. They attack different halves of the training memory…
micro-batching, effective batch size, memory efficient training, large batch simulation
Gradient Accumulation and Micro-Batching is a training technique that simulates large effective batch sizes by accumulating gradients across multiple small forward/backward passes…
large batch training, distributed gradient synchronization, effective batch size, memory efficient training
Gradient Accumulation and Large Batch Training — Scaling Optimization Beyond Memory Limits Gradient accumulation enables training with effectively large batch sizes…
gradient accumulation effective batch size and memory efficiency, gradient accumulation effective, accumulation effective batch, effective batch size, batch size memory
Gradient Accumulation Effective Batch Size and Memory Efficiency Introduction & Motivation Gradient Accumulation Effective Batch Size and Memory Efficiency addresses a central…
gradient accumulation hardware and synchronization overhead, gradient accumulation hardware, accumulation hardware synchronization, hardware synchronization overhead
Gradient Accumulation Hardware and Synchronization Overhead Introduction & Motivation Gradient Accumulation Hardware and Synchronization Overhead addresses a central problem…
gradient accumulation large batch, gradient accumulation large, accumulation large batch, gradient accumulation, accumulation large, large batch
Gradient Accumulation Large Batch Introduction & Motivation Gradient Accumulation Large Batch addresses a central problem in systems process control…
optimization
Gradient checkpointing and gradient accumulation are the two techniques that let you train a model that does not fit in memory. They attack different halves of the training memory…
gradient accumulation and large batch training, gradient accumulation large, accumulation large batch, large batch training, gradient accumulation, accumulation large, large batch
Gradient Accumulation and Large Batch Training Introduction & Motivation Gradient Accumulation and Large Batch Training addresses a central problem in ml-training process control…
micro batch accumulation, memory efficient training, gradient accumulation steps, effective batch size
Gradient Accumulation is the training technique that simulates large batch sizes by accumulating gradients over multiple forward-backward passes (micro-batches) before performing…
Gradient Accumulation for Large Effective Batch Sizes Introduction & Motivation Gradient Accumulation for Large Effective Batch Sizes addresses a central problem in optimization…
Gradient-Based Architecture Search with Gumbel Softmax Introduction & Motivation Gradient-Based Architecture Search with Gumbel Softmax addresses a central problem in automated…
Gradient-Based Architecture Search with Gumbel Softmax (Extended Study) Introduction & Motivation Gradient-Based Architecture Search with Gumbel Softmax (Extended Study) addresses…