1,016 technical terms and definitions
Gradient-Based Architecture Search with Gumbel Softmax (Extended Study) Introduction & Motivation Gradient-Based Architecture Search with Gumbel Softmax (Extended Study) addresses…
Implicit Differentiation for Gradient Hyperparameter Tuning Introduction & Motivation Implicit Differentiation for Gradient Hyperparameter Tuning addresses a central problem…
nlp
Gradient-Based Masking is a technique that selects tokens to mask based on their influence on the loss gradient — identifying tokens that are most critical for the model's current…
Gradient-Based Meta-Learning for RL Introduction & Motivation Gradient-Based Meta-Learning for RL addresses a central problem in metalearning learning and rapid task adaptation…
Gradient-Based Meta-Learning for RL (Extended Study) Introduction & Motivation Gradient-Based Meta-Learning for RL (Extended Study) addresses a central problem in metalearning…
Gradient-Based Meta-Learning for RL (Extended Study) Introduction & Motivation Gradient-Based Meta-Learning for RL (Extended Study) addresses a central problem in metalearning…
neural architecture
Gradient-Based NAS is a family of NAS methods that reformulate the architecture search as a continuous optimization problem — making architecture parameters differentiable…
fine-tuning
Gradient-Based Prompt Tuning is the parameter-efficient fine-tuning technique that prepends learnable continuous embedding vectors ("soft prompts") to the model input…
model optimization
Gradient-Based Pruning is a more principled pruning criterion — using gradient information (or second-order derivatives) to estimate the impact of removing a weight on the loss…
model optimization
Gradient-Based Pruning is pruning strategies that rank parameters using gradient-derived importance signals - It leverages optimization sensitivity to remove low-impact…
xgboost, lgbm
Gradient Boosting is an ensemble machine learning technique where models are built sequentially — each new model correcting the errors (residuals) of the previous one…
gradient boosting, ensemble learning iterative, ctr f1 gb gpu l1 l2, gradient boosting ensemble learning & iterative refinement, gradient boosting ensemble, boosting ensemble
Gradient Boosting: Ensemble Learning & Iterative Refinement Introduction & Motivation Gradient boosting constructs strong learners by sequentially training weak learners…
data analysis
Gradient Boosting for Defect Detection is the application of gradient boosted tree models (XGBoost, LightGBM, CatBoost) to identify and classify wafer defects — sequentially…
gradient boosting sequential residuals prediction, gradient boosting sequential, boosting sequential residuals, sequential residuals prediction, gradient boosting
Gradient Boosting Sequential Residuals Prediction with Machine Learning Introduction & Motivation Gradient Boosting Sequential Residuals Prediction with Machine Learning addresses…
Gradient Boosting Uncertainty Introduction & Motivation Gradient Boosting Uncertainty addresses a central problem in uncertainty methods and prediction intervals: how to Quantify…
distributed training
Gradient bucketing is the grouping of many small gradient tensors into larger communication chunks before collective operations - it improves network efficiency by reducing…
optimization
Gradient Centralization (GC) is a simple optimization technique that centralizes (zero-means) gradients before each update — subtracting the mean of the gradient vector from each…
activation recomputation, memory optimization training
Gradient checkpointing and gradient accumulation are the two techniques that let you train a model that does not fit in memory. They attack different halves of the training memory…
activation checkpointing, memory efficient training, recomputation training, checkpointing deep learning
Gradient checkpointing and gradient accumulation are the two techniques that let you train a model that does not fit in memory. They attack different halves of the training memory…
activation recomputation, memory efficient training, checkpoint segment, rematerialization
Gradient Checkpointing (Activation Recomputation) is the memory optimization technique for training deep neural networks that trades compute for memory by storing only a subset…
max norm, stability
Gradient clipping limits gradient magnitude during training to prevent exploding gradients, stabilizing optimization of deep networks and recurrent architectures. Methods: (1)…
model training
Gradient clipping caps gradient magnitude to prevent exploding gradients that destabilize training. The problem: Large gradients cause huge weight updates, loss spikes, or NaN…
training techniques
Gradient Clipping is operation that limits gradient magnitude to a fixed norm before optimization updates - It is a core method in modern semiconductor AI serving…
gradient explosion, clip grad norm
Gradient Clipping — a technique that limits the magnitude of gradients during backpropagation to prevent exploding gradients from destabilizing training. The Problem In deep…
clip gradient, global norm clipping, clip grad norm, exploding gradients
Gradient clipping limits gradient magnitude before an optimizer update to prevent rare extreme steps from destabilizing training. It is standard protection for recurrent networks…
Gradient Clipping Strategies Introduction & Motivation Gradient Clipping Strategies addresses a central problem in optimization algorithm tuning and improvement: how to Optimize…
communication
Gradient Compression is a distributed training optimization that reduces the communication overhead of synchronizing gradients across GPU workers — using quantization (reducing…
gradient sparsification, powersgd, topk gradients, communication compression
Gradient Compression is a distributed training optimization technique that reduces the communication volume of gradients — sending only the most important gradient information…
gradient compression communication-efficient prediction, gradient compression communication-efficient, compression communication-efficient prediction
Gradient Compression Communication-Efficient Prediction with Machine Learning Introduction & Motivation Gradient Compression Communication-Efficient Prediction with Machine…
privacy
Gradient Compression for Privacy is the use of gradient compression techniques (sparsification, quantization) to reduce privacy leakage in distributed training — by transmitting…
gradient compression and quantization for communication efficiency, gradient compression quantization, compression quantization communication, quantization communication efficiency
Gradient Compression and Quantization for Communication Efficiency Introduction & Motivation Gradient Compression and Quantization for Communication Efficiency addresses a central…
distributed training
Gradient compression techniques is the communication-reduction methods that lower distributed training bandwidth demand by encoding or sparsifying gradients - they reduce…
top k sparsification, gradient sparsity training, magnitude based pruning, sparse gradient communication
Gradient Compression Techniques are the family of methods that reduce gradient communication volume by transmitting only the most important gradient components — using…
gradient descent, descent optimizers, cifar glue lamb lars nlp sgd
Gradient Descent Optimizers Introduction & Motivation Optimization: update parameters to minimize loss. SGD, Adam, AdamW. Applications: training convergence, hyperparameter…
gradient descent variants batch mini batch stochastic, gradient descent, variants batch mini, bert cifar gd gpu lr mnist, gradient descent variants batch mini-batch & stochastic
Gradient Descent Variants: Batch, Mini-Batch & Stochastic Introduction & Motivation Gradient descent update policy: determines optimization trajectory. Batch GD: full dataset per…
Gradient Descent Algorithm Variants Introduction & Motivation Gradient Descent Algorithm Variants addresses a central problem in optimization algorithm tuning and improvement: how…
gem, continual learning
Gradient episodic memory is a continual-learning algorithm that constrains new-task gradients so they do not increase loss on stored past-task examples - Projected gradients…
computer vision
Gradient flow in deep ViTs is the mechanism that determines whether supervision signals can propagate across many transformer layers without vanishing or exploding - controlling…
model training
Gradient Flow Preservation is a design principle for pruning and sparse training — ensuring that removing weights does not disrupt the backpropagation signal, keeping gradient…
gradient flow stability in deep networks prediction, gradient flow stability, flow stability deep, stability deep networks, deep networks prediction, gradient flow
Gradient Flow Stability in Deep Networks Prediction with Machine Learning Introduction & Motivation Gradient Flow Stability in Deep Networks Prediction with Machine Learning…
Evolution Strategies for Gradient-Free Meta-Learning Introduction & Motivation Evolution Strategies for Gradient-Free Meta-Learning addresses a central problem in fewshot…
Nelder-Mead Simplex Direct Search Algorithm Introduction & Motivation Nelder-Mead Simplex Direct Search Algorithm addresses a central problem in optimization algorithm tuning…
Score-CAM Gradient-Free Visual Activation Mapping Introduction & Motivation Score-CAM Gradient-Free Visual Activation Mapping addresses a central problem in explainability…
ai safety
Gradient Masking is a phenomenon where a defense accidentally or intentionally makes the model's gradients uninformative — causing gradient-based attacks to fail while the model…
Gradient Masking Pitfalls Introduction & Motivation Gradient Masking Pitfalls addresses a central problem in robustness of advrobust models under adversarial conditions: how…
Gradient Masking Pitfalls (Extended Study) Introduction & Motivation Gradient Masking Pitfalls (Extended Study) addresses a central problem in robustness of advrobust models under…
Gradient Masking Pitfalls (Extended Study) Introduction & Motivation Gradient Masking Pitfalls (Extended Study) addresses a central problem in robustness of advrobust models under…
optimization
Gradient Noise is the deliberate addition of noise to gradient updates during training — typically Gaussian noise with decaying variance, which helps escape local minima, improves…
Gradient Norm Expectation for Deep Model Active Selection Introduction & Motivation Gradient Norm Expectation for Deep Model Active Selection addresses a central problem…
optimization
Gradient Normalization is the practice of normalizing gradient magnitudes during training — either by clipping the gradient norm to a maximum value (gradient clipping) or…