Grid Search

Grid Search is a hyperparameter tuning technique that exhaustively evaluates all combinations of specified parameter values — testing every possibility to find optimal hyperparameters, simple but computationally expensive.

What Is Grid Search?

• Purpose: Find best hyperparameters for machine learning models.
• Method: Test every combination of parameter values.
• Cost: Exponential (10 parameters × 5 values = 9.7M combinations).
• Completeness: Guaranteed to find best in search space.
• Speed: Slow for large spaces, fast for small spaces.

Why Grid Search Matters

• Simple: Easy to understand and implement.
• Guaranteed: Will find best in defined space.
• Interpretable: Results show how each parameter affects performance.
• Baseline: Good starting point before advanced methods.
• Parallelizable: Run combinations simultaneously.

Grid Search vs Alternatives

Grid Search: Exhaustive, guaranteed optimal, expensive.
Random Search: Sample randomly, faster, may miss optimal.
Bayesian Optimization (Hyperopt): Intelligent sampling, 10-100× faster.
Evolutionary Algorithms: Population-based, good for large spaces.

Quick Example

from sklearn.model_selection import GridSearchCV
from sklearn.ensemble import RandomForest

param_grid = {
    'n_estimators': [100, 200, 500],
    'max_depth': [5, 10, 20],
    'min_samples_split': [2, 5, 10]
}

grid = GridSearchCV(
    RandomForest(),
    param_grid,
    cv=5,
    n_jobs=-1
)

grid.fit(X_train, y_train)
print(grid.best_params_)

Best Practices

• Define reasonable parameter ranges first
• Use cross-validation (prevent overfitting)
• Parallelize with n_jobs=-1
• For large spaces, use Random or Bayesian instead
• Use GridSearchCV from sklearn (not manual loops)

Grid Search is the foundational hyperparameter tuning method — exhaustive, simple, guaranteed optimal but computationally expensive for large spaces.