skills.rest
snoodleboot-io
snoodleboot-ioOFFICIAL
Explore all skills by this author →
GitHub

hyperparameter-optimization

Guides hyperparameter search strategy, cross-validation design, and pruning for machine learning models.

Updated Dec 29, 2025
Search Tags:#hyperparameter tuning#model selection#cross-validation#bayesian optimization#scikit-learn#optuna#random search
One-click install
npx skills add https://github.com/snoodleboot-io/discrecontinual_equations --skill hyperparameter-optimization-snoodleboot-io
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hyperparameter-optimization
Source: https://github.com/snoodleboot-io/discrecontinual_equations/tree/main/.claude/skills/hyperparameter-optimization
Command: npx skills add https://github.com/snoodleboot-io/discrecontinual_equations --skill hyperparameter-optimization-snoodleboot-io

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Tuning machine learning models often wastes compute on grid searches, leaks validation data through preprocessing, and reports optimistically biased scores. This Skill provides concrete patterns for choosing search strategies, defining search spaces, and obtaining honest generalization estimates. ## Core Features & Use Cases - Search Strategy Selection: Compares grid, random, successive halving, TPE/Bayesian, and evolutionary search with budget guidance for each. - Honest Evaluation: Implements nested cross-validation and once-touched test sets so reported scores reflect true generalization rather than selection bias. - Early Stopping and Pruning: Shows Optuna pruners and scikit-learn HalvingRandomSearchCV to cut wasted compute on poor configurations. - Use Case: When tuning an XGBoost classifier, use this Skill to define a log-scaled search space, run TPE with median pruning, and report a nested-CV score instead of the biased best_score_. ## Quick Start Ask the AI to design an Optuna-based hyperparameter search with nested cross-validation for your scikit-learn model.

Frequently Asked Questions about hyperparameter-optimization

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I choose between grid search, random search, and Bayesian optimization?▼

Random search is the default for unknown spaces with cheap fits, needing roughly 30-100 trials. Use TPE or Bayesian methods when fits cost minutes and you have few parameters, and reserve grid search for two or fewer discrete parameters.

How do I tune hyperparameters with Optuna and cross-validation?▼

Define an objective function that samples parameters with trial.suggest_float and trial.suggest_int, then score with cross_val_score inside the objective. Use log=True for parameters spanning orders of magnitude like learning rate and regularization strength.

Why is best_score_ from RandomizedSearchCV biased?▼

best_score_ is the maximum over many noisy validation estimates, so it includes selection luck and overstates true performance. Report a nested cross-validation score or a test set touched exactly once instead.

When should I use successive halving or pruning for hyperparameter search?▼

Use pruning or successive halving when fits are iterative or expensive, such as neural network training or large datasets. MedianPruner in Optuna or HalvingRandomSearchCV in scikit-learn kill underperforming trials early, but may discard configs that improve late.

Does preprocessing need to be inside the cross-validation pipeline?▼

Yes. Fitting a StandardScaler or feature selector on the full dataset before CV leaks validation-fold statistics into training. Place every fitted transform inside a scikit-learn Pipeline and pass the pipeline to the search.

Related Skills

Explore More Data & Analytics Skills→
obinopaulobinopaul
2

ml-pipeline

Develop and optimize scikit-learn machine learning pipelines for model training, evaluation, and selection.

Community
Intermediate
jsdfhasuhjsdfhasuh

fine-tuning-expert

Guide end-to-end LLM fine-tuning with dataset preparation, PEFT selection, and evaluation.

Community
Advanced
thistleknotthistleknot

mlflow

Persist MLflow experiment parameters, metrics, artifacts, and nested run lineage.

Community
Intermediate
seb1nseb1n
147

hyperparameter-tuning

Optimize machine learning model hyperparameters using grid search, random search, and Bayesian optimization.

Community
Advanced