Optuna Hyperparameter Optimization Cheat Sheet

Updated 2026-05-21

Next Topic: Physical AI and Robotics AI Cheat Sheet

Optuna is an automatic hyperparameter optimization framework for machine learning, built around a define-by-run API where search spaces are constructed dynamically using ordinary Python control flow. Unlike grid or random search, Optuna learns from previous trials to intelligently direct sampling toward promising regions, while simultaneously pruning unpromising trials to save compute. The mental model to keep in mind is that every optimization run is a Study composed of individual Trials, and the sampler + pruner pair you choose determines both exploration efficiency and resource utilization.

What This Cheat Sheet Covers

This topic spans 17 focused tables and 119 indexed concepts. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Core Study and Trial APITable 2: Define-by-Run Search SpacesTable 3: SamplersTable 4: TPESampler Key ParametersTable 5: PrunersTable 6: Storage BackendsTable 7: Distributed and Parallel OptimizationTable 8: Study Persistence and Warm-StartTable 9: Multi-Objective OptimizationTable 10: Ask-and-Tell InterfaceTable 11: Callbacks and TerminationTable 12: Framework Integrations (optuna-integration)Table 13: VisualizationTable 14: Hyperparameter Importance EvaluatorsTable 15: optuna-dashboard (Real-Time Web UI)Table 16: Logging and ReproducibilityTable 17: OptunaHub (Community Package Registry)

Table 1: Core Study and Trial API

The Study and Trial objects are the two central abstractions in Optuna. A Study coordinates the entire optimization campaign; each Trial represents a single evaluation of the objective function. Understanding the lifecycle of both — from creation through optimization to result retrieval — is the prerequisite for using every other Optuna feature.

Method	Example	Description
optuna.create_study()	`study = optuna.create_study(direction="minimize")`	Creates a new Study object; `direction` is `"minimize"` or `"maximize"`; without a `storage` argument uses in-memory storage.
study.optimize()	`study.optimize(objective, n_trials=100, timeout=600)`	Runs the optimization loop; accepts `n_trials`, `timeout` (seconds), `n_jobs`, `callbacks`, and `catch` (exception types to suppress).
trial.suggest_float()	`lr = trial.suggest_float("lr", 1e-5, 1e-1, log=True)`	Samples a float; `log=True` samples in log-space; `step` discretizes the range (mutually exclusive with `log`).
trial.suggest_int()	`depth = trial.suggest_int("depth", 2, 32, log=True)`	Samples an integer from `[low, high]`; `log=True` and `step` behave the same as in `suggest_float`.
trial.suggest_categorical()	`opt = trial.suggest_categorical("optimizer", ["adam", "sgd"])`	Samples one value from a list; supports `None`, `bool`, `int`, `float`, or `str` elements.
trial.report()	`trial.report(val_loss, step=epoch)`	Reports an intermediate objective value at a given step so pruners can evaluate the trial.
trial.should_prune()	`if trial.should_prune(): raise optuna.TrialPruned()`	Returns `True` if the active pruner decides this trial should be stopped early; must be called after `trial.report()`.

Table 1: Core Study and Trial API

Method	Example	Description
optuna.create_study()	`study = optuna.create_study(direction="minimize")`	Creates a new Study object; `direction` is `"minimize"` or `"maximize"`; without a `storage` argument uses in-memory storage.
study.optimize()	`study.optimize(objective, n_trials=100, timeout=600)`	Runs the optimization loop; accepts `n_trials`, `timeout` (seconds), `n_jobs`, `callbacks`, and `catch` (exception types to suppress).
trial.suggest_float()	`lr = trial.suggest_float("lr", 1e-5, 1e-1, log=True)`	Samples a float; `log=True` samples in log-space; `step` discretizes the range (mutually exclusive with `log`).
trial.suggest_int()	`depth = trial.suggest_int("depth", 2, 32, log=True)`	Samples an integer from `[low, high]`; `log=True` and `step` behave the same as in `suggest_float`.
trial.suggest_categorical()	`opt = trial.suggest_categorical("optimizer", ["adam", "sgd"])`	Samples one value from a list; supports `None`, `bool`, `int`, `float`, or `str` elements.
trial.report()	`trial.report(val_loss, step=epoch)`	Reports an intermediate objective value at a given step so pruners can evaluate the trial.
trial.should_prune()	`if trial.should_prune(): raise optuna.TrialPruned()`	Returns `True` if the active pruner decides this trial should be stopped early; must be called after `trial.report()`.