Optuna

Install

If using anaconda python

1
 conda install -c conda-forge -y optuna

or if using standard python distribution

1
 pip install optuna

Useful resources:

Usage

Import module ...

1
import optuna

Define objective function (to maximise/minimise) ...

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
def objective(trial):

    # Parameter space
    parameter_space = {
        "criterion": trial.suggest_categorical('criterion', ['gini','entropy']),
        "max_depth": trial.suggest_int("max_depth", 1, 20),
        "max_features": trial.suggest_float("max_features", 0.1, 0.9),
        "n_estimators": trial.suggest_int("n_estimators", 2, 10),
    }

    # Setup model using hyper-parameters values
    model = RandomForestClassifier(**parameter_space)

    # Scoring model
    score = cross_val_score(model, X, y, n_jobs=-1, cv=10)

    return score.mean()

Define search ...

1
study = optuna.create_study(direction="maximize")

Carry out the search ...

1
2
3
start = time.time()
study.optimize(objective, n_trials=10)
end = time.time()

Report results ...

1
2
3
print("Fit Time:", end - start)
print("Best Param:", study.best_params)
print("Best score:", study.best_value)