Optuna or skopt are open source and won't take any GPU time at all to do it.

Optuna requires exploring the hyperparameter space which means running the experiments with those hyperparameters.

For a fixed search space it will almost certainly be better though.