A hierarchical expected improvement method for Bayesian optimization
share
Expected improvement (EI) is one of the most popular Bayesian optimization (BO) methods, due to its closed-form acquisition function which allows for efficient optimization. However, one key drawback of EI is that it is overly greedy; this results in suboptimal solutions even for large sample sizes. To address this, we propose a new hierarchical EI (HEI) framework, which makes use of a hierarchical Gaussian process model. HEI preserves a closed-form acquisition function, and corrects the over-greediness of EI by encouraging exploration of the optimization space. Under certain prior specifications, we prove the global convergence of HEI over a broad objective function space, and derive global convergence rates under smoothness assumptions on the objective function. We then introduce several hyperparameter estimation methods, which allow HEI to mimic a fully Bayesian optimization procedure while avoiding expensive Markov-chain Monte Carlo sampling. Numerical experiments show the improvement of HEI over existing BO methods, for synthetic functions as well as a semiconductor manufacturing optimization problem.
READ FULL TEXT