[edit]
Batch Bayesian Optimization via Local Penalization
Proceedings of the Nineteenth International Workshop on Artificial Intelligence and Statistics, PMLR 51:648-657, 2016.
Abstract
The popularity of Bayesian optimization methods for efficient exploration
of parameter spaces has lead to a series of papers applying Gaussian processes as
surrogates in the optimization of functions. However, most proposed approaches only
allow the exploration of the parameter space to occur sequentially. Often, it is
desirable to simultaneously propose batches of parameter values to explore. This
is particularly the case when large parallel processing facilities are available.
These could either be computational or physical facets of the process being optimized.
Batch methods, however, require the modeling of the interaction between the different
evaluations in the batch, which can be expensive in complex scenarios. We investigate
this issue and propose a highly effective heuristic based on an estimate of the
function's Lipschitz constant that captures the most important aspect of this
interaction---local repulsion---at negligible computational overhead. A penalized
acquisition function is used to collect batches of points minimizing the non-parallelizable
computational effort. The resulting algorithm compares very well, in run-time, with
much more elaborate alternatives.