Preferential Bayesian Optimization

Javier González, Zhenwen Dai, Andreas Damianou, Neil D. Lawrence
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:1282-1291, 2017.

Abstract

Bayesian optimization (BO) has emerged during the last few years as an effective approach to optimize black-box functions where direct queries of the objective are expensive. We consider the case where direct access to the function is not possible, but information about user preferences is. Such scenarios arise in problems where human preferences are modeled, such as A/B tests or recommender systems. We present a new framework for this scenario that we call Preferential Bayesian Optimization (PBO) and that allows to find the optimum of a latent function that can only be queried through pairwise comparisons, so-called duels. PBO extend the applicability of standard BO ideas and generalizes previous discrete dueling approaches by modeling the probability of the the winner of each duel by means of Gaussian process model with a Bernoulli likelihood. The latent preference function is used to define a family of acquisition functions that extend usual policies used in BO. We illustrate the benefits of PBO in a variety of experiments in which we show how the way correlations are modeled is the key ingredient to drastically reduce the number of comparisons to find the optimum of the latent function of interest.

Cite this Paper

BibTeX
@InProceedings{preferential-bayesian-optimization, title = {Preferential {B}ayesian Optimization}, author = {Javier Gonz{\'a}lez and Zhenwen Dai and Andreas Damianou and Neil D. Lawrence}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {1282--1291}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/gonzalez17a/gonzalez17a.pdf}, url = {http://inverseprobability.com/publications/preferential-bayesian-optimization.html}, abstract = {Bayesian optimization (BO) has emerged during the last few years as an effective approach to optimize black-box functions where direct queries of the objective are expensive. We consider the case where direct access to the function is not possible, but information about user preferences is. Such scenarios arise in problems where human preferences are modeled, such as A/B tests or recommender systems. We present a new framework for this scenario that we call Preferential Bayesian Optimization (PBO) and that allows to find the optimum of a latent function that can only be queried through pairwise comparisons, so-called duels. PBO extend the applicability of standard BO ideas and generalizes previous discrete dueling approaches by modeling the probability of the the winner of each duel by means of Gaussian process model with a Bernoulli likelihood. The latent preference function is used to define a family of acquisition functions that extend usual policies used in BO. We illustrate the benefits of PBO in a variety of experiments in which we show how the way correlations are modeled is the key ingredient to drastically reduce the number of comparisons to find the optimum of the latent function of interest.} }
Endnote
%0 Conference Paper %T Preferential Bayesian Optimization %A Javier González %A Zhenwen Dai %A Andreas Damianou %A Neil D. Lawrence %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F preferential-bayesian-optimization %I PMLR %P 1282--1291 %U http://inverseprobability.com/publications/preferential-bayesian-optimization.html %V 70 %X Bayesian optimization (BO) has emerged during the last few years as an effective approach to optimize black-box functions where direct queries of the objective are expensive. We consider the case where direct access to the function is not possible, but information about user preferences is. Such scenarios arise in problems where human preferences are modeled, such as A/B tests or recommender systems. We present a new framework for this scenario that we call Preferential Bayesian Optimization (PBO) and that allows to find the optimum of a latent function that can only be queried through pairwise comparisons, so-called duels. PBO extend the applicability of standard BO ideas and generalizes previous discrete dueling approaches by modeling the probability of the the winner of each duel by means of Gaussian process model with a Bernoulli likelihood. The latent preference function is used to define a family of acquisition functions that extend usual policies used in BO. We illustrate the benefits of PBO in a variety of experiments in which we show how the way correlations are modeled is the key ingredient to drastically reduce the number of comparisons to find the optimum of the latent function of interest.
RIS
TY - CPAPER TI - Preferential Bayesian Optimization AU - Javier González AU - Zhenwen Dai AU - Andreas Damianou AU - Neil D. Lawrence BT - Proceedings of the 34th International Conference on Machine Learning DA - 2017/07/17 ED - Doina Precup ED - Yee Whye Teh ID - preferential-bayesian-optimization PB - PMLR DP - Proceedings of Machine Learning Research VL - 70 SP - 1282 EP - 1291 L1 - http://proceedings.mlr.press/v70/gonzalez17a/gonzalez17a.pdf UR - http://inverseprobability.com/publications/preferential-bayesian-optimization.html AB - Bayesian optimization (BO) has emerged during the last few years as an effective approach to optimize black-box functions where direct queries of the objective are expensive. We consider the case where direct access to the function is not possible, but information about user preferences is. Such scenarios arise in problems where human preferences are modeled, such as A/B tests or recommender systems. We present a new framework for this scenario that we call Preferential Bayesian Optimization (PBO) and that allows to find the optimum of a latent function that can only be queried through pairwise comparisons, so-called duels. PBO extend the applicability of standard BO ideas and generalizes previous discrete dueling approaches by modeling the probability of the the winner of each duel by means of Gaussian process model with a Bernoulli likelihood. The latent preference function is used to define a family of acquisition functions that extend usual policies used in BO. We illustrate the benefits of PBO in a variety of experiments in which we show how the way correlations are modeled is the key ingredient to drastically reduce the number of comparisons to find the optimum of the latent function of interest. ER -
APA
González, J., Dai, Z., Damianou, A. & Lawrence, N.D.. (2017). Preferential Bayesian Optimization. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:1282-1291 Available from http://inverseprobability.com/publications/preferential-bayesian-optimization.html.

Related Material