Preferential Bayesian Optimization

Javier González; Zhenwen Dai; Andreas Damianou; Neil D. Lawrence

edit

Back to publications

Preferential Bayesian Optimization

Javier González, Zhenwen Dai, Andreas Damianou, Neil D. Lawrence

Proceedings of the 34th International Conference on Machine Learning, PMLR 70:1282-1291, 2017.

Abstract

Bayesian optimization (BO) has emerged during the last few years as an effective approach to optimize black-box functions where direct queries of the objective are expensive. We consider the case where direct access to the function is not possible, but information about user preferences is. Such scenarios arise in problems where human preferences are modeled, such as A/B tests or recommender systems. We present a new framework for this scenario that we call Preferential Bayesian Optimization (PBO) and that allows to find the optimum of a latent function that can only be queried through pairwise comparisons, so-called duels. PBO extend the applicability of standard BO ideas and generalizes previous discrete dueling approaches by modeling the probability of the the winner of each duel by means of Gaussian process model with a Bernoulli likelihood. The latent preference function is used to define a family of acquisition functions that extend usual policies used in BO. We illustrate the benefits of PBO in a variety of experiments in which we show how the way correlations are modeled is the key ingredient to drastically reduce the number of comparisons to find the optimum of the latent function of interest.

Links

Cite this Paper

BibTeX


@InProceedings{preferential-bayesian-optimization,
  title = 	 {Preferential {B}ayesian Optimization},
  author =       {Javier Gonz{\'a}lez and Zhenwen Dai and Andreas Damianou and Neil D. Lawrence},
  booktitle = 	 {Proceedings of the 34th International Conference on Machine Learning},
  pages = 	 {1282--1291},
  year = 	 {2017},
  editor = 	 {Precup, Doina and Teh, Yee Whye},
  volume = 	 {70},
  series = 	 {Proceedings of Machine Learning Research},
  publisher =    {PMLR},
  pdf = 	 {http://proceedings.mlr.press/v70/gonzalez17a/gonzalez17a.pdf},
  url = 	 {/publications/preferential-bayesian-optimization.html},
  abstract = 	 {Bayesian optimization (BO) has emerged during the last few years as an effective approach to optimize black-box functions where direct queries of the objective are expensive. We consider the case where direct access to the function is not possible, but information about user preferences is. Such scenarios arise in problems where human preferences are modeled, such as A/B tests or recommender systems. We present a new framework for this scenario that we call Preferential Bayesian Optimization (PBO) and that allows to find the optimum of a latent function that can only be queried through pairwise comparisons, so-called duels. PBO extend the applicability of standard BO ideas and generalizes previous discrete dueling approaches by modeling the probability of the the winner of each duel by means of Gaussian process model with a Bernoulli likelihood. The latent preference function is used to define a family of acquisition functions that extend usual policies used in BO. We illustrate the benefits of PBO in a variety of experiments in which we show how the way correlations are modeled is the key ingredient to drastically reduce the number of comparisons to find the optimum of the latent function of interest.}
}

Endnote

%0 Conference Paper
%T Preferential Bayesian Optimization
%A Javier González
%A Zhenwen Dai
%A Andreas Damianou
%A Neil D. Lawrence
%B Proceedings of the 34th International Conference on Machine Learning
%C Proceedings of Machine Learning Research
%D 2017
%E Doina Precup
%E Yee Whye Teh	
%F preferential-bayesian-optimization
%I PMLR
%P 1282--1291
%U /publications/preferential-bayesian-optimization.html
%V 70
%X Bayesian optimization (BO) has emerged during the last few years as an effective approach to optimize black-box functions where direct queries of the objective are expensive. We consider the case where direct access to the function is not possible, but information about user preferences is. Such scenarios arise in problems where human preferences are modeled, such as A/B tests or recommender systems. We present a new framework for this scenario that we call Preferential Bayesian Optimization (PBO) and that allows to find the optimum of a latent function that can only be queried through pairwise comparisons, so-called duels. PBO extend the applicability of standard BO ideas and generalizes previous discrete dueling approaches by modeling the probability of the the winner of each duel by means of Gaussian process model with a Bernoulli likelihood. The latent preference function is used to define a family of acquisition functions that extend usual policies used in BO. We illustrate the benefits of PBO in a variety of experiments in which we show how the way correlations are modeled is the key ingredient to drastically reduce the number of comparisons to find the optimum of the latent function of interest.

RIS


TY  - CPAPER
TI  - Preferential Bayesian Optimization
AU  - Javier González
AU  - Zhenwen Dai
AU  - Andreas Damianou
AU  - Neil D. Lawrence
BT  - Proceedings of the 34th International Conference on Machine Learning
DA  - 2017/07/17
ED  - Doina Precup
ED  - Yee Whye Teh	
ID  - preferential-bayesian-optimization
PB  - PMLR
DP  - Proceedings of Machine Learning Research
VL  - 70
SP  - 1282
EP  - 1291
L1  - http://proceedings.mlr.press/v70/gonzalez17a/gonzalez17a.pdf
UR  - /publications/preferential-bayesian-optimization.html
AB  - Bayesian optimization (BO) has emerged during the last few years as an effective approach to optimize black-box functions where direct queries of the objective are expensive. We consider the case where direct access to the function is not possible, but information about user preferences is. Such scenarios arise in problems where human preferences are modeled, such as A/B tests or recommender systems. We present a new framework for this scenario that we call Preferential Bayesian Optimization (PBO) and that allows to find the optimum of a latent function that can only be queried through pairwise comparisons, so-called duels. PBO extend the applicability of standard BO ideas and generalizes previous discrete dueling approaches by modeling the probability of the the winner of each duel by means of Gaussian process model with a Bernoulli likelihood. The latent preference function is used to define a family of acquisition functions that extend usual policies used in BO. We illustrate the benefits of PBO in a variety of experiments in which we show how the way correlations are modeled is the key ingredient to drastically reduce the number of comparisons to find the optimum of the latent function of interest.
ER  -

APA


González, J., Dai, Z., Damianou, A. & Lawrence, N.D.. (2017). Preferential Bayesian Optimization. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:1282-1291 Available from /publications/preferential-bayesian-optimization.html.