layout: inproceedings title: Structured Variationally Auto-encoded Optimization booktitle: Proceedings of the 35th International Conference on Machine Learning year: ‘2018’ volume: ‘80’ series: Proceedings of Machine Learning Research address: month: 0 publisher: PMLR pdf: http://proceedings.mlr.press/v80/lu18c/lu18c.pdf url: http://proceedings.mlr.press/v80/lu2018c.html abstract: We tackle the problem of optimizing a black-box objective function defined over a highly-structured input space. This problem is ubiquitous in science and engineering. In machine learning, inferring the structure of a neural network or the Automatic Statistician (AS), where the optimal kernel combination for a Gaussian process is selected, are two important examples. We use the \as as a case study to describe our approach, that can be easily generalized to other domains. We propose an Structure Generating Variational Auto-encoder (SG-VAE) to embed the original space of kernel combinations into some low-dimensional continuous manifold where Bayesian optimization (BO) ideas are used. This is possible when structural knowledge of the problem is available, which can be given via a simulator or any other form of generating potentially good solutions. The right exploration-exploitation balance is imposed by propagating into the search the uncertainty of the latent space of the SG-VAE, that is computed using variational inference. The key aspect of our approach is that the SG-VAE can be used to bias the search towards relevant regions, making it suitable for transfer learning tasks. Several experiments in various application domains are used to illustrate the utility and generality of the approach described in this work. layout: inproceedings id: lu18c tex_title: Structured Variationally Auto-encoded Optimization firstpage: 3273 lastpage: 3281 page: 3273-3281 order: 3273 cycles: false bibtex_editor: Dy, Jennifer and Krause, Andreas editor:

  • given: Jennifer family: Dy
  • given: Andreas family: Krause bibtex_author: Lu, Xiaoyu and Gonzalez, Javier and Dai, Zhenwen and Lawrence, Neil D. author:
  • given: Xiaoyu family: Lu
  • given: Javier family: Gonzalez
  • given: Zhenwen family: Dai
  • given: Neil D. family: Lawrence gscholar: r3SJcvoAAAAJ institute: University of Sheffield twitter: lawrennd url: http://inverseprobability.com date: 2018-07-03 container-title: Proceedings of the 35th International Conference on Machine Learning genre: inproceedings issued: date-parts:
    • 2018
    • 7
    • 3 extras:
  • label: Supplementary PDF link: http://proceedings.mlr.press/v80/lu18c/lu18c-supp.pdf