Variationally Auto-Encoded Deep Gaussian Processes


Zhenwen Dai, Inferentia Ltd
Andreas Damianou, University of Sheffield
Javier Gonzalez, University of Sheffield
Neil D. Lawrence, University of Sheffield
in Proceedings of the International Conference on Learning Representations 3

Related Material


We develop a scalable deep non-parametric generative model by augmenting deep Gaussian processes with a recognition model. Inference is performed in a novel scalable variational framework where the variational posterior distributions are reparametrized through a multilayer perceptron. The key aspect of this reformulation is that it prevents the proliferation of variational parameters which otherwise grow linearly in proportion to the sample size. We derive a new formulation of the variational lower bound that allows us to distribute most of the computation in a way that enables to handle datasets of the size of mainstream deep learning tasks. We show the efficacy of the method on a variety of challenges including deep unsupervised learning and deep Bayesian optimization.