Generalised Gaussian Process Latent Variable Models (GPLVM) with Stochastic Variational Inference

Vidhi Lalchand; Aditya Ravuri; Neil D. Lawrence

edit

Back to publications

Generalised Gaussian Process Latent Variable Models (GPLVM) with Stochastic Variational Inference

Vidhi Lalchand, Aditya Ravuri, Neil D. Lawrence

Proceedings of the 25th International Conference on Artificial Intelligence and Statistics (AISTATS), 2022.

Abstract

Gaussian process latent variable models (GPLVM) are a flexible and non-linear approach to dimensionality reduction, extending classical Gaussian processes to an unsupervised learning context. The Bayesian incarnation of the GPLVM [Titsias and Lawrence, 2010] uses a variational framework, where the posterior over latent variables is approximated by a well-behaved variational family, a factorized Gaussian yielding a tractable lower bound. However, the non-factorisability of the lower bound prevents truly scalable inference. In this work, we study the doubly stochastic formulation of the Bayesian GPLVM model amenable with minibatch training. We show how this framework is compatible with different latent variable formulations and perform experiments to compare a suite of models. Further, we demonstrate how we can train in the presence of massively missing data and obtain high-fidelity reconstructions. We demonstrate the model’s performance by benchmarking against the canonical sparse GPLVM for high-dimensional data examples.

Links

Cite this Paper

BibTeX


@InProceedings{generalised-gplvm-with-stochastic-variational-inference,
  title = 	 {Generalised Gaussian Process Latent Variable Models (GPLVM) with Stochastic Variational Inference},
  author = 	 {Lalchand, Vidhi and Ravuri, Aditya and Lawrence, Neil D.},
  booktitle = 	 {Proceedings of the 25th International Conference on Artificial Intelligence and Statistics (AISTATS)},
  year = 	 {2022},
  url = 	 {/publications/generalised-gplvm-with-stochastic-variational-inference.html},
  abstract = 	 {Gaussian process latent variable models (GPLVM) are a flexible and non-linear approach to dimensionality reduction, extending classical Gaussian processes to an unsupervised learning context. The Bayesian incarnation of the GPLVM [Titsias and Lawrence, 2010] uses a variational framework, where the posterior over latent variables is approximated by a well-behaved variational family, a factorized Gaussian yielding a tractable lower bound. However, the non-factorisability of the lower bound prevents truly scalable inference. In this work, we study the doubly stochastic formulation of the Bayesian GPLVM model amenable with minibatch training. We show how this framework is compatible with different latent variable formulations and perform experiments to compare a suite of models. Further, we demonstrate how we can train in the presence of massively missing data and obtain high-fidelity reconstructions. We demonstrate the model's performance by benchmarking against the canonical sparse GPLVM for high-dimensional data examples.
}
}

Endnote

%0 Conference Paper
%T Generalised Gaussian Process Latent Variable Models (GPLVM) with Stochastic Variational Inference
%A Vidhi Lalchand
%A Aditya Ravuri
%A Neil D. Lawrence
%B Proceedings of the 25th International Conference on Artificial Intelligence and Statistics (AISTATS)
%D 2022	
%F generalised-gplvm-with-stochastic-variational-inference
%U /publications/generalised-gplvm-with-stochastic-variational-inference.html
%X Gaussian process latent variable models (GPLVM) are a flexible and non-linear approach to dimensionality reduction, extending classical Gaussian processes to an unsupervised learning context. The Bayesian incarnation of the GPLVM [Titsias and Lawrence, 2010] uses a variational framework, where the posterior over latent variables is approximated by a well-behaved variational family, a factorized Gaussian yielding a tractable lower bound. However, the non-factorisability of the lower bound prevents truly scalable inference. In this work, we study the doubly stochastic formulation of the Bayesian GPLVM model amenable with minibatch training. We show how this framework is compatible with different latent variable formulations and perform experiments to compare a suite of models. Further, we demonstrate how we can train in the presence of massively missing data and obtain high-fidelity reconstructions. We demonstrate the model's performance by benchmarking against the canonical sparse GPLVM for high-dimensional data examples.

RIS


TY  - CPAPER
TI  - Generalised Gaussian Process Latent Variable Models (GPLVM) with Stochastic Variational Inference
AU  - Vidhi Lalchand
AU  - Aditya Ravuri
AU  - Neil D. Lawrence
BT  - Proceedings of the 25th International Conference on Artificial Intelligence and Statistics (AISTATS)
DA  - 2022/02/25	
ID  - generalised-gplvm-with-stochastic-variational-inference
UR  - /publications/generalised-gplvm-with-stochastic-variational-inference.html
AB  - Gaussian process latent variable models (GPLVM) are a flexible and non-linear approach to dimensionality reduction, extending classical Gaussian processes to an unsupervised learning context. The Bayesian incarnation of the GPLVM [Titsias and Lawrence, 2010] uses a variational framework, where the posterior over latent variables is approximated by a well-behaved variational family, a factorized Gaussian yielding a tractable lower bound. However, the non-factorisability of the lower bound prevents truly scalable inference. In this work, we study the doubly stochastic formulation of the Bayesian GPLVM model amenable with minibatch training. We show how this framework is compatible with different latent variable formulations and perform experiments to compare a suite of models. Further, we demonstrate how we can train in the presence of massively missing data and obtain high-fidelity reconstructions. We demonstrate the model's performance by benchmarking against the canonical sparse GPLVM for high-dimensional data examples.

ER  -

APA


Lalchand, V., Ravuri, A. & Lawrence, N.D.. (2022). Generalised Gaussian Process Latent Variable Models (GPLVM) with Stochastic Variational Inference. Proceedings of the 25th International Conference on Artificial Intelligence and Statistics (AISTATS) Available from /publications/generalised-gplvm-with-stochastic-variational-inference.html.