Probabilistic Non-linear Component Analysis through Gaussian Process Latent Variable Models


at Microsoft Research, Cambridge, U.K. on Mar 15, 2005 [Gzipped Powerpoint]
Neil D. Lawrence, University of Sheffield


It is known that Principal Component Analysis has an underlying probabilistic representation based on a latent variable model. Principal component analysis (PCA) is recovered when the latent variables are integrated out and the parameters of the model are optimised by maximum likelihood. It is less well known that the dual approach of integrating out the parameters and optimising with respect to the latent variables also leads to PCA. The marginalised likelihood in this case takes the form of Gaussian process mappings, with linear Covariance functions, from a latent space to an observed space, which we refer to as a Gaussian Process Latent Variable Model (GPLVM). This dual probabilistic PCA is still a linear latent variable model, but by looking beyond the inner product kernel as a covariance function we can develop a non-linear probabilistic PCA. In the talk we will introduce the GPLVM and illustrate its application on a range of high dimensional data sets including motion capture data, hand written digits, a medical diagnosis data set and images.