A Unifying Review of Spectral Methods for Dimensionality Reduction


at Robotics Research Group Tutorial, Department of Engineering Science, University of Oxford on Feb 13, 2012 [pdf]
Neil D. Lawrence, University of Sheffield



In this tutorial we will review spectral approaches to dimensionality reduction, introducing a unifying probabilistic perspective. Our unifying perspective is based on the maximum entropy principle and the resulting probabilistic models are based on GRFs. We will review maximum variance unfolding, Laplacian eigenmaps, locally linear embeddings and Isomap. Under the framework, these approaches can be divided into those that preserve local distances and those that don’t. For two small data sets we show that local distance preserving methods tend to perform better. Finally we use the unifying framework to relate these approaches to the Gaussian process latent variable model.