A Unifying Probabilistic Perspective on Spectral Approaches to Dimensionality Reduction


at Microsoft Research, Cambridge on Sep 5, 2013 [pdf]
Neil D. Lawrence, University of Sheffield



Spectral approaches to dimensionality reduction typically reduce the dimensionality of a data set through taking the eigenvectors of a Laplacian or a similarity matrix. Classical multidimensional scaling also makes use of the eigenvectors of a similarity matrix. In this talk we introduce a maximum entropy approach to designing this similarity matrix. The approach is closely related to maximum variance unfolding. Other spectral approaches such as locally linear embeddings and Laplacian eigenmaps also turn out to be closely related. Each method can be seen as a sparse Gaussian graphical model where correlations between data points (rather than across data features) are specified in the graph. This also suggests optimization via sparse inverse covariance techniques such as the graphical LASSO. The hope is that this unifying perspective will allow the relationships between these methods to be better understood and will also provide the groundwork for further research.