A Maximum Entropy Perspective on Spectral Dimensionality Reduction


at Machine Learning @ CUED, University of Cambridge, U.K. on Nov 16, 2011 [pdf]
Neil D. Lawrence, University of Sheffield



Spectral approaches to dimensionality reduction typically reduce the dimensionality of a data set through taking the eigenvectors of a Laplacian or a similarity matrix. Classical multidimensional scaling also makes use of the eigenvectors of a similarity matrix. In this talk we introduce a maximum entropy approach to designing this similarity matrix. The approach is closely related to maximum variance unfolding. Other spectral approaches, e.g. locally linear embeddings, turn out to also be closely related. These methods can be seen as a sparse Gaussian graphical model where correlations between data points (rather than across data features) are specified in the graph. The hope is that this unifying perspective will allow the relationships between these methods to be better understood and will also provide the groundwork for further research.