Spectral Dimensionality Reduction via Maximum Entropy

[edit]

Neil D. Lawrence, University of Sheffield

in Proceedings of the Fourteenth International Workshop on Artificial Intelligence and Statistics 15, pp 51-59

Related Material

Errata

  • Equation (4) on Page 55, the sum index outside the norm should be over $i$, not $j$.
    Thanks to:

Abstract

We introduce a new perspective on spectral dimensionality reduction which views these methods as Gaussian random fields (GRFs). Our unifying perspective is based on the maximum entropy principle which is in turn inspired by maximum variance unfolding. The resulting probabilistic models are based on GRFs. The resulting model is a nonlinear generalization of principal component analysis. We show that parameter fitting in the locally linear embedding is approximate maximum likelihood in these models. We directly maximize the likelihood and show results that are competitive with the leading spectral approaches on a robot navigation visualization and a human motion capture data set.


@InProceedings{lawrence-spectral11,
  title = 	 {Spectral Dimensionality Reduction via Maximum Entropy},
  author = 	 {Neil D. Lawrence},
  booktitle = 	 {Proceedings of the Fourteenth International Workshop on Artificial Intelligence and Statistics},
  pages = 	 {51},
  year = 	 {2011},
  editor = 	 {Geoffrey Gordon and David Dunson},
  volume = 	 {15},
  address = 	 {Fort Lauderdale, FL, USA},
  month = 	 {00},
  publisher = 	 {JMLR W\&CP 15},
  edit = 	 {https://github.com/lawrennd//publications/edit/gh-pages/_posts/2011-01-01-lawrence-spectral11.md},
  url =  	 {http://inverseprobability.com/publications/lawrence-spectral11.html},
  abstract = 	 {We introduce a new perspective on spectral dimensionality reduction which views these methods as Gaussian random fields (GRFs). Our unifying perspective is based on the maximum entropy principle which is in turn inspired by maximum variance unfolding. The resulting probabilistic models are based on GRFs. The resulting model is a nonlinear generalization of principal component analysis. We show that parameter fitting in the locally linear embedding is approximate maximum likelihood in these models. We directly maximize the likelihood and show results that are competitive with the leading spectral approaches on a robot navigation visualization and a human motion capture data set.},
  crossref =  {Gordon:aistats11},
  key = 	 {Lawrence:spectral11},
  note = 	 {Notable Paper},
  linkpdf = 	 {http://jmlr.csail.mit.edu/proceedings/papers/v15/lawrence11a/lawrence11a.pdf},
  linksoftware = {https://github.com/SheffieldML/meu},
  OPTgroup = 	 {}
 

}
%T Spectral Dimensionality Reduction via Maximum Entropy
%A Neil D. Lawrence
%B 
%C Proceedings of the Fourteenth International Workshop on Artificial Intelligence and Statistics
%D 
%E Geoffrey Gordon and David Dunson
%F lawrence-spectral11
%I JMLR W\&CP 15	
%P 51--59
%R 
%U http://inverseprobability.com/publications/lawrence-spectral11.html
%V 15
%X We introduce a new perspective on spectral dimensionality reduction which views these methods as Gaussian random fields (GRFs). Our unifying perspective is based on the maximum entropy principle which is in turn inspired by maximum variance unfolding. The resulting probabilistic models are based on GRFs. The resulting model is a nonlinear generalization of principal component analysis. We show that parameter fitting in the locally linear embedding is approximate maximum likelihood in these models. We directly maximize the likelihood and show results that are competitive with the leading spectral approaches on a robot navigation visualization and a human motion capture data set.
TY  - CPAPER
TI  - Spectral Dimensionality Reduction via Maximum Entropy
AU  - Neil D. Lawrence
BT  - Proceedings of the Fourteenth International Workshop on Artificial Intelligence and Statistics
PY  - 2011/01/01
DA  - 2011/01/01
ED  - Geoffrey Gordon
ED  - David Dunson	
ID  - lawrence-spectral11
PB  - JMLR W\&CP 15	
SP  - 51
EP  - 59
L1  - http://jmlr.csail.mit.edu/proceedings/papers/v15/lawrence11a/lawrence11a.pdf
UR  - http://inverseprobability.com/publications/lawrence-spectral11.html
AB  - We introduce a new perspective on spectral dimensionality reduction which views these methods as Gaussian random fields (GRFs). Our unifying perspective is based on the maximum entropy principle which is in turn inspired by maximum variance unfolding. The resulting probabilistic models are based on GRFs. The resulting model is a nonlinear generalization of principal component analysis. We show that parameter fitting in the locally linear embedding is approximate maximum likelihood in these models. We directly maximize the likelihood and show results that are competitive with the leading spectral approaches on a robot navigation visualization and a human motion capture data set.
ER  -

Lawrence, N.D.. (2011). Spectral Dimensionality Reduction via Maximum Entropy. Proceedings of the Fourteenth International Workshop on Artificial Intelligence and Statistics 15:51-59