A Unifying Probabilistic Perspective for Spectral Dimensionality Reduction: Insights and New Models

Neil D. Lawrence

edit

Back to publications

A Unifying Probabilistic Perspective for Spectral Dimensionality Reduction: Insights and New Models

Neil D. Lawrence

Journal of Machine Learning Research, 13(51):1609-1638, 2012.

Abstract

We introduce a new perspective on spectral dimensionality reduction which views these methods as Gaussian Markov random fields (GRFs). Our unifying perspective is based on the maximum entropy principle which is in turn inspired by maximum variance unfolding. The resulting model, which we call maximum entropy unfolding (MEU) is a nonlinear generalization of principal component analysis. We relate the model to Laplacian eigenmaps and isomap. We show that parameter fitting in the locally linear embedding (LLE) is approximate maximum likelihood MEU. We introduce a variant of LLE that performs maximum likelihood exactly: Acyclic LLE (ALLE). We show that MEU and ALLE are competitive with the leading spectral approaches on a robot navigation visualization and a human motion capture data set. Finally the maximum likelihood perspective allows us to introduce a new approach to dimensionality reduction based on L1 regularization of the Gaussian random field via the graphical lasso.

Links

Cite this Paper

BibTeX


@Article{Lawrence-unifying12,
  title = 	 {A Unifying Probabilistic Perspective for Spectral Dimensionality Reduction: Insights and New Models},
  author = 	 {Lawrence, Neil D.},
  journal =      {Journal of Machine Learning Research},
  pages = 	 {1609--1638},
  year = 	 {2012},
  volume = 	 {13},
  number =       {51},
  pdf = 	 {ftp://ftp.dcs.shef.ac.uk/home/neil/spectral.pdf},
  url = 	 {/publications/a-unifying-probabilistic-perspective-for-spectral-dimensionality-reduction-insights-and-new-models.html},
  abstract = 	 {We introduce a new perspective on spectral dimensionality reduction which
views these methods as Gaussian Markov random fields (GRFs). Our unifying perspective
is based on the maximum entropy principle which is in turn inspired by maximum variance
unfolding. The resulting model, which we call maximum entropy unfolding (MEU) is
a nonlinear generalization of principal component analysis. We relate the model
to Laplacian eigenmaps and isomap. We show that parameter fitting in the locally
linear embedding (LLE) is approximate maximum likelihood MEU. We introduce a variant
of LLE that performs maximum likelihood exactly: Acyclic LLE (ALLE). We show that
MEU and ALLE are competitive with the leading spectral approaches on a robot navigation
visualization and a human motion capture data set. Finally the maximum likelihood
perspective allows us to introduce a new approach to dimensionality reduction based
on L1 regularization of the Gaussian random field via the graphical lasso.
}
}

Endnote

%0 Journal Article
%T A Unifying Probabilistic Perspective for Spectral Dimensionality Reduction: Insights and New Models
%A Neil D. Lawrence
%J Journal of Machine Learning Research
%D 2012	
%F Lawrence-unifying12
%P 1609--1638
%U /publications/a-unifying-probabilistic-perspective-for-spectral-dimensionality-reduction-insights-and-new-models.html
%V 13
%N 51
%X We introduce a new perspective on spectral dimensionality reduction which
views these methods as Gaussian Markov random fields (GRFs). Our unifying perspective
is based on the maximum entropy principle which is in turn inspired by maximum variance
unfolding. The resulting model, which we call maximum entropy unfolding (MEU) is
a nonlinear generalization of principal component analysis. We relate the model
to Laplacian eigenmaps and isomap. We show that parameter fitting in the locally
linear embedding (LLE) is approximate maximum likelihood MEU. We introduce a variant
of LLE that performs maximum likelihood exactly: Acyclic LLE (ALLE). We show that
MEU and ALLE are competitive with the leading spectral approaches on a robot navigation
visualization and a human motion capture data set. Finally the maximum likelihood
perspective allows us to introduce a new approach to dimensionality reduction based
on L1 regularization of the Gaussian random field via the graphical lasso.

RIS


TY  - JOUR
TI  - A Unifying Probabilistic Perspective for Spectral Dimensionality Reduction: Insights and New Models
AU  - Neil D. Lawrence
DA  - 2012/01/01	
ID  - Lawrence-unifying12
VL  - 13
IS  - 51
SP  - 1609
EP  - 1638
L1  - ftp://ftp.dcs.shef.ac.uk/home/neil/spectral.pdf
UR  - /publications/a-unifying-probabilistic-perspective-for-spectral-dimensionality-reduction-insights-and-new-models.html
AB  - We introduce a new perspective on spectral dimensionality reduction which
views these methods as Gaussian Markov random fields (GRFs). Our unifying perspective
is based on the maximum entropy principle which is in turn inspired by maximum variance
unfolding. The resulting model, which we call maximum entropy unfolding (MEU) is
a nonlinear generalization of principal component analysis. We relate the model
to Laplacian eigenmaps and isomap. We show that parameter fitting in the locally
linear embedding (LLE) is approximate maximum likelihood MEU. We introduce a variant
of LLE that performs maximum likelihood exactly: Acyclic LLE (ALLE). We show that
MEU and ALLE are competitive with the leading spectral approaches on a robot navigation
visualization and a human motion capture data set. Finally the maximum likelihood
perspective allows us to introduce a new approach to dimensionality reduction based
on L1 regularization of the Gaussian random field via the graphical lasso.

ER  -

APA


Lawrence, N.D.. (2012). A Unifying Probabilistic Perspective for Spectral Dimensionality Reduction: Insights and New Models. Journal of Machine Learning Research 13(51):1609-1638 Available from /publications/a-unifying-probabilistic-perspective-for-spectral-dimensionality-reduction-insights-and-new-models.html.