Deep Gaussian Processes

Andreas Damianou; Neil D. Lawrence

edit

Back to publications

Deep Gaussian Processes

Andreas Damianou, Neil D. Lawrence

Proceedings of the Sixteenth International Workshop on Artificial Intelligence and Statistics, PMLR 31:207-215, 2013.

Abstract

In this paper we introduce deep Gaussian process (GP) models. Deep GPs are a deep belief network based on Gaussian process mappings. The data is modeled as the output of a multivariate GP. The inputs to that Gaussian process are then governed by another GP. A single layer model is equivalent to a standard GP or the GP latent variable model (GP-LVM). We perform inference in the model by approximate variational marginalization. This results in a strict lower bound on the marginal likelihood of the model which we use for model selection (number of layers and nodes per layer). Deep belief networks are typically applied to relatively large data sets using stochastic gradient descent for optimization. Our fully Bayesian treatment allows for the application of deep models even when data is scarce. Model selection by our variational bound shows that a five layer hierarchy is justified even when modelling a digit data set containing only 150 examples.

Links

Cite this Paper

BibTeX


@InProceedings{Damianou-deepgp13,
  title = 	 {Deep {G}aussian Processes},
  author = 	 {Damianou, Andreas and Lawrence, Neil D.},
  booktitle = 	 {Proceedings of the Sixteenth International Workshop on Artificial Intelligence and Statistics},
  pages = 	 {207--215},
  year = 	 {2013},
  editor = 	 {Carvalho, Carlos and Ravikumar, Pradeep},
  volume = 	 {31},
  address = 	 {Scottsdale, Arizona, USA},
  publisher =    {PMLR},
  pdf = 	 {http://proceedings.mlr.press/v31/damianou13a.pdf},
  url = 	 {/publications/deep-gaussian-processes.html},
  abstract = 	 {In this paper we introduce deep Gaussian process (GP) models. Deep GPs are
a deep belief network based on Gaussian process mappings. The data is modeled as
the output of a multivariate GP. The inputs to that Gaussian process are then governed
by another GP. A single layer model is equivalent to a standard GP or the GP latent
variable model (GP-LVM). We perform inference in the model by approximate variational
marginalization. This results in a strict lower bound on the marginal likelihood
of the model which we use for model selection (number of layers and nodes per layer).
Deep belief networks are typically applied to relatively large data sets using stochastic
gradient descent for optimization. Our fully Bayesian treatment allows for the application
of deep models even when data is scarce. Model selection by our variational bound
shows that a five layer hierarchy is justified even when modelling a digit data
set containing only 150 examples.
}
}

Endnote

%0 Conference Paper
%T Deep Gaussian Processes
%A Andreas Damianou
%A Neil D. Lawrence
%B Proceedings of the Sixteenth International Workshop on Artificial Intelligence and Statistics
%D 2013
%E Carlos Carvalho
%E Pradeep Ravikumar	
%F Damianou-deepgp13
%I PMLR
%P 207--215
%U /publications/deep-gaussian-processes.html
%V 31
%X In this paper we introduce deep Gaussian process (GP) models. Deep GPs are
a deep belief network based on Gaussian process mappings. The data is modeled as
the output of a multivariate GP. The inputs to that Gaussian process are then governed
by another GP. A single layer model is equivalent to a standard GP or the GP latent
variable model (GP-LVM). We perform inference in the model by approximate variational
marginalization. This results in a strict lower bound on the marginal likelihood
of the model which we use for model selection (number of layers and nodes per layer).
Deep belief networks are typically applied to relatively large data sets using stochastic
gradient descent for optimization. Our fully Bayesian treatment allows for the application
of deep models even when data is scarce. Model selection by our variational bound
shows that a five layer hierarchy is justified even when modelling a digit data
set containing only 150 examples.

RIS


TY  - CPAPER
TI  - Deep Gaussian Processes
AU  - Andreas Damianou
AU  - Neil D. Lawrence
BT  - Proceedings of the Sixteenth International Workshop on Artificial Intelligence and Statistics
DA  - 2013/04/29
ED  - Carlos Carvalho
ED  - Pradeep Ravikumar	
ID  - Damianou-deepgp13
PB  - PMLR
VL  - 31
SP  - 207
EP  - 215
L1  - http://proceedings.mlr.press/v31/damianou13a.pdf
UR  - /publications/deep-gaussian-processes.html
AB  - In this paper we introduce deep Gaussian process (GP) models. Deep GPs are
a deep belief network based on Gaussian process mappings. The data is modeled as
the output of a multivariate GP. The inputs to that Gaussian process are then governed
by another GP. A single layer model is equivalent to a standard GP or the GP latent
variable model (GP-LVM). We perform inference in the model by approximate variational
marginalization. This results in a strict lower bound on the marginal likelihood
of the model which we use for model selection (number of layers and nodes per layer).
Deep belief networks are typically applied to relatively large data sets using stochastic
gradient descent for optimization. Our fully Bayesian treatment allows for the application
of deep models even when data is scarce. Model selection by our variational bound
shows that a five layer hierarchy is justified even when modelling a digit data
set containing only 150 examples.

ER  -

APA


Damianou, A. & Lawrence, N.D.. (2013). Deep Gaussian Processes. Proceedings of the Sixteenth International Workshop on Artificial Intelligence and Statistics 31:207-215 Available from /publications/deep-gaussian-processes.html.