Computationally Efficient Convolved Multiple Output Gaussian Processes

[edit]

Mauricio A. Álvarez, Universidad Tecnológica de Pereira, Colombia
Neil D. Lawrence, University of Sheffield

Journal of Machine Learning Research 12, pp 1425-1466

Related Material

Abstract

Recently there has been an increasing interest in regression methods that deal with multiple outputs. This has been motivated partly by frameworks like multitask learning, multisensor networks or structured output data. From a Gaussian processes perspective, the problem reduces to specifying an appropriate covariance function that, whilst being positive semi-definite, captures the dependencies between all the data points and across all the outputs. One approach to account for non-trivial correlations between outputs employs convolution processes. Under a latent function interpretation of the convolution transform we establish dependencies between output variables. The main drawbacks of this approach are the associated computational and storage demands. In this paper we address these issues. We present different efficient approximations for dependent output Gaussian processes constructed through the convolution formalism. We exploit the conditional independencies present naturally in the model. This leads to a form of the covariance similar in spirit to the so called PITC and FITC approximations for a single output. We show experimental results with synthetic and real data, in particular, we show results in school exams score prediction, pollution prediction and gene expression data


@Article{alvarez-computationally11,
  title = 	 {Computationally Efficient Convolved Multiple Output Gaussian Processes},
  journal =  	 {Journal of Machine Learning Research},
  author = 	 {Mauricio A. Álvarez and Neil D. Lawrence},
  pages = 	 {1425},
  year = 	 {2011},
  volume = 	 {12},
  month = 	 {00},
  edit = 	 {https://github.com/lawrennd//publications/edit/gh-pages/_posts/2011-05-01-alvarez-computationally11.md},
  url =  	 {http://inverseprobability.com/publications/alvarez-computationally11.html},
  abstract = 	 {Recently there has been an increasing interest in regression methods that deal with multiple outputs. This has been motivated partly by frameworks like multitask learning, multisensor networks or structured output data. From a Gaussian processes perspective, the problem reduces to specifying an appropriate covariance function that, whilst being positive semi-definite, captures the dependencies between all the data points and across all the outputs. One approach to account for non-trivial correlations between outputs employs convolution processes. Under a latent function interpretation of the convolution transform we establish dependencies between output variables. The main drawbacks of this approach are the associated computational and storage demands. In this paper we address these issues. We present different efficient approximations for dependent output Gaussian processes constructed through the convolution formalism. We exploit the conditional independencies present naturally in the model. This leads to a form of the covariance similar in spirit to the so called PITC and FITC approximations for a single output. We show experimental results with synthetic and real data, in particular, we show results in school exams score prediction, pollution prediction and gene expression data},
  key = 	 {Alvarez-computationally11},
  linkpdf = 	 {http://www.jmlr.org/papers/volume12/alvarez11a/alvarez11a.pdf},
  linksoftware = {https://github.com/SheffieldML/multigp},
  OPTgroup = 	 {}
 

}
%T Computationally Efficient Convolved Multiple Output Gaussian Processes
%A Mauricio A. Álvarez and Neil D. Lawrence
%B 
%C Journal of Machine Learning Research
%D 
%F alvarez-computationally11
%J Journal of Machine Learning Research	
%P 1425--1466
%R 
%U http://inverseprobability.com/publications/alvarez-computationally11.html
%V 12
%X Recently there has been an increasing interest in regression methods that deal with multiple outputs. This has been motivated partly by frameworks like multitask learning, multisensor networks or structured output data. From a Gaussian processes perspective, the problem reduces to specifying an appropriate covariance function that, whilst being positive semi-definite, captures the dependencies between all the data points and across all the outputs. One approach to account for non-trivial correlations between outputs employs convolution processes. Under a latent function interpretation of the convolution transform we establish dependencies between output variables. The main drawbacks of this approach are the associated computational and storage demands. In this paper we address these issues. We present different efficient approximations for dependent output Gaussian processes constructed through the convolution formalism. We exploit the conditional independencies present naturally in the model. This leads to a form of the covariance similar in spirit to the so called PITC and FITC approximations for a single output. We show experimental results with synthetic and real data, in particular, we show results in school exams score prediction, pollution prediction and gene expression data
TY  - CPAPER
TI  - Computationally Efficient Convolved Multiple Output Gaussian Processes
AU  - Mauricio A. Álvarez
AU  - Neil D. Lawrence
PY  - 2011/05/01
DA  - 2011/05/01	
ID  - alvarez-computationally11	
SP  - 1425
EP  - 1466
L1  - http://www.jmlr.org/papers/volume12/alvarez11a/alvarez11a.pdf
UR  - http://inverseprobability.com/publications/alvarez-computationally11.html
AB  - Recently there has been an increasing interest in regression methods that deal with multiple outputs. This has been motivated partly by frameworks like multitask learning, multisensor networks or structured output data. From a Gaussian processes perspective, the problem reduces to specifying an appropriate covariance function that, whilst being positive semi-definite, captures the dependencies between all the data points and across all the outputs. One approach to account for non-trivial correlations between outputs employs convolution processes. Under a latent function interpretation of the convolution transform we establish dependencies between output variables. The main drawbacks of this approach are the associated computational and storage demands. In this paper we address these issues. We present different efficient approximations for dependent output Gaussian processes constructed through the convolution formalism. We exploit the conditional independencies present naturally in the model. This leads to a form of the covariance similar in spirit to the so called PITC and FITC approximations for a single output. We show experimental results with synthetic and real data, in particular, we show results in school exams score prediction, pollution prediction and gene expression data
ER  -

Álvarez, M.A. & Lawrence, N.D.. (2011). Computationally Efficient Convolved Multiple Output Gaussian Processes. Journal of Machine Learning Research 12:1425-1466