Gaussian Processes and the Null-Category Noise Model

[edit]

Neil D. Lawrence, University of Sheffield
Michael I. Jordan, UC Berkeley

in Semi-supervised Learning, pp 152-165

Related Material

Abstract

With Gaussian process classifiers (GPC) we aim to predict the posterior probability of the class labels given an input data point, $p(y_i x_i)$. In general we find that this posterior distribution is unaffected by unlabeled data points during learning. Support vector machines are strongly related to GPCs, but one notable difference is that the decision boundary in an SVM can be influenced by unlabeled data. The source of this discrepancy is the SVM’s margin: a characteristic which is not shared with the GPC. The presence of the marchin allows the support vector machine to seek low data density regions for the decision boundary, effectively allowing it to incorporate the cluster assumption (see Chapter 6). In this chapter we present the null category noise model. A probabilistic equivalent of the margin. By combining this noise model with a GPC we are able to incorporated the cluster assumption without explicitly modeling the input data density distributions and without a special choice of kernel.

@InCollection{lawrence-gpncnm05,
  title = 	 {Gaussian Processes and the Null-Category Noise Model},
  author = 	 {Neil D. Lawrence and Michael I. Jordan},
  booktitle = 	 {Semi-supervised Learning},
  pages = 	 {152},
  year = 	 {2006},
  editor = 	 {Olivier Chapelle and Bernhard Schölkopf and Alex Zien},
  address = 	 {Cambridge, MA},
  month = 	 {00},
  publisher = 	 {MIT Press},
  edit = 	 {https://github.com/lawrennd//publications/edit/gh-pages/_posts/2006-01-01-lawrence-gpncnm05.md},
  url =  	 {http://inverseprobability.com/publications/lawrence-gpncnm05.html},
  abstract = 	 {With Gaussian process classifiers (GPC) we aim to predict the posterior probability of the class labels given an input data point, $p(y_i|x_i)$. In general we find that this posterior distribution is unaffected by unlabeled data points during learning. Support vector machines are strongly related to GPCs, but one notable difference is that the decision boundary in an SVM can be influenced by unlabeled data. The source of this discrepancy is the SVM’s margin: a characteristic which is not shared with the GPC. The presence of the marchin allows the support vector machine to seek low data density regions for the decision boundary, effectively allowing it to incorporate the cluster assumption (see Chapter 6). In this chapter we present the *null category noise model*. A probabilistic equivalent of the margin. By combining this noise model with a GPC we are able to incorporated the cluster assumption without explicitly modeling the input data density distributions and without a special choice of kernel.},
  crossref =  {Chapelle:semisuper06},
  key = 	 {Lawrence:gpncnm05},
  linkps = 	 {ftp://ftp.dcs.shef.ac.uk/home/neil/gpncnm.ps},
  OPTgroup = 	 {}
 

}
%T Gaussian Processes and the Null-Category Noise Model
%A Neil D. Lawrence and Michael I. Jordan
%B 
%C Semi-supervised Learning
%D 
%E Olivier Chapelle and Bernhard Schölkopf and Alex Zien
%F lawrence-gpncnm05
%I MIT Press	
%P 152--165
%R 
%U http://inverseprobability.com/publications/lawrence-gpncnm05.html
%X With Gaussian process classifiers (GPC) we aim to predict the posterior probability of the class labels given an input data point, $p(y_i|x_i)$. In general we find that this posterior distribution is unaffected by unlabeled data points during learning. Support vector machines are strongly related to GPCs, but one notable difference is that the decision boundary in an SVM can be influenced by unlabeled data. The source of this discrepancy is the SVM’s margin: a characteristic which is not shared with the GPC. The presence of the marchin allows the support vector machine to seek low data density regions for the decision boundary, effectively allowing it to incorporate the cluster assumption (see Chapter 6). In this chapter we present the *null category noise model*. A probabilistic equivalent of the margin. By combining this noise model with a GPC we are able to incorporated the cluster assumption without explicitly modeling the input data density distributions and without a special choice of kernel.
TY  - CPAPER
TI  - Gaussian Processes and the Null-Category Noise Model
AU  - Neil D. Lawrence
AU  - Michael I. Jordan
BT  - Semi-supervised Learning
PY  - 2006/01/01
DA  - 2006/01/01
ED  - Olivier Chapelle
ED  - Bernhard Schölkopf
ED  - Alex Zien	
ID  - lawrence-gpncnm05
PB  - MIT Press	
SP  - 152
EP  - 165
UR  - http://inverseprobability.com/publications/lawrence-gpncnm05.html
AB  - With Gaussian process classifiers (GPC) we aim to predict the posterior probability of the class labels given an input data point, $p(y_i|x_i)$. In general we find that this posterior distribution is unaffected by unlabeled data points during learning. Support vector machines are strongly related to GPCs, but one notable difference is that the decision boundary in an SVM can be influenced by unlabeled data. The source of this discrepancy is the SVM’s margin: a characteristic which is not shared with the GPC. The presence of the marchin allows the support vector machine to seek low data density regions for the decision boundary, effectively allowing it to incorporate the cluster assumption (see Chapter 6). In this chapter we present the *null category noise model*. A probabilistic equivalent of the margin. By combining this noise model with a GPC we are able to incorporated the cluster assumption without explicitly modeling the input data density distributions and without a special choice of kernel.
ER  -

Lawrence, N.D. & Jordan, M.I.. (2006). Gaussian Processes and the Null-Category Noise Model. Semi-supervised Learning :152-165