Fast Variational Inference for Gaussian Process Models through KL-Correction

Nathaniel J. King, Neil D. Lawrence
:270-281, 2006.

Abstract

Variational inference is a exible approach to solving problems of intractability in Bayesian models. Unfortunately the convergence of variational methods is often slow. We review a recently suggested variational approach for approximate inference in Gaussian process (GP) models and show how convergence may be dramatically improved through the use of a positive correction term to the standard variational bound. We refer to the modied bound as a KL-corrected bound. The KL-corrected bound is a lower bound on the true likelihood, but an upper bound on the original variational bound. Timing comparisons between optimisation of the two bounds show that optimisation of the new bound consistently improves the speed of convergence.

Cite this Paper


BibTeX
@InProceedings{pmlr-v-king-klcorrection06, title = {Fast Variational Inference for Gaussian Process Models through KL-Correction}, author = {Nathaniel J. King and Neil D. Lawrence}, pages = {270--281}, year = {}, editor = {}, address = {Berlin}, url = {http://inverseprobability.com/publications/king-klcorrection06.html}, abstract = {Variational inference is a exible approach to solving problems of intractability in Bayesian models. Unfortunately the convergence of variational methods is often slow. We review a recently suggested variational approach for approximate inference in Gaussian process (GP) models and show how convergence may be dramatically improved through the use of a positive correction term to the standard variational bound. We refer to the modied bound as a KL-corrected bound. The KL-corrected bound is a lower bound on the true likelihood, but an upper bound on the original variational bound. Timing comparisons between optimisation of the two bounds show that optimisation of the new bound consistently improves the speed of convergence.} }
Endnote
%0 Conference Paper %T Fast Variational Inference for Gaussian Process Models through KL-Correction %A Nathaniel J. King %A Neil D. Lawrence %B %C Proceedings of Machine Learning Research %D %E %F pmlr-v-king-klcorrection06 %I PMLR %J Proceedings of Machine Learning Research %P 270--281 %U http://inverseprobability.com %V %W PMLR %X Variational inference is a exible approach to solving problems of intractability in Bayesian models. Unfortunately the convergence of variational methods is often slow. We review a recently suggested variational approach for approximate inference in Gaussian process (GP) models and show how convergence may be dramatically improved through the use of a positive correction term to the standard variational bound. We refer to the modied bound as a KL-corrected bound. The KL-corrected bound is a lower bound on the true likelihood, but an upper bound on the original variational bound. Timing comparisons between optimisation of the two bounds show that optimisation of the new bound consistently improves the speed of convergence.
RIS
TY - CPAPER TI - Fast Variational Inference for Gaussian Process Models through KL-Correction AU - Nathaniel J. King AU - Neil D. Lawrence BT - PY - DA - ED - ID - pmlr-v-king-klcorrection06 PB - PMLR SP - 270 DP - PMLR EP - 281 L1 - UR - http://inverseprobability.com/publications/king-klcorrection06.html AB - Variational inference is a exible approach to solving problems of intractability in Bayesian models. Unfortunately the convergence of variational methods is often slow. We review a recently suggested variational approach for approximate inference in Gaussian process (GP) models and show how convergence may be dramatically improved through the use of a positive correction term to the standard variational bound. We refer to the modied bound as a KL-corrected bound. The KL-corrected bound is a lower bound on the true likelihood, but an upper bound on the original variational bound. Timing comparisons between optimisation of the two bounds show that optimisation of the new bound consistently improves the speed of convergence. ER -
APA
King, N.J. & Lawrence, N.D.. (). Fast Variational Inference for Gaussian Process Models through KL-Correction. , in PMLR :270-281

Related Material