A Variational Bayesian Committee of Neural Networks

Neil D. Lawrence, Mehdi Azzouzi
, 1999.

Abstract

Exact inference in Bayesian neural networks is non analytic to compute and as a result approximate approaches such as the evidence procedure, Monte-Carlo sampling and variational inference have been proposed. In this paper we present a general overview of the Bayesian approach with a particular emphasis on the variational procedure. We then present a new approximating distribution based on *mixtures* of Gaussian distributions and show how it may be implemented. We present results on a simple toy problem and on two real world data-sets.

Cite this Paper

BibTeX
@Misc{Lawrence:nnmixtures99, title = {A Variational Bayesian Committee of Neural Networks}, author = {Lawrence, Neil D. and Azzouzi, Mehdi}, year = {1999}, pdf = {http://www.thelawrences.net/neil/nnmixture.pdf}, url = {http://inverseprobability.com/publications/lawrence-nnmixtures99.html}, abstract = {Exact inference in Bayesian neural networks is non analytic to compute and as a result approximate approaches such as the evidence procedure, Monte-Carlo sampling and variational inference have been proposed. In this paper we present a general overview of the Bayesian approach with a particular emphasis on the variational procedure. We then present a new approximating distribution based on *mixtures* of Gaussian distributions and show how it may be implemented. We present results on a simple toy problem and on two real world data-sets. } }
Endnote
%0 Generic %T A Variational Bayesian Committee of Neural Networks %A Neil D. Lawrence %A Mehdi Azzouzi %D 1999 %F Lawrence:nnmixtures99 %U http://inverseprobability.com/publications/lawrence-nnmixtures99.html %X Exact inference in Bayesian neural networks is non analytic to compute and as a result approximate approaches such as the evidence procedure, Monte-Carlo sampling and variational inference have been proposed. In this paper we present a general overview of the Bayesian approach with a particular emphasis on the variational procedure. We then present a new approximating distribution based on *mixtures* of Gaussian distributions and show how it may be implemented. We present results on a simple toy problem and on two real world data-sets.
RIS
TY - GEN TI - A Variational Bayesian Committee of Neural Networks AU - Neil D. Lawrence AU - Mehdi Azzouzi DA - 1999/09/17 ID - Lawrence:nnmixtures99 L1 - http://www.thelawrences.net/neil/nnmixture.pdf UR - http://inverseprobability.com/publications/lawrence-nnmixtures99.html AB - Exact inference in Bayesian neural networks is non analytic to compute and as a result approximate approaches such as the evidence procedure, Monte-Carlo sampling and variational inference have been proposed. In this paper we present a general overview of the Bayesian approach with a particular emphasis on the variational procedure. We then present a new approximating distribution based on *mixtures* of Gaussian distributions and show how it may be implemented. We present results on a simple toy problem and on two real world data-sets. ER -
APA
Lawrence, N.D. & Azzouzi, M.. (1999). A Variational Bayesian Committee of Neural Networks. Available from http://inverseprobability.com/publications/lawrence-nnmixtures99.html.

Related Material