# Variational Learning for Multi-layer networks of Linear Threshold Units

Neil D. Lawrence, University of Sheffield

#### Abstract

Linear threshold units were originally proposed as models of biological neurons. They were widely studied in the context of the perceptron @Rosenblatt:book62. Due to the difficulties of finding a general algorithm for networks with hidden nodes, they never passed into general use. We derive an algorithm in the context of graphical models and show how it may be applied in multi-layer networks of linear threshold units. We demonstrate the algorithm through three well known datasets.

  @TechReport{lawrence-ltu_report00, title = {Variational Learning for Multi-layer networks of Linear Threshold Units}, author = {Neil D. Lawrence}, year = {2000}, month = {00}, edit = {https://github.com/lawrennd//publications/edit/gh-pages/_posts/2000-01-01-lawrence-ltu_report00.md}, url = {http://inverseprobability.com/publications/lawrence-ltu_report00.html}, abstract = {Linear threshold units were originally proposed as models of biological neurons. They were widely studied in the context of the perceptron @Rosenblatt:book62. Due to the difficulties of finding a general algorithm for networks with hidden nodes, they never passed into general use. We derive an algorithm in the context of graphical models and show how it may be applied in multi-layer networks of linear threshold units. We demonstrate the algorithm through three well known datasets.}, key = {Lawrence:ltu_report00}, note = {Draft report, slightly extended version of \cite{Lawrence:ltu01}.}, linkpsgz = {http://www.thelawrences.net/neil/ltupaper.ps.gz}, OPTgroup = {} }
 %T Variational Learning for Multi-layer networks of Linear Threshold Units %A Neil D. Lawrence %B %D %F lawrence-ltu_report00 %P -- %R %U http://inverseprobability.com/publications/lawrence-ltu_report00.html %X Linear threshold units were originally proposed as models of biological neurons. They were widely studied in the context of the perceptron @Rosenblatt:book62. Due to the difficulties of finding a general algorithm for networks with hidden nodes, they never passed into general use. We derive an algorithm in the context of graphical models and show how it may be applied in multi-layer networks of linear threshold units. We demonstrate the algorithm through three well known datasets. 
 TY - CPAPER TI - Variational Learning for Multi-layer networks of Linear Threshold Units AU - Neil D. Lawrence PY - 2000/01/01 DA - 2000/01/01 ID - lawrence-ltu_report00 SP - EP - UR - http://inverseprobability.com/publications/lawrence-ltu_report00.html AB - Linear threshold units were originally proposed as models of biological neurons. They were widely studied in the context of the perceptron @Rosenblatt:book62. Due to the difficulties of finding a general algorithm for networks with hidden nodes, they never passed into general use. We derive an algorithm in the context of graphical models and show how it may be applied in multi-layer networks of linear threshold units. We demonstrate the algorithm through three well known datasets. ER - 
 Lawrence, N.D.. (2000). Variational Learning for Multi-layer networks of Linear Threshold Units.:-