Perceptron
The perceptron is a type of artificial neural network invented in 1957 at the Cornell Aeronautical Laboratory by Frank Rosenblatt.The perceptron consists of one or more layers of artificial neurons; the inputs are fed directly to the outputs via a series of weights. In this way it can be considered the simplest kind of feedforward network. Each neuron calculates a weighted sum of its inputs - that is, the sum for all inputs of the product of an input and its corresponding weight. If this value is above some threshold, the neuron is said to 'fire', outputting the value 1; otherwise it takes the value -1. To simplify training, the threshold is often represented as an extra weight attached to a constant input, with the actual threshold function centred on 0.
Artificial neurons with this kind of activation function are also called McCulloch-Pitts neurons or threshold neurons. In the literature the term perceptron sometimes also refers to networks consisting of just one of these units.
Perceptrons can be trained by a simple learning algorithm that is usually called the delta-rule. It calculates the errors between calculated output and sample output data, and uses this to create an adjustment to the weights, thus implementing a form of gradient descent.
Although the perceptron initially seemed promising, it was quickly proved that simple perceptrons could not be trained to recognise many classes of patterns. This led to the field of neural network research stagnating for many years, before it was recognised that neural networks with three or more layers had far greater processing power than simpler perceptrons.
Simple perceptrons with one or two layers are only capable of learning linearly seperable patterns; in 1969 a famous monograph entitled Perceptrons by Marvin Minsky and Seymour Papert showed that it was impossible for these classes of network to learn an XOR function. They conjectured (incorrectly) that a similar result would hold for a perceptron with three or more layers.
The discovery in the 1980s that multi-layer neural networks did not, in fact, have these problems led to the resurgence of neural network research.
References:
- Rosenblatt, Frank (1958), The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain, Cornell Aeronautical Laboratory, Psychological Review, v65, No. 6, pp. 386-408.
- Minsky M L and Papert S A 1969 Perceptrons (Cambridge, MA: MIT Press)
