Adaline Example and Exercise

Example:

In the [1], they have the same example that they have considered before for the Perceptron with multiple output neurons. We use bipolar output neurons and the training set:

(class 1)



(class 2)



(class 3)



(class 4)



It is clear that N = 2, Q = 8, and the number of classes is 4. The number of output neuron is chosen to be M = 2 so that 2M = 4 classes can be represented.

Our exact calculation of the weights and bias for the case of a single output neuron can be extended to the case of multiple output neurons.
One can then obtain the following exact results for the weights and biases:




Using these exact results, we can easily see how good or bad our iterative solutions are.

It should be remarked that the most robust set of weights and biases is determined only by a few training vectors that lie very close to the decision boundaries. However in the Delta rule, all training
vectors contribute in some way. Therefore the set of weights and biases obtained by the Delta rule is not necessarily always the most robust.

The Delta rule usually gives convergent results if the learning rate is not too large. The resulting set of weights and biases typically leads to correct classification of all the training vectors, provided such a set exist. How close this set is to the best choice depends on the starting weights and biases, the learning rate and the number of iterations. We find that for this example much better convergence can be obtained if the learning rate at step k is set to be α = 1/k.



Exercise:


  1. Classify the next patterns set using the LMS algorithm and an ADALINE net

The initial parameters are:


a) Calculate the weights vector W and the threshold b


SOLUTION

First Period:

For the first pair of Input/Output


Step 1: It calculates the output using the activation own function of the network:


Step 2: It calculates the error:


Step 3: Using the Learning rule of Widrow - Hoff, it modifies the weights vector and the threshold




For the second pair of Input/Output


Step 1: It calculates the output using the activation own function of the network:



Step 2: It calculates the error:




Step 3: Using the Learning rule of Widrow - Hoff, it modifies the weights vector and the threshold




References:

[1] [1] K. Ming Leung. ADALINE fro Pattern Classification. POLYTECHNIC UNIVERSITY.



Comentarios

Publicar un comentario

Entradas más populares de este blog

Learning Algorithm (LMS rule or Widrow - Hoff)

Imagen
Learning Algorithm (LMS rule or Widrow - Hoff) An input pattern P is applied. The output of the Adaptive Linear Combiner (ALC) is obtained and the difference is calculated with respect to the desired, that is, the error. The weights are updated. Steps 1 to 3 are repeated with all input vectors. If the error is an acceptable value, stop, otherwise repeat the algorithm. The Widrow-Hoff or LMS (Least Mean Square) learning rule, which uses the Adaline network for its training, makes it possible to perform step 3. By means of the following equations, the network parameters are updated: For the vector of weights W For the threshold b For the error e Where is known as ratio or learning rate, such that, 0 < a  <= 1 . The calculus of this parameter is done by a correlation matrix R : The eigenvalues Ii of the correlation matrix will be useful for parameter determining a , that is: References: [1] Kim Seng Chia. Predicting the boiling point of
Leer más

ADALINE Architecture

Imagen
ADALINE Architecture Description This content is based in the [1] reference. The general structure of the ADALINE type network is: where: p = input patterns b = activation thresholds a = neuron output The output nets is given by: For an ADALNE network of a single neuron with two inputs the diagram corresponds to the following figure: In similarity to Perceptron, the limit of the decision characteristic for the ADALINE network is presented when n = 0, therefore: In the next figure, the line separating the input space in two regions, as shown in the following figure: The output of the neuron is major than zero in the gray area, in the white output area is less than zero. The Adaline network can correctly classify linearly separable patterns in two categories. The neuronal architecture has a layer of neurons connected to  R  inputs through a matrix of weights  W . This network is often called  MADALINE  or  Multiple ADALINE . It defines an output vector a o
Leer más