The example of image
INNE: a Neural Network
Simulation of Back-error propagation
One of the neural models implemented in INNE is the Hebbian model to which we can apply the mentioned rules: the fundamental Hebbian, Oja and Sanger rules [Oja92], [San89]. These three updating rules allow a different component analysis of the input data and we have added to the environment a number of tools to visualize the 2D and the 3D cases.
In order to produce meaningful examples of these networks, a graphical tool for generating and visualizing 2D and 3D gausssian distributions has been added to the environment. Every input neuron is connected to all output neurons, therefore their arcs define the vectors which can be represented in the input space. The coordinates of the points, randomly generated according to a distribution in the 2D or 3D space introduced by users, are given in input to the network. During the learning phase we can observe how the network analyses the gaussian distribution given in input. Adopting the Sanger rule (see Fig. 4-a and Fig. 4-b), the weight vectors of the output nodes individuate the direction of the principal components of the distribution in a 3D and 2D space. On the contrary adopting the Oja rule (see Fig. 4-c), the network only finds the principal subspace.
In Fig. 4 we can see the input space and the vectors defined by the ouptput arc weights after a simulation of 10000 steps with and linear decay. During the learning phase we can observe how the vectors change their position in the space while the network tries to find the principal components of the distribution. At first ( is high) the swings are wide, while in the end there are only small swings.
The example of image
INNE: a Neural Network
Simulation of Back-error propagation