Neural Networks - NN
Topics to be covered:
In this topic we going to get a brief idea about NN-
1.3 Different types of NN
?
1.3 Different types of Neural Networks ?
There are different types of Neural Networks available. The following are
the types
- Feedforward Neural Network- Artificial Neuron
- Radial basis function Neural Network
- Kohonen Self Organinzing Neural Network
- Recurrent Neural Network
- Convolutional Neural Network
- Modular Neural Network
Feedforward Neural Network – Artificial Neuron:
This neural
network is one of the simplest form of ANN, where the data or the input
travels in one direction. The data passes through the input nodes and exit
on the output nodes. This neural network may or may not have the hidden
layers. In simple words, it has a front propagated wave and no back
propagation by using a classifying activation function usually. Below is a
Single layer feed forward network. Here, the sum of the products of inputs
and weights are calculated and fed to the output. The output is considered
if it is above a certain value i.e threshold(usually 0) and the neuron fires
with an activated output (usually 1) and if it does not fire, the
deactivated value is emitted (usually -1).
|
|
Fig: Feedforward Neural Network |
Radial basis function Neural Network:
Radial
basic functions consider the distance of a point with respect to the
center. RBF functions have two layers, first where the features are
combined with the Radial Basis Function in the inner layer and then the
output of these features are taken into consideration while computing
the same output in the next time-step which is basically a memory.
Above
is a diagram which represents the distance calculating from the center
to a point in the plane similar to a radius of the circle. Here, the
distance measure used in euclidean, other distance measures can also
be used. The model depends on the maximum reach or the radius of the
circle in classifying the points into different categories. If the
point is in or around the radius, the likelihood of the new point
begin classified into that class is high. There can be a transition
while changing from one region to another and this can be controlled
by the beta function.
|
| Fig: Radial basis Neural Network |
Kohonen Self Organizing Neural Network:
The
objective of a Kohonen map is to input vectors of arbitrary dimension
to discrete map comprised of neurons. The map needs to me trained to
create its own organization of the training data. It comprises of
either one or two dimensions. When training the map the location of
the neuron remains constant but the weights differ depending on the
value. This self organization process has different parts, in the
first phase every neuron value is initialized with a small weight and
the input vector.
In the
second phase, the neuron closest to the point is the ‘winning neuron’
and the neurons connected to the winning neuron will also move towards
the point like in the graphic below. The distance between the point
and the neurons is calculated by the euclidean distance, the neuron
with the least distance wins. Through the iterations, all the points
are clustered and each neuron represents each kind of cluster. This is
the gist behind the organization of Kohonen Neural Network.
|
| Fig: Kohonen Self Organinzing Neural Network |
Kohonen Neural Network is used to
recognize patterns in the data. Its application can be found in
medical analysis to cluster data into different categories. Kohonen
map was able to classify patients having glomerular or tubular with an
high accuracy.
Recurrent Neural Network(RNN)-Long Short Term Memory:
The Recurrent Neural Network works on the principle of saving the output of a layer and feeding this back to the input to help in predicting the outcome of the layer. Here, the first layer is formed similar to the feed forward neural network with the product of the sum of the weights and the features.
The
recurrent neural network process starts once this is computed, this
means that from one time step to the next each neuron will remember
some information it had in the previous time-step. This makes each
neuron act like a memory cell in performing computations. In this
process, we need to let the neural network to work on the front
propagation and remember what information it needs for later use.
Here, if the prediction is wrong we use the learning rate or error
correction to make small changes so that it will gradually work
towards making the right prediction during the back propagation.
This is how a basic Recurrent Neural Network looks like,
|
| Fig: Recurrent Neural Network |
CNN (Convolutional neural network):
Convolutional neural networks are
basically applied on image data. Suppose we have an input of size
(28*28*3), If we use a normal neural network, there would be
2352(28*28*3) parameters. And as the size of the image increases the
number of parameters becomes very large. We “convolve” the images to
reduce the number of parameters (as shown above in filter
definition). As we slide the filter over the width and height of the
input volume we will produce a 2-dimensional activation map that
gives the output of that filter at every position.
|
| Fig: Convolutional Neural Network |
Modular Neural Network:
Modular Neural Networks have a
collection of different networks working independently and
contributing towards the output. Each neural network has a set of
inputs which are unique compared to other networks constructing and
performing sub-tasks. These networks do not interact or signal each
other in accomplishing the tasks. The advantage of a modular neural
network is that it breakdowns a large computational process into
smaller components decreasing the complexity. This breakdown will
help in decreasing the number of connections and negates the
interaction of these network with each other, which in turn will
increase the computation speed. However, the processing time will
depend on the number of neurons and their involvement in computing
the results.
|
| Fig: Modular Neural Network |
