1. Artificial Neural Networks

2. CONTENTS INTRODUCTION HISTORY BIOLOGICAL NEURON MODEL
ARTIFICIAL NEURON MODEL
ARTIFICIAL NEURAL NETWORK
NEURAL NETWORK ARCHITECTURE
LEARNING
BACKPROPAGATION ALGORITHM
APPLICATIONS
ADVANTAGES
CONCLUSION

3. INTRODUCTION
“Neural“ is an adjective for neuron, and “network” denotes a graph like structure.
Artificial Neural Networks are also referred to as “neural nets” , “artificial neural systems”, “parallel distributed processing systems”, “connectionist systems”.
For a computing systems to be called by these pretty names, it is necessary for the system to have a labeled directed graph structure where nodes performs some simple computations.
“Directed Graph” consists of set of “nodes”(vertices) and a set of “connections”(edges/links/arcs) connecting pair of nodes.
A graph is said to be “labeled graph” if each connection is associated with a label to identify some property of the connection

4. CONTD…
x1 x2
x1ϵ{0,1}
o = x1 AND x2
multiplier
(x1 w1)
(x2w2)
o = x1 AND x2 x1 x2 w1 w2
Fig 1: AND gate graph This graph cannot be considered a neural network since the connections between the nodes are fixed and appear to play no other role than carrying the inputs to the node that computed their conjunction.
Fig 2: AND gate network The graph structure which connects the weights modifiable using a learning algorithm, qualifies the computing system to be called an artificial neural networks.
x2ϵ{0,1}
The field of neural network was pioneered by BERNARD WIDROW of Stanford University in 1950’s.

5. HISTORY
late-1800's - Neural Networks appear as an analogy to biological systems
1960's and 70's – Simple neural networks appear
Fall out of favour because the perceptron is not effective by itself, and there were no good algorithms for multilayer nets
1986 – Backpropagation algorithm appears
Neural Networks have a resurgence in popularity

6. FEATURES OF ARTIFICIAL NEURAL NETWORKS
Records (examples) need to be represented as a (possibly large) set of tuples of <attribute, value>
The output values can be represented as a discrete value, a real value, or a vector of values
Tolerant to noise in input data
Time factor
It takes long time for training
Once trained, an ANN produces output values (predictions) fast
It is hard for human to interpret the process of prediction by ANN

7. BIOLOGICAL NEURON MODEL
Four parts of a typical nerve cell :
DENDRITES: Accepts the inputs
SOMA : Process the inputs
AXON : Turns the processed inputs into outputs.
SYNAPSES : The electrochemical contact between the neurons.

8. ARTIFICIAL NEURON MODEL
Inputs to the network are represented by the mathematical symbol, xn
Each of these inputs are multiplied by a connection weight , wn
sum = w1 x1 + ……+ wnxn
These products are simply summed, fed through the transfer function, f( ) to generate a result and then output. f w1 w2 xn x2 x1 wn
f(w1 x1 + ……+ wnxn)

9. Biological Terminology Artificial Neural Network Terminology
Neuron
Node/Unit/Cell/Neurode
Synapse
Connection/Edge/Link
Synaptic Efficiency
Connection Strength/Weight
Firing frequency
Node output

10. ARTIFICIAL NEURAL NETWORK
Artificial Neural Network (ANNs) are programs designed to solve any problem by trying to mimic the structure and the function of our nervous system.
Neural networks are based on simulated neurons, Which are joined together in a variety of ways to form networks.
Neural network resembles the human brain in the following two ways:
* A neural network acquires knowledge through learning.
*A neural network’s knowledge is stored within the interconnection strengths known as synaptic weight.

11. ARTIFICIAL NEURAL NETWORK MODEL
CONTD…
ARTIFICIAL NEURAL NETWORK MODEL
output layer
connections
Input layer
Hidden layers
Neural network
Including connections
(called weights) between neuron
Compare
Actual
output
Desired
Input
Artificial neural network model
Figure showing adjust of neural network

12. NEURAL NETWORK ARCHITECTURES
Layer 1 Layer2
Layer Layer3
(Input layer)
(Output layer)
Hidden Layer
Input node
output node
Hidden node
The neural network in which every node is connected to every other nodes, and these connections may be either excitatory (positive weights), inhibitory (negative weights), or irrelevant (almost zero weights).
These are networks in which nodes are partitioned into subsets called layers, with no connections from layer j to k if j > k.
Fully connected network
Layered network

13. CONTD…
Layer 1 Layer2
Layer Layer3
(Input layer)
(Output layer)
Hidden Layer
Layer 1 Layer2
Layer Layer3
(Input layer)
(Output layer)
Hidden Layer
Feedforward network This is a subclass of acyclic networks in which a connection is allowed from a node in layer i only to nodes in layer i+1
This is the subclass of the layered networks in which there is no intra-layer connections. In other words, a connection may exist between any node in layer i and any node in layer j for i < j, but a connection is not allowed for i=j.
Acyclic network

14. Modular neural network
CONTD…
Modular neural network
Many problems are best solved using neural networks whose architecture consists of several modules, with sparse interconnections between them. Modules can be organized in several different ways as Hierarchical organization, Successive refinement, Input modularity

15. LEARNING
Neurons in an animal’s brain are “hard wired”. It is equally obvious that animals, especially higher order animals, learn as they grow.
How does this learning occur?
What are possible mathematical models of learning?
In artificial neural networks, learning refers to the method of modifying the weights of connections between the nodes of a specified network.
The learning ability of a neural network is determined by its architecture and by the algorithmic method chosen for training.

16. UNSUPERVISED LEARNING
CONTD…
UNSUPERVISED LEARNING
SUPERVISED LEARNING
This is learning by doing.
In this approach no sample outputs are provided to the network against which it can measure its predictive performance for a given vector of inputs.
One common form of unsupervised learning is clustering where we try to categorize data in different clusters by their similarity.
A teacher is available to indicate whether a system is performing correctly, or to indicate the amount of error in system performance. Here a teacher is a set of training data.
The training data consist of pairs of input and desired output values that are traditionally represented in data vectors.
Supervised learning can also be referred as classification, where we have a wide range of classifiers, (Multilayer perceptron, k nearest neighbor..etc)

17. THE BACKPROPAGATION ALGORITHM
The backpropagation algorithm (Rumelhart and McClelland, 1986) is used in layered feed-forward Artificial Neural Networks.
Back propagation is a multi-layer feed forward, supervised learning network based on gradient descent learning rule.
we provide the algorithm with examples of the inputs and outputs we want the network to compute, and then the error (difference between actual and expected results) is calculated.
The idea of the backpropagation algorithm is to reduce this error, until the Artificial Neural Network learns the training data.

18. Basic Block of Back propagation
The activation function of the artificial neurons in ANNs implementing the backpropagation algorithm is a weighted sum (the sum of the inputs xi multiplied by their respective weights wji)
The most common output function is the sigmoidal function:
Since the error is the difference between the actual and the desired output, the error depends on the weights, and we need to adjust the weights in order to minimize the error. We can define the error function for the output of each neuron:
Inputs, x
Weights, v weights, w
output
Basic Block of Back propagation
neural network

19. First, we need to calculate how much the error depends on the output
CONTD…
The backpropagation algorithm now calculates how the error depends on the output, inputs, and weights. The adjustment of each weight (Δwji ) will be the negative of a constant eta (η) multiplied by the dependence of the “wji” previous weight on the error of the network.
First, we need to calculate how much the error depends on the output
Next, how much the output depends on the activation, which in turn depends on the weights
And so, the adjustment to each weight will be

20. CONTD…
If we want to adjust vik, the weights (let’s call them vik ) of a previous layer, we need first to calculate how the error depends not on the weight, but in the input from the previous layer i.e. replacing w by x as shown in below equation.
where
and
Inputs, x
Weights, v weights, w
output

21. APPLICATIONS Neural networks in medicine Neural Networks in business
Character Recognition - The idea of character recognition has become very important as handheld devices like the Palm Pilot are becoming increasingly popular. Neural networks can be used to recognize handwritten characters. Image Compression - Neural networks can receive and process vast amounts of information at once, making them useful in image compression. With the Internet explosion and more sites using more images on their sites, using neural networks for image compression is worth a look. Stock Market Prediction - The day-to-day business of the stock market is extremely complicated. Many factors weigh in whether a given stock will go up or down on any given day. Since neural networks can examine a lot of information quickly and sort it all out, they can be used to predict stock prices. Traveling Saleman's Problem - Interestingly enough, neural networks can solve the traveling salesman problem, but only to a certain degree of approximation. Medicine, Electronic Nose, Security, and Loan Applications - These are some applications that are in their proof-of-concept stage, with the acception of a neural network that will decide whether or not to grant a loan, something that has already been used more successfully than many humans. Miscellaneous Applications - These are some very interesting (albeit at times a little absurd) applications of neural networks.
Neural networks in medicine
Modelling and Diagnosing the Cardiovascular System
Electronic noses
Instant Physician
Neural Networks in business
Marketing
Credit Evaluation

22. ADVANTAGES It involves human like thinking.
They handle noisy or missing data.
They can work with large number of variables or parameters.
They provide general solutions with good predictive accuracy.
System has got property of continuous learning.
They deal with the non-linearity in the world in which we live.

23. CONCLUSION
Artificial neural networks are inspired by the learning processes that take place in biological systems.
Artificial neurons and neural networks try to imitate the working mechanisms of their biological counterparts.
Learning can be perceived as an optimisation process.
Biological neural learning happens by the modification of the synaptic strength. Artificial neural networks learn in the same way.
The synapse strength modification rules for artificial neural networks can be derived by applying mathematical optimisation methods.

24. CONT…
Learning tasks of artificial neural networks can be reformulated as function approximation tasks.
Neural networks can be considered as nonlinear function approximating tools (i.e., linear combinations of nonlinear basis functions), where the parameters of the networks should be found by applying optimisation methods.
The optimisation is done with respect to the approximation error measure.
In general it is enough to have a single hidden layer neural network (MLP, RBF or other) to learn the approximation of a nonlinear function. In such cases general optimisation can be applied to find the change rules for the synaptic weights.

25. Thanks