DEPARTMENT: COMPUTER SC. & ENGG. SEMESTER : VII COURSE NEURAL NETWORKS SATENDRA Assistant Professor SITM,REWARI . Manav Rachna College of Engineering

TOPIC Fundamental concepts of Artificial Neural Networks Content FeedForward & Feedback Networks Learning Rules: Supervised & Unsupervised Hebbian Learning Rule, Perception Learning Rule, Delta Learning Rule, Widrow-Hoff Learning Rule Correction Learning Rule Winner –Take All Learning Rule Manav Rachna College of Engineering

Neural Network Architectures

Neural Network Learning Process

Neural Network Learning Rules The learning signal r in general a function of wi, x and sometimes of teacher’s signal di. Incremental weight vector wi at step t becomes: Where c is a learning constant having +ve value.

NN Learning Rule 1) Hebbian Learning Rule The Hebbian learning rule represents a purely feedforward, unsupervised learning. This learning rule requires the weight initialization at small random values around wi = 0 prior to learning. Here, Learning signal is as neuron’s output. Incremental weight vector becomes:- For single weight wij

2) Perceptron Learning Rule:- For the perceptron learning rule, the learning signal is between the desired and actual neuron's response. This learning is supervised learning. This rule is applicable for binary neuron response and relationships express the rule for the bipolar binary case. Learning Signal r becomes Weight adjustment becomes:-

Where η is a positive constant. 3) Delta Training Rule This rule valid for only continuous activation functions It is of supervised training mode. The learning signal of this rule is called delta and defined as Calculating the gradient vector with respect to wi of the squared error defined as Error gradient vector Since minimization of error requires the weight changes to be in the negative gradient direction, therefore Where η is a positive constant.

4) Windrow-Hoff Learning Rule The Widrow-Hoff learning is applicable for the supervised training of neural networks. It is independent of the activation function of neurons used since it minimizes the squared error between the desired output value d, and the neuron's activation value net The learning rule is defined as The weight vector increment under this learning rule is for the single weight the adjustment is This rule is also called LMS (least Mean square) learning rule. Weights are initialized at any values in this method.

5) Correlation Learning Rule This rule is typically applied for recording data in memory networks with binary response neurons. This is supervised learning rule. This learning rule also requires the weight initialization w = 0. As we know, general learning rule is :- And by substitution r=di into general rule, we get correlation rule. The adjustment for weight vector and single weights resp. are

6) Winner –Take- All Learning Rule It is used for unsupervised network training. The learning is based on the premise that one of the neurons in the layer say m’th has maximum response due to input x. This neuron is declared as Winner. As a result of this winning event, the weight vector Wm becomes Incremental weight adjustment Where α > 0 is a small learning constant. The winner selection is based on the following criterion of maximum activation among all p neurons participating in a competition

Thank you