Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer

Bab /21 Artificial Neural Networks (ANN) / 1 Output Y is 1 if at least two of the three inputs are equal to 1.

Bab /21 Artificial Neural Networks (ANN) / 2

Bab /21 Artificial Neural Networks (ANN) / 3 l Model is an assembly of inter-connected nodes and weighted links l Output node sums up each of its input value according to the weights of its links l Compare output node against some threshold t Perceptron Model or

Bab /21 General Structure of ANN Training ANN means learning the weights of the neurons

Bab /21 Algorithm for Learning ANN l Initialize the weights (w 0, w 1, …, w k ) l Adjust the weights in such a way that the output of ANN is consistent with class labels of training examples –Objective function: –Find the weights w i ’s that minimize the above objective function  e.g., backpropagation algorithm

Bab /21 Artificial Neural Networks (ANN) / 2

Bab /21 Perceptron

Bab /21 l Let D = {(x i, y i ) | i= 1,2,…,N} be the set of training examples l Initialize the weights l Repeat –For each training example (x i, y i ) do  Compute f(w, x i )  For each weight w j do Update the weight l Until stopping condition is met Perceptron Learning Rule / 1

Bab /21 l Weight update formula: l Intuition: –Update weight based on error –If y = f(w,x), e = 0, no update is needed –If y > f(w,x), e = 2, weight must be increased so that f(w,x) will increase –If y < f(w,x), e = -2, weight must be decreased so that f(w,x) will decrease Perceptron Learning Rule / 2

Bab /21 l Terminating condition: Training stops when either 1. all  w ij in the previous epoch (i.e., iteration) were so small as to be below some specified threshold, or 2. the percentage of samples misclassified in the previous epoch is below some threshold, or 3. a pre-specified number of epochs has expired. In practice, several hundreds of thousands of epochs may be required before the weights will converge Perceptron Learning Rule / 3

Bab /21 Example of Perceptron Learning

Bab /21 Perceptron Learning

Bab /21 Nonlinearly Separable Data

Bab /21 Multilayer Neural Network / 1

Bab /21 Multilayer Neural Network / 2

Bab /21 Learning Multilayer Neural Network

Bab /21 Gradient Descent for Multilayer NN / 1

Bab /21 Gradient Descent for Multilayer NN / 2

Bab /21 Design Issues in ANN

Bab /21 Characteristics of ANN