Start with student evals

What function does perceptron #4 represent?

What function does perceptron #5 represent?

Each neuron has a threshold value Each neuron has weighted inputs from other neurons The input signals form a weighted sum If the activation level exceeds the threshold, the neuron “fires” The McCulloch-Pitts “Unit” The single neuron (Perceptron)

“Simple” Learning algorithm While epoch produces an error Present network with next inputs from epoch Error = T – O If Error <> 0 then W j = W j + LR * I j * Error End If End While

Training Perceptrons t = 0.0 y x W0 = ? W2 = ? W1 = ? For AND x y Output What are the weight values?What are the weight values? Initialize with random weight valuesInitialize with random weight values

Training Perceptrons t = 0.0 y x W 0 = 0.3 W 2 =-0.4 W 1 = 0.5 For AND x y Output

“Simple” Learning algorithm While epoch produces an error Present network with next inputs from epoch Error = T – O If Error <> 0 then W j = W j + LR * I j * Error End If End While

Training Perceptrons t = 0.0 y x W0 = 0.3 W2 =-0.4 W1 = 0.5 For AND x y Output

“Simple” Learning algorithm While epoch produces an error Present network with next inputs from epoch Error = T – O If Error <> 0 then W j = W j + LR * I j * Error End If End While

Training Perceptrons t = 0.0 y x W 0 = 0.3 W 2 =-0.4 W 1 = 0.5 For AND x y Output

“Simple” Learning algorithm While epoch produces an error Present network with next inputs from epoch Error = T – O If Error <> 0 then W j = W j + LR * I j * Error End If End While

Training Perceptrons t = 0.0 y x W 0 = 0.3 W 2 =-0.4 W 1 = 0.5 For AND x y Output If Error <> 0 then Wj = Wj + LR * Ij * Error

Learning in Neural Networks Will demo the online applet at New38.html

Decision boundaries In simple cases, divide feature space by drawing a hyperplane across it. Known as a decision boundary. Discriminant function: returns different values on opposite sides. (straight line) Problems which can be thus classified are linearly separable.

Decision Surface of a Perceptron x1x1 x2x x1x1 x2x2 Perceptron is able to represent some useful functions AND(x 1,x 2 ) choose weights w 0 =-1.5, w 1 =1, w 2 =1 But functions that are not linearly separable (e.g. XOR) are not representable Linearly separableNon-Linearly separable

How do the weights define the boundary line? Consider the y-intercept X value is zero so I1 is zero and that term drops out. We are interested in I2 which in this case is the y coordinate on the boundary line So the boundary line is on I0*W0 + I2*W2 = 0 -1*W0 + y*W2 = 0 -W0 + y*W2 = 0 y*W2 = W0 y = W0/W2

How do the weights define the boundary line? Via similar math, the x-intercept is where y value is zero so I2 is zero and that term drops out. We are interested in I1 which in this case is the x coordinate on the boundary line So the boundary line is on I0*W0 + I1*W1 = 0 -1*W0 + x*W1 = 0 -W0 + x*W1 = 0 x*W1 = W0 x = W0/W1

How do the weights define the boundary line? Slope is defined as (y1 – y2) / (x1-x2) If we think of point one as the y intercept (0,w2/w0) and point two as the x intercept (w1/w0,0) Slope = (w2/w0 – 0) / (0 – w1/w0) = (w2/w0) / (-w1 / w0) = -(w2/w1)