Perceptron Networks and Vector Notation n CS/PY 231 Lab Presentation # 3 n January 31, 2005 n Mount Union College

A Multiple Perceptron Network for computing the XOR function n We found that a single perceptron could not compute the XOR function n Solution: set up one perceptron to detect if x 1 = 1 and x 2 = 0 n set up another perceptron for x 1 = 0 and x 2 = 1 n feed the outputs of these two perceptrons into a third one that produces an output of 1 if either input is a 1

A Nightmare! n Even for this simple example, choosing the weights that cause a network to compute the desired output takes skill and lots of patience n Much more difficult than programming a conventional computer: OR function: if x1 + x2 > 1, output 1; otherwise output 0 XOR function: if x1 + x2 = 1, output 1; otherwise output 0

There must be a better way…. n These labs and demos were designed to show that manually adjusting weights is tedious and difficult n This is not what happens in nature –No creature says, “Hmmm, what weight should I choose for this neural connection?” n Formal training methods exist that allow networks to learn by updating weights automatically (explored next week)

Expanding to More Inputs n artificial neurons may have many more than two input connections n calculation performed is the same: multiply each input by the weight of the connection, and find the sum of all of these products n notation can become unwieldy: sum = x 1 ·w 1 + x 2 ·w 2 + x 3 ·w 3 + … + x 100 ·w 100

Some Mathematical Notation n Most references (e.g., Plunkett & Elman text) use mathematical summation notation n Sums of large numbers of terms are represented by the symbol Sigma (  ) –previous sum is denoted as: 100  x k ·w k k = 1

Summation Notation Basics n Terms are described once, generally n Index variable shows range of possible values n Example: 5  k / (k - 1) = 3/2 + 4/3 + 5/4 k = 3

Summation Notation Example n Write the following sum using Sigma notation: 3·x 0 + 4·x 1 + 5·x 2 + 6·x 3 + 7·x 4 + 8·x 5 + 9·x ·x 7 n Answer: 7  (k + 3) ·x k k = 0

Vector Notation n The most compact way to specify values for inputs and weights when we have many connections n The ORDER in which values are specified is important n Example: if w 1 = 3.5, w 2 = 1.74, and w 3 = 18.2, we say that the weight vector w = (3.5, 1.74, 18.2)

Vector Operations n Vector Addition: adding two vectors means adding the values from the same position in each vector –result is a new vector n Example: (9.2, 0, 17) + (1, 2, 3) = (10.2, 2, 20) n Vector Subtraction: subtract corresponding values n (9.2, 0, 17) - (1, 2, 3) = (8.2, -2, 14)

Vector Operations n Dot Product: mathematical name for what a perceptron does n x · m = x 1 ·m 1 + x 2 ·m 2 + x 3 ·m 3 + … + x last ·m last n Result of a Dot Product is a single number n example: (9.2, 0, 17) · (1, 2, 3) = = 60.2

Perceptron Networks and Vector Notation n CS/PY 231 Lab Presentation # 3 n January 31, 2005 n Mount Union College