Mehdi Ghayoumi MSB rm 132 Ofc hr: Thur, a Machine Learning

HW 1 is available due to Wednesday 02-18

Machine Learning A neural network isn’t magic. It’s not going to be able to guess anything correctly unless we teach it how to!

Machine Learning

The process is as follows: 1.Provide the perceptron with inputs for which there is a known answer. 2.Ask the perceptron to guess an answer. 3.Compute the error. 4.Adjust all the weights according to the error. 5.Return to Step 1 and repeat!

Perceptron Learning Rule Update weights by: where η is the “learning rate” r i is the teacher specified output for unit i. Machine Learning

Equivalent to rules: –If output is correct do nothing. –If output is high, lower weights on active inputs –If output is low, increase weights on active inputs Machine Learning

Since perceptron uses linear threshold function, it is searching for a linear separator that discriminates the classes. o3o3 o2o2 ?? Machine Learning

Perceptron convergence theorem: If : 1. The data is linearly separable 2. A set of weights exist that are consistent with the data, then The Perceptron algorithm will eventually converge to a consistent set of weights. Machine Learning

Perceptron cycling theorem: If The data is not linearly separable, Then the Perceptron algorithm will eventually repeat a set of weights and threshold at the end of some epoch and therefore enter an infinite loop. Machine Learning

where ρ is the parameter controlling convergence, and x (t) denotes the point considered in the t iteration. The class labels y(t) are equal to −1 and +1 for the two classes ω2 and ω1, respectively. A pseudo code for this scheme is given as:

Machine Learning

Example Generate four 2-dimensional data sets Xi, i = 1,...,4, each containing data vectors from two classes. In all Xi’s The first class (denoted −1) contains 100 vectors uniformly distributed in the square [0, 2]×[0, 2]. The second class (denoted +1) contains another 100 vectors uniformly distributed in the squares [3, 5]×[3, 5], [2, 4]×[2, 4], [0, 2]×[2, 4], and [1, 3]×[1, 3] for X1, X2, X3, and X4, respectively. Each data vector is augmented with a third coordinate that equals 1.

Machine Learning 1. Plot the four data sets and notice that as we move from X1 to X3 the classes approach each other but remain linearly separable. In X4 the two classes overlap.

Machine Learning 2. Run the perceptron algorithm for each Xi, i = 1,...,4, with learning rate parameters 0.01 and 0.05 and initial estimate for the parameter vector [1, 1, −0.5]T. 3. Run the perceptron algorithm for X3 with learning rate 0.05 using as initial estimates for w [1, 1, −0.5]T and [1, 1, 0.5]T. w = iter = mis_clas = 25

Feeding data through the net: (1  0.25) + (0.5  (-1.5)) = (-0.75) = Machine Learning

Price of meal = 850 portions of fish portions of chips portions of beer Linear neuron Machine Learning

NEW WEIGHT = WEIGHT + ΔWEIGHT ΔWEIGHT = ERROR * INPUT NEW WEIGHT = WEIGHT + ERROR * INPUT NEW WEIGHT = WEIGHT + ERROR * INPUT * LEARNING CONSTANT

Machine Learning ERROR = DESIRED OUTPUT - GUESS OUTPUT DesiredGuessError

x+y-2 1 * -2 y * +1 x * +1 if sum<0 : 0 else : 1 inputs weights sum output x & yyx output inputs Truth Table for Logical AND Machine Learning

Learning Boolean AND

x+y-1 1 * -1 y * +1 x * +1 if sum<0 : 0 else : 1 inputs weights sum output x | yyx output inputs Truth Table for Logical OR Machine Learning

Thank you!