1. Mehdi Ghayoumi MSB rm 132 mghayoum@kent.edu Ofc hr: Thur, 11-12 a Machine Learning

2. HW 1 is available due to Wednesday 02-18

3. Machine Learning A neural network isn’t magic. It’s not going to be able to guess anything correctly unless we teach it how to!

4. Machine Learning

10. The process is as follows: 1.Provide the perceptron with inputs for which there is a known answer. 2.Ask the perceptron to guess an answer. 3.Compute the error. 4.Adjust all the weights according to the error. 5.Return to Step 1 and repeat!

11. Perceptron Learning Rule Update weights by: where η is the “learning rate” r i is the teacher specified output for unit i. Machine Learning

12. Equivalent to rules: –If output is correct do nothing. –If output is high, lower weights on active inputs –If output is low, increase weights on active inputs Machine Learning

13. Since perceptron uses linear threshold function, it is searching for a linear separator that discriminates the classes. o3o3 o2o2 ?? Machine Learning

15. Perceptron convergence theorem: If : 1. The data is linearly separable 2. A set of weights exist that are consistent with the data, then The Perceptron algorithm will eventually converge to a consistent set of weights. Machine Learning

16. Perceptron cycling theorem: If The data is not linearly separable, Then the Perceptron algorithm will eventually repeat a set of weights and threshold at the end of some epoch and therefore enter an infinite loop. Machine Learning

17. where ρ is the parameter controlling convergence, and x (t) denotes the point considered in the t iteration. The class labels y(t) are equal to −1 and +1 for the two classes ω2 and ω1, respectively. A pseudo code for this scheme is given as:

18. Machine Learning

19. Example Generate four 2-dimensional data sets Xi, i = 1,...,4, each containing data vectors from two classes. In all Xi’s The first class (denoted −1) contains 100 vectors uniformly distributed in the square [0, 2]×[0, 2]. The second class (denoted +1) contains another 100 vectors uniformly distributed in the squares [3, 5]×[3, 5], [2, 4]×[2, 4], [0, 2]×[2, 4], and [1, 3]×[1, 3] for X1, X2, X3, and X4, respectively. Each data vector is augmented with a third coordinate that equals 1.

20. Machine Learning 1. Plot the four data sets and notice that as we move from X1 to X3 the classes approach each other but remain linearly separable. In X4 the two classes overlap.

21. Machine Learning 2. Run the perceptron algorithm for each Xi, i = 1,...,4, with learning rate parameters 0.01 and 0.05 and initial estimate for the parameter vector [1, 1, −0.5]T. 3. Run the perceptron algorithm for X3 with learning rate 0.05 using as initial estimates for w [1, 1, −0.5]T and [1, 1, 0.5]T. w = 0.0146 0.0161 -0.0419 iter = 20000 mis_clas = 25

22. Feeding data through the net: (1  0.25) + (0.5  (-1.5)) = 0.25 + (-0.75) = - 0.5 Machine Learning

23. Price of meal = 850 portions of fish portions of chips portions of beer 150 50 100 2 5 3 Linear neuron Machine Learning

24. NEW WEIGHT = WEIGHT + ΔWEIGHT ΔWEIGHT = ERROR * INPUT NEW WEIGHT = WEIGHT + ERROR * INPUT NEW WEIGHT = WEIGHT + ERROR * INPUT * LEARNING CONSTANT

25. Machine Learning ERROR = DESIRED OUTPUT - GUESS OUTPUT DesiredGuessError 0 +1-2 +1+2 +1 0

26. x+y-2 1 * -2 y * +1 x * +1 if sum<0 : 0 else : 1 inputs weights sum output 111 01 01 00 0 0 0 x & yyx output inputs Truth Table for Logical AND Machine Learning

27. Learning Boolean AND

28. x+y-1 1 * -1 y * +1 x * +1 if sum<0 : 0 else : 1 inputs weights sum output 111 01 01 00 1 1 0 x | yyx output inputs Truth Table for Logical OR Machine Learning

29. Thank you!