1. Mehdi Ghayoumi MSB rm 132 mghayoum@kent.edu Ofc hr: Thur, 11-12 a Machine Learning

2. Grader: Safa Shubbar sshubbar@kent.edu Please, Send Her your Group members and your project name due to this weekend

3. Machine Learning  1.Vision: Face recognition Facial expression recognition Object tracking 2.Big Data: Data mining Streaming data over the Internet Fraud detection

4. Maximum Likelihood (ML) Machine Learning

5. Maximum Likelihood (ML) Any hypothesis that maximizes P(d|h) is called the maximum likelihood hypothesis Machine Learning

6. An Illustrating Example Classifying days according to whether someone will play tennis. Each day is described by the attributes, Outlook, Temperature, Humidity and Wind. Based on the training data in the table, classify the following instance: Outlook = sunny Temperature = cool Humidity = high Wind = strong Machine Learning

7. Training sample pairs (X,D) where: X = (x 1, x 2, …, x n ) is The feature vector representing the instance. D= (d 1, d 2, … d m ) is The desired (target) output of the classifier Machine Learning

8. Training sample pairs (X, D) X = (x 1, x 2, …, x n ) is the feature vector representing the instance. Here n = 4 x 1 = outlook = {sunny, overcast, rain} x 2 = temperature = {hot, mild, cool} x 3 = humidity = {high, normal} x 4 = wind = {weak, strong} D= (d 1, d 2, … d m ) is the desired output of the classifier Here m = 1 d= Play Tennis = {yes, no} Machine Learning

9. Bayesian Classifier The Bayesian approach to classifying a new instance X is to assign it to the most probable target value Y (MAP classifier) Machine Learning

10. 10 P(d i ) is easy to calculate: simply counting how many times each target value d i occurs in the training set P(d = yes) = 9/14 P(d = no) = 5/14 Machine Learning

11. P(x 1, x 2, x 3, x 4 |d i ) is much more difficult to estimate. In this simple example, there are: 3x3x2x2x2 = 72 possible terms to obtain a reliable estimate, we need to see each terms many times, Hence, we need a very, very large training set! Machine Learning

12. Naïve Bayes Classifier Naïve Bayes classifier is based on the simplifying assumption that the attribute values are conditionally independent given the target value. This means, we have Naïve Bayes Classifier Machine Learning

14. P(d=yes)=9/14 = 0.64 P(d=no)= 5/14 = 0.36 P(x 1 =sunny|yes)=2/9 P(x 1 =sunny|no)=3/5 P(x 2 =cool|yes)= P(x 2 =cool|no)= P(x 3 =high|yes)= P(x 3 =high|no)= P(x 4 =strong|yes)= P(x 4 =strong|no)= Machine Learning

15. Bayesian Learning Machine Learning

16. BAYES DECISION THEORY We are given a pattern whose class label is unknown and we let x≡[x(1),x(2),...,x(l)] T ∈ R l be Its corresponding feature vector, which results from some measurements. Also we let the number of Possible classes be equal to c, that is,ω1,...,ωc.

17. Machine Learning According to the Bayes decision theory, x is assigned to the class ωi iif: P(ωi|x)>P(ωj|x), ∀ j#i or, taking into account above Eq and given that p(x) is positive and the same for all classes, if: p(x|ωi)P(ωi)>p(x|ωj)P(ωj), ∀ j#i

18. Machine Learning THE GAUSSIAN PROBABILITY DENSITY FUNCTION The Gaussian pdf is extensively used here because of: 1.Its Mathematical tractability as well as because of the central limit theorem. 2.The latter states that the pdf Of the sum of a number of statistically independent random variables tends to the Gaussian one as the Number of summands tends to infinity. 3.In practice, this is approximately true for a large enough number Of summands. The multidimensional Gaussian pdf has the form

19. Machine Learning Where m=E[x] is the mean vector, S is the covariance matrix defined as S=E[(x−m)(x−m)T], |S| is the determinant of S. Often we refer to the Gaussian pdf as the normal pdf and we use the notation N(m,S). For the 1-dimensional case,x ∈ R, the above becomes

20. Machine Learning Example: Compute the value of a Gaussian pdf, N(m,S),at x1=[0.2,1.3] T and x2=[2.2,−1.3] T, where: Z=? x1=[0.2 1.3]'; x2=[2.2 -1.3]'; z1 = 0.1491 z2 = 0.0010

21. Machine Learning Example: Consider a 2-class classification task in the 2-dimensional space, where the data in both classes, ω 1, ω 2, are distributed according to the Gaussian distributions N(m 1,S 1 ) and N(m 2,S 2 ), respectively. Assuming that P(ω1) = P(ω2) = 1/2, classify x = [1.8, 1.8]T into ω1 or ω2. Use: p(x|ωi)P(ωi)>p(x|ωj)P(ωj), ∀ j#i

22. Machine Learning The resulting values for p1 and p2 are 0.042 and 0.0189, respectively, and x is classified to ω1 according to the Bayesian classifier. p(x|ωi)P(ωi)>p(x|ωj)P(ωj), ∀ j#i

23. Thank you!