1. Naive Bayes Classifier Christopher Gonzalez

2. Outline Bayes’ Theorem What is a Naive Bayes Classifier (NBC)? Why/when to use NBC? How does NBC work? Applications of NBC Spam Filtering Paper Where do NBCs fail?

3. Bayes’ Theorem P(A) is the probability of A P(B) is the probability of B P(B|A) is the probability of B given A is true P(A|B) is the probability of A given B is true

4. What is a Naive Bayes Classifier (NBC)? A family of probabilistic classifiers in machine learning Relies heavily on Bayes Theorem Introduced in early 60s Uses a training set of data and a features list to determine how to classify a given object

5. Why/When to use NBC? When you can assume that the feature set is independent Easy to understand and extremely effective Works well even with limited sized training set Quick to implement When the feature set is dependent (see The Optimality of Naive Bayes by Harry Zhang)

6. How does NBC work?

8. Example Training Set FeverCoughHeadacheFlu YNNY YNNY YYYY YYNN NYNN NNYN YYNY NYNY Priors: P(Flu)=5/8 P(!Flu)=3/8 P(!fever|flu) = 1/5 P(cough|flu) = 1/5 P(headache|flu) = 1/5 P(flu |!fever,cough,headache) = 5/8 * 1/5 * 1/5 * 1/5 = 0.005 P(!fever|!flu) = 1/3 P(cough|!flu) = 2/3 P(headache|!flu) = 1/3 P(!flu |!fever,cough,headache) = 3/8 * 1/3 * 2/3 * 1/3 = 0.027 Does someone have the flu if they have no fever but have a cough and headache?

9. Applications of NBC Text Classification Hybrid Recommender Systems Emotional Modeling Medical Diagnosis Spam Filtering

10. Spam Filtering Paper

11. Spam Filtering Results

12. Where do NBCs Fail? Better solutions for dependent feature sets If you have a large amount of feature, NBC wont tell you which features to use and not use Continuous variables Zero frequency problem

13. Questions