1. 29 August 2013 Venkat Naïve Bayesian on CDF Pair Scores

2. Outline Naïve Bayesian Overview Adapting Naïve Bayesian to CDF Pairscores Comparisons with Logistic Regression Comparisons with the Voting Scheme

3. Bayesian Framework

4. Unbiased Learning requires O(NK d ) samples for reasonable parameter estimation Impractical for most values of d

5. Naïve Bayes Assumption Let The Naïve Bayes Assumption implies class conditional independence Requires O(NK) samples

6. Gaussian Naïve Bayes What are the parameters to be estimated ? N Priors Nd Likelihood functions

7. Naïve Bayes on CDF Pairscores Direct application of GNB on CDF Pairscores guaranteed to give poor results. Must make use of which features are irrelevant conditioned on a class. For instance, conditioned on class 7, the score for the say class-36-vs-class-9 model is irrelevant.

8. Naïve Bayes on CDF Pairscores We have N=50 classes and d = 1225 pairscores

9. Naïve Bayes on CDF Pairscores

15. Likelihood Distributions

16. P(s(c,c’)|y=c ) P(s(c’,c)|y=c )

17. Results (2 nd level) Naïve Bayesian : 57.18% Voting : 59.01% Logistic Regression : 57.51% So, which is the overall best scheme ??

18. Naïve Bayes vs Logistic Regression GNB (generative) and LR (discriminative) essentially model the same classifier when Naïve Bayesian Assumptions hold. However, LR converges to asymptotic accuracies slower than GNB This is due to LR requiring exponentially higher number of samples compared to GNB for good parameter estimates

19. Naïve Bayes vs Logistic Regression

20. LOGISTIC REGRESSION

21. Naïve Bayes vs Logistic Regression NAÏVE BAYESIAN

22. Naïve Bayes vs Logistic Regression NAÏVE BAYESIAN

23. Naïve Bayes vs Logistic Regression When training data is scarce, GNB theoretically outperforms LR Moreover, if LR only marginally outperforms GNB, then GNB should still be chosen due to its low variance property.

24. Naïve Bayes vs Voting Scheme Naïve Bayes is equivalent to a weighted voting scheme. Unweighted voting scheme takes unbiased votes from pairwise models, ignoring scores and scales. The binary structure of the unweighted scheme has ill- defined bias-variance properties. One can argue that it just happens to work well in this case.