Discriminative Training of Chow-Liu tree Multinet Classifiers Huang, Kaizhu Dept. of Computer Science and Engineering, CUHK

Outline Background Motivation Classifiers Discriminative classifiers Generative classifiers Bayesian Multinet Classifiers Motivation Discriminative Bayesian Multinet Classifiers Experiments Conclusion

Discriminative Classifiers Directly maximize a discriminative function SVM

Generative Classifiers Estimate the distribution for each class, and then use Bayes rule to perform classification P1(x|C1) P2(x|C2)

Comparison Example of Missing Information: From left to right: Original digit, Cropped and resized digit, 50% missing digit, 75% missing digit, and occluded digit.

Comparison (Continue) Discriminative Classifiers cannot deal with missing information problems easily. Generative Classifiers provide a principled way to handle missing information problems. When is missing, we can use Marginalized P1 and P2 to perform classification

Handling Missing Information Problem SVM TJT: a generative model

Motivation It seems that a good classifier should combine the strategies of discriminative classifiers and generative classifiers Our work trains the one of the generative classifier: the generative Bayesian Multinet classifier in a discriminative way

Roadmap of our work

How our work relates to other work? Discriminative Classifiers Generative Classifiers 1. Jaakkola and Haussler NIPS98 Difference: Our method performs a reverse process: From Generative classifiers to Discriminative classifiers HMM and GMM Discriminative training 2. Beaufays etc., ICASS99, Hastie etc., JRSS 96 Difference: Our method is designed for Bayesian Multinet Classifiers, a more general classifier.

Problems of Bayesian Multinet Classifiers Pre-classified dataset Sub-dataset D1 for Class I Sub-dataset D2 for Class 2 Estimate the distribution P1 to approximate D1 accurately Estimate the distribution P2 to approximate D2 accurately Use Bayes rule to perform classification Comments: This framework discards the divergence information between classes.

Our Training Scheme

Mathematic Explanation Bayesian Multinet Classifiers (BMC) Discriminative Training of BMC

Mathematic Explanation

Finding P1 and P2

Finding P1 and P2

Experimental Setup Datasets Experimental Environments 2 benchmark datasets from UCI machine learning repository Tic-tac-toe Vote Experimental Environments Platform:Windows 2000 Developing tool: Matlab 6.5

Error Rate

Convergence Performance

Conclusion A discriminative training procedure for generative Bayesian Multinet Classifiers is presented This approach improves the recognition rate for two benchmark datasets significantly The theoretic exploration on the convergence performance of this approach is on the way.