1. Discriminative Training of Chow-Liu tree Multinet Classifiers
Huang, Kaizhu
Dept. of Computer Science and Engineering,
CUHK

2. Outline Background Motivation
Classifiers
Discriminative classifiers
Generative classifiers
Bayesian Multinet Classifiers
Motivation
Discriminative Bayesian Multinet Classifiers
Experiments
Conclusion

3. Discriminative Classifiers
Directly maximize a discriminative function
SVM

4. Generative Classifiers
Estimate the distribution for each class, and then use Bayes rule to perform classification
P1(x|C1)
P2(x|C2)

5. Comparison Example of Missing Information:
From left to right: Original digit, Cropped and resized digit, 50% missing digit, 75% missing digit, and occluded digit.

6. Comparison (Continue)
Discriminative Classifiers cannot deal with missing information problems easily.
Generative Classifiers provide a principled way to handle missing information problems.
When is missing, we can use Marginalized P1 and P2 to perform classification

7. Handling Missing Information Problem
SVM
TJT: a generative model

8. Motivation
It seems that a good classifier should combine the strategies of discriminative classifiers and generative classifiers
Our work trains the one of the generative classifier: the generative Bayesian Multinet classifier in a discriminative way

9. Roadmap of our work

10. How our work relates to other work?
Discriminative Classifiers
Generative Classifiers 1.
Jaakkola and Haussler NIPS98
Difference: Our method performs a reverse process:
From Generative classifiers to Discriminative classifiers
HMM and GMM
Discriminative training 2.
Beaufays etc., ICASS99, Hastie etc., JRSS 96
Difference: Our method is designed for Bayesian Multinet Classifiers, a more general classifier.

12. Problems of Bayesian Multinet Classifiers
Pre-classified dataset
Sub-dataset D1
for Class I
Sub-dataset D2
for Class 2
Estimate the distribution P1 to
approximate D1 accurately
Estimate the distribution P2 to
approximate D2 accurately
Use Bayes rule to
perform classification
Comments: This framework discards the divergence information between classes.

13. Our Training Scheme

14. Mathematic Explanation
Bayesian Multinet Classifiers (BMC)
Discriminative Training of BMC

15. Mathematic Explanation

16. Finding P1 and P2

17. Finding P1 and P2

18. Experimental Setup Datasets Experimental Environments
2 benchmark datasets from UCI machine learning repository
Tic-tac-toe
Vote
Experimental Environments
Platform:Windows 2000
Developing tool: Matlab 6.5

19. Error Rate

20. Convergence Performance

21. Conclusion
A discriminative training procedure for generative Bayesian Multinet Classifiers is presented
This approach improves the recognition rate for two benchmark datasets significantly
The theoretic exploration on the convergence performance of this approach is on the way.