1. Universum Support Vector Machine -A generalized approach Junfeng He with help from Professor Tony Jebara, Gerry Tesauro and Vladimir Naumovich Vapnik

2. SVM for Classification

3. Universum SVM for Classification Idea: Contradiction on Universum

4. Universum SVM for Classification Approximation: If is close to zero, then a small change in will cause a contradiction on universum data

5. Universum SVM for Classification

6. Dual form: (With U as ε-insenstive function)

7. Problem Only suitable for two-label classification Can we generalize universum SVM to both classification and regression?

8. Idea View regression as many two-label classification problems: For any given y, For this two-lable classification problem, using the idea of universum SVM, the loss function should be: With all possible y, the total loss function on universum data:

9. Generalized Universum Support Vector Machine For two classification, i.e., y = {+1,-1}, if p(y=+1)=p(y=-1) = 0.5, degenerated as Universum SVM:

11. Generalized Support Vector Machine

12. Dual form Replacing by, we get the kernel version.

13. Property Suitable for both classification and regresson. Without the universum part traditional SVR. Sparse in training data, not sparse in universum data ( because of loss function).

14. L 2 version

16. Dual form

17. Property Suitable for both regression and classification. Without the universum part LS-SVM. For classification y={+1,-1}, if E = 0, degenerated to Universum LS-SVM [Fabian Sinz 2007].

18. Property Not sparse in training or universum data. Because of loss function: It can be used for online learning. can be computed based on

19. Experiments - male/female face classification Yale Face Dataset Training: male 250 female168 Test: male 171 female 168 Universum: 1700. Created by: a * male + (1-a) * female Classification Error on Test Set a SVMLS- SVM Universum LS- SVM (i.e.,E=0) Our result (L2 version) 0.50.22120.20940.23300.1858 (E = 0.2) 0.70.22120.20940.33330.1799 (E = 0.6) 0.10.22120.20940.43070.2006 (E=-0.6)

20. More experiments Coming soon…

21. Thank You! 谢谢！ ありがとう ！ Vielen Dank ！ Kop Koon Ka! 謝謝！ Merci beaucoup ！ 감사합니다 ！ Spasiba ！ Ευχαριστίες ! شكور ！ Grazias ！ Köszönöm ！ Obrigado ！

22. Q & A?