1. Learning 2006 - Vilanova Multi-classification by using tri- class SVM Luis González and Francisco Velasco Appl. Economics – University of Sevilla Cecilio Angulo GREC – Technical Univ. of Catalonia

2. Learning 2006 - Vilanova Outline Bi-class SV Machine Multi-class SV approaches Tri-class SV Machine First approach Second approach Third approach Results Conclusions

3. Learning 2006 - Vilanova We are looking for a solution in the form with largest geometrical margin for a bi-class problem Bi-class SV Machine

4. Learning 2006 - Vilanova Associated QP problem Bi-class SV Machine

5. Learning 2006 - Vilanova Multi-class SV approaches In pairs – 1 versus 1 (pair wise coupling): too specialized – 1 versus rest (one against all): too general All the data at once – Different approaches, but all them are equivalents – Ordinal regression problem (Shashua-Levin 2002)

6. Learning 2006 - Vilanova Tri-class SV Machine – Data is partitioned in three classes Two classes are considered +1, -1 A third ‘class 0’ is generated by the rest of the patterns – Associated QP problem First approach K-SVCR

7. Learning 2006 - Vilanova Tri-class SV Machine – A similar machine called ν-K-SVCR was designed (Zhong- Fukushima 2006) which has two parameters more enabling us effectively controlling the number of support vectors and margin errors – This approach has good results on standard `benchmarks’ – The δ parameter is similar to that used in the ε-insensitivity Vapnik’s function – A drawback for K-SVCR is the high number of parameters to be tuned: three in the non-linearly separable case more the parameters associated of the kernel. First approach K-SVCR

8. Learning 2006 - Vilanova Tri-class SV Machine – A new margin to be maximized – Associated QP problem Second approach 1-v-1 Tri-class

9. Learning 2006 - Vilanova Tri-class SV Machine – In this approach, the width of the ‘decision tube’ along the decision hyperplane is no considered ‘a priori’ and the δ parameter is automatically tuned. – This approach has a major drawback: the number of constraints is much longer than for the K-SVCR approach. Second approach 1-v-1 Tri-class

10. Learning 2006 - Vilanova Tri-class SV Machine – Using the ordinal regression approach in Shashua-Levin 2002, and the tri-class machine idea, a faster novel machine is proposed – A new margin to be maximized – Associated QP problem Third approach 1-v-1-v-r Tri-class

11. Learning 2006 - Vilanova Tri-class SV Machine Third approach 1-v-1-v-r Tri-class Fixed margin strategy Sum of margin strategy The number of constraints is much smaller and the computational cost is lower than for the 1-v-1 triclass approach There are two possible approaches to deal with the large margin principle.

12. Learning 2006 - Vilanova Results

13. Learning 2006 - Vilanova Results Table 2. A comparison of the best accuracy rates using the RBF kernel. Dataset CV1-v-1 1-v-r Tri-class Iris3096.73 96.00 95.49 Wine2598.39 97.86 97.06 Glass1070.91 71.11 71.81 Vowel1098.95 98.48 99.36 Vehicle 384.17 86.21 88.18 DNA–95.45 95.78 95.86

14. Learning 2006 - Vilanova Concluding Remarks Tri-class machines are more robust than bi-class machines and they allow incorporating all the information contained into the training patterns. New research lines can be started about theoretical generalization bounds of this machine. An initiated line is the probabilistic interpretation of the outputs according to their value

15. Learning 2006 - Vilanova Thanks for your attention