1. A support Vector Method for Multivariate performance Measures
Author: Thorsten Joachims (ICML’05)
Presenter: Lei Tang

2. Motivation
Current classifier focus on error-rate, how to optimize it directly for different performance measures?
Precision, recall, F-measure etc.

3. Existing Approach
Accurately estimate the probabilities of class membership of each example. (Difficult)
Optimize tractable different variants. But for non-linear measure(F-measure), extensive CV is required.
Directly optimize the measure like ROCArea. But non on F-measure.

4. Reformulation Given training examples
Sample-based Loss
Given training examples
and test examples S’, our goal is to minimize
Decompose the loss function linearly:
Empirical loss:
Example-Based Loss

5. SVM Prediction: Original SVM: Multivariate SVM:
Here, is a function that returns a feature vector of x,y
Prediction:

6. Problems Too many constraints!!!!
N samples, k class labels, then |Y|=k^N.
Do we really need to include all the constraints?

7. Algorithm
Constraint Selection

8. N samples, how many different tables?
Contingency Table
Still impractical!! We have to calculate
Contingency table
N samples, how many different tables?

9. What should the assignment be?
Algorithm for argmax
Given a table,
Exhaustive search all the possible contingency tables and get the maximum.
What should the assignment be?

10. Various Loss F-measure: Precision /Recall
(Just look at top k data points)
Precision/Recall Break-Even Point
The search space is reduced as
a+b=a+c