1. Minimax Probability Machine (MPM)
Jay Silver

2. Very High Level Diagram of Training a Pattern Classifier
Augmented
Testing a New Data Point

3. Finding a Function that Decides
Decision If
, choose class wy
, choose class wx
Assume Binary
Non Parametric
Parametric
Support Vector Machine (SVM)
Minimax Probability Machine (MPM)
Gaussian

4. MPM SVM Non-Parametric Linear Decision Boundaries
Maximal Margin Classifier
Minimize Worst Future Error
An SVM and MPM toolbox were used for implementation [1,4]. MPM figure borrowed from [2].

5. MPM Problem Statement Lower bound on test accuracy
Upper bound of misclassifying future point
with
Mahalanobis Distance
Equal
Problem Statement
s.t.
Lower bound on test accuracy
An SVM and MPM toolbox were used for implementation [1,4]. MPM figure borrowed from [2].

6. Expanding the Feature Space with Kernels
Original Feature Space
Expanded Feature Space
XOR: {x1, x2}
XOR: {x1, x2, x1x2}
Not Linearly Separable
Linearly Separable
Kernel Examples
Gaussian Kernel:
Polynomial Kernel:

7. Take a Look at Some Linear Decision Boundaries
Key

8. Results for the Distribution We Just Saw
SVM Performs Best
MPM Performs Well
SVM Homogeneous Polynomial Fails to Converge

9. Alpha as an Underbound to Test Accuracy
Compare Alpha to Test Accuracy
Just Note Correlation Between
Alpha and Test Accuracy
Key

10. Testing on a Real Speech Task
Deterding Data – 11 vowel sounds with 10 features
Multiple classes – Use 1 vs. 1 voting to generalize binary classifiers
Test Accuracy for the Gaussian Kernel
MPM Peaks At 67.3%
Key
SVM Peaks At 68.4%

11. Summary of Deterding Results
Distill Results Further
Linear
Nonlinear
Classifier
Accuracy
Bayes
50.7%
SVM
51.7%
MPM
48.7%
Classifier
Accuracy
Bayes
47.2%
SVM
68.4%
MPM
67.3%

12. Conclusions Alpha is an accurate lower bound for all cases but one.
Alpha was reasonably well correlated with test accuracy.
SVM homogeneous polynomial kernel outperformed MPM
But MPM homo. poly. kernel was more consistent
MPM Gaussian kernel performed 1% below SVM on Deterding
MPM:
Competitive, including realistic speech tasks
Mathematically pleasing
Room to grow
Not quite as accurate as SVMs

13. References

14. Questions?
The Rainbow Linear Discriminant Between CSTIT Students