1. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 1 Support Vector Regression

2. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 2 SVR Drawings and illustrations from Bernhard Schölkopf, and Alex Smola: Learning with Kernels (MIT Press, Cambridge, MA, 2002)

3. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 3 SVR - History Based on Learning Theory, consisting of few axioms on learning errors Started in 1960’s, still actively developed SVRs recently outperformed NNs in recognition tests on US Postal Service’s standard set of handwritten characters libSVM by Chih-Chung Chang and Chih-Jen Lin provides fast and simple to use implementation, extended as requests (e.g. from HEP) come in

4. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 4 Training sample X, observed results Y Goal: f with y=f(x) Simplicity: Linear case, Formulation of Problem

5. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 5 Optimal confidence = maximal margin Minimize quadratic problem with Quadratic problem: Unique solution! Optimizing the Confidence

6. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 6 Non-Linearity Introduce mapping to higher dimensional space e.g. Gaussian kernel:

7. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 7 Calculation

8. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 8 L2 b Tagger Parameters

9. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 9 L2 b Tagger Parameters

10. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 10 L2 b Tagger Output SVRNN

11. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 11 L2 b Tagger Discussion Complex problem increases number of SVs Almost non-separable classes still almost non-separable in high dimensional space High processing time due to large number of SVs NNs show better performance for low-information, low- separability problems

12. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 12 Higgs Parameters Higgs SVR analysis by Daniel Whiteson, UC Berkley

13. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 13 Higgs Parameters

14. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 14 Higgs Output  Background Signal 

15. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 15 Higgs Purity / Efficiency Purity

16. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 16 Kernel Width Integrated Significance

17. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 17 Summary SVR often superior to NN Not stuck in local minima: unique solution Better performance for many problems Implementation exists, actively supported by the development community Further information: www.kernel-machines.org Time for SVR @ HEP!

18. Axel Naumann, DØ University of Nijmegen, The Netherlands June 24, 2002 ACAT02, Moscow 18 L2 b Tagger Correlation budcs SVR NN