Computing and Statistical Data Analysis Stat 5: Multivariate Methods

Slides:

Advertisements

Similar presentations

The Software Infrastructure for Electronic Commerce Databases and Data Mining Lecture 4: An Introduction To Data Mining (II) Johannes Gehrke

Advertisements

Computing and Statistical Data Analysis / Stat 4

G. Cowan Statistical Methods in Particle Physics1 Statistical Methods in Particle Physics Day 2: Multivariate Methods (I) 清华大学高能物理研究中心 2010 年 4 月 12—16.

Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?

Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0) ETHEM ALPAYDIN © The MIT Press, 2010

Support Vector Machines

Statistical Data Analysis / Stat 2

MACHINE LEARNING 9. Nonparametric Methods. Introduction Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 2 

Searching for Single Top Using Decision Trees G. Watts (UW) For the DØ Collaboration 5/13/2005 – APSNW Particles I.

Optimization of Signal Significance by Bagging Decision Trees Ilya Narsky, Caltech presented by Harrison Prosper.

Principle of Locality for Statistical Shape Analysis Paul Yushkevich.

G. Cowan 2007 CERN Summer Student Lectures on Statistics1 Introduction to Statistics − Day 3 Lecture 1 Probability Random variables, probability densities,

G. Cowan Lectures on Statistical Data Analysis 1 Statistical Data Analysis: Lecture 6 1Probability, Bayes’ theorem, random variables, pdfs 2Functions of.

Multivariate Analysis A Unified Perspective

G. Cowan RHUL Physics Statistical Methods for Particle Physics / 2007 CERN-FNAL HCP School page 1 Statistical Methods for Particle Physics CERN-FNAL Hadron.

G. Cowan RHUL Physics Bayesian Higgs combination page 1 Bayesian Higgs combination using shapes ATLAS Statistics Meeting CERN, 19 December, 2007 Glen Cowan.

Machine Learning Usman Roshan Dept. of Computer Science NJIT.

G. Cowan SUSSP65, St Andrews, August 2009 / Statistical Methods 3 page 1 Statistical Methods in Particle Physics Lecture 3: Multivariate Methods.

July 11, 2001Daniel Whiteson Support Vector Machines: Get more Higgs out of your data Daniel Whiteson UC Berkeley.

G. Cowan Lectures on Statistical Data Analysis Lecture 7 page 1 Statistical Data Analysis: Lecture 7 1Probability, Bayes’ theorem 2Random variables and.

Machine Learning1 Machine Learning: Summary Greg Grudic CSCI-4830.

G. Cowan Statistical Methods in Particle Physics1 Statistical Methods in Particle Physics Day 3: Multivariate Methods (II) 清华大学高能物理研究中心 2010 年 4 月 12—16.

Comparison of Bayesian Neural Networks with TMVA classifiers Richa Sharma, Vipin Bhatnagar Panjab University, Chandigarh India-CMS March, 2009 Meeting,

G. Cowan Computing and Statistical Data Analysis / Stat 2 1 Computing and Statistical Data Analysis Stat 2: Catalogue of pdfs London Postgraduate Lectures.

G. Cowan Lectures on Statistical Data Analysis Lecture 1 page 1 Lectures on Statistical Data Analysis London Postgraduate Lectures on Particle Physics;

G. Cowan CLASHEP 2011 / Topics in Statistical Data Analysis / Lecture 21 Topics in Statistical Data Analysis for HEP Lecture 2: Statistical Tests CERN.

Non-Bayes classifiers. Linear discriminants, neural networks.

1 Glen Cowan Multivariate Statistical Methods in Particle Physics Machine Learning and Multivariate Statistical Methods in Particle Physics Glen Cowan.

G. Cowan Lectures on Statistical Data Analysis Lecture 2 page 1 Lecture 2 1 Probability Definition, Bayes’ theorem, probability densities and their properties,

G. Cowan 2011 CERN Summer Student Lectures on Statistics / Lecture 31 Introduction to Statistics − Day 3 Lecture 1 Probability Random variables, probability.

G. Cowan iSTEP 2015, Jinan / Statistics for Particle Physics / Lecture 21 Statistical Methods for Particle Physics Lecture 2: multivariate methods iSTEP.

G. Cowan Statistical Data Analysis / Stat 2 1 Statistical Data Analysis Stat 2: Monte Carlo Method, Statistical Tests London Postgraduate Lectures on Particle.

G. Cowan Weizmann Statistics Workshop, 2015 / GDC Lecture 21 Statistical Methods for Particle Physics Lecture 2: hypothesis tests I; multivariate methods.

G. Cowan Computing and Statistical Data Analysis / Stat 9 1 Computing and Statistical Data Analysis Stat 9: Parameter Estimation, Limits London Postgraduate.

G. Cowan IDPASC School of Flavour Physics, Valencia, 2-7 May 2013 / Statistical Analysis Tools 1 Statistical Analysis Tools for Particle Physics IDPASC.

METU Informatics Institute Min720 Pattern Classification with Bio-Medical Applications Part 9: Review.

1 Introduction to Statistics − Day 2 Glen Cowan Lecture 1 Probability Random variables, probability densities, etc. Brief catalogue of probability densities.

C. Kiesling, MPI for Physics, Munich - ACAT03 Workshop, KEK, Japan, Dec Jens Zimmermann, Christian Kiesling Max-Planck-Institut für Physik, München.

G. Cowan Lectures on Statistical Data Analysis Lecture 6 page 1 Statistical Data Analysis: Lecture 6 1Probability, Bayes’ theorem 2Random variables and.

G. Cowan Aachen 2014 / Statistics for Particle Physics, Lecture 21 Statistical Methods for Particle Physics Lecture 2: statistical tests, multivariate.

Computer Vision Lecture 7 Classifiers. Computer Vision, Lecture 6 Oleh Tretiak © 2005Slide 1 This Lecture Bayesian decision theory (22.1, 22.2) –General.

1 Kernel Machines A relatively new learning methodology (1992) derived from statistical learning theory. Became famous when it gave accuracy comparable.

Helge VossAdvanced Scientific Computing Workshop ETH Multivariate Methods of data analysis Helge Voss Advanced Scientific Computing Workshop ETH.

Multivariate Methods of

Multivariate Methods of

iSTEP 2016 Tsinghua University, Beijing July 10-20, 2016

Tutorial on Statistics TRISEP School 27, 28 June 2016 Glen Cowan

Multivariate Analysis Past, Present and Future

The Elements of Statistical Learning

Comment on Event Quality Variables for Multivariate Analyses

Multi-dimensional likelihood

Recent progress in multivariate methods for particle physics

Overview of Supervised Learning

Tutorial on Multivariate Methods (TMVA)

Computing and Statistical Data Analysis / Stat 8

Computing and Statistical Data Analysis Stat 3: The Monte Carlo Method

Lectures on Statistics TRISEP School 27, 28 June 2016 Glen Cowan

TAE 2018 Benasque, Spain 3-15 Sept 2018 Glen Cowan Physics Department

Computing and Statistical Data Analysis / Stat 6

Computing and Statistical Data Analysis / Stat 7

TRISEP 2016 / Statistics Lecture 2

Statistical Analysis Tools for Particle Physics

SUSSP65, St Andrews, August 2009 / Statistical Methods 3

Model generalization Brief summary of methods

Parametric Methods Berlin Chen, 2005 References:

COSC 4368 Machine Learning Organization

Machine Learning and Multivariate Statistical Methods in Particle Physics Glen Cowan RHUL Physics RHUL Computer Science Seminar.

Computing and Statistical Data Analysis / Stat 10

Support Vector Machines 2

Presentation transcript:

Computing and Statistical Data Analysis Stat 5: Multivariate Methods London Postgraduate Lectures on Particle Physics; University of London MSci course PH4515 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk www.pp.rhul.ac.uk/~cowan Course web page: www.pp.rhul.ac.uk/~cowan/stat_course.html G. Cowan Computing and Statistical Data Analysis / Stat 5

Finding an optimal decision boundary H0 In particle physics usually start by making simple “cuts”: xi < ci xj < cj H1 Maybe later try some other type of decision boundary: H0 H0 H1 H1 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 Multivariate methods Many new (and some old) methods: Fisher discriminant Neural networks Kernel density methods Support Vector Machines Decision trees Boosting Bagging New software for HEP, e.g., TMVA , Höcker, Stelzer, Tegenfeldt, Voss, Voss, physics/0703039 StatPatternRecognition, I. Narsky, physics/0507143 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 2 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 Overtraining If decision boundary is too flexible it will conform too closely to the training points → overtraining. Monitor by applying classifier to independent validation sample. training sample independent validation sample G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 Choose classifier that minimizes error function for validation sample. G. Cowan Computing and Statistical Data Analysis / Stat 5

Neural network example from LEP II Signal: e+e- → W+W- (often 4 well separated hadron jets) Background: e+e- → qqgg (4 less well separated hadron jets) ← input variables based on jet structure, event shape, ... none by itself gives much separation. Neural network output: (Garrido, Juste and Martinez, ALEPH 96-144) G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Kernel-based PDE (KDE, Parzen window) Consider d dimensions, N training events, x1, ..., xN, estimate f (x) with bandwidth (smoothing parameter) kernel Use e.g. Gaussian kernel: Need to sum N terms to evaluate function (slow); faster algorithms only count events in vicinity of x (k-nearest neighbor, range search). G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 G. Cowan Computing and Statistical Data Analysis / Stat 5

Computing and Statistical Data Analysis / Stat 5 Find these on next homework assignment. G. Cowan Computing and Statistical Data Analysis / Stat 5