1. Comparison of Bayesian Neural Networks with TMVA classifiers Richa Sharma, Vipin Bhatnagar Panjab University, Chandigarh India-CMS March, 2009 Meeting, DU, Delhi

2. CONTENTS ● Multivariate techniques and their advantages ● BNN -Introduction -Algorithm -Software used ● Work done ● Comparison of BNN with some of the classifiers in TMVA

3. MULTIVARIATE TECHNIQUES AND THEIR ADVANTAGES ● Many statistical techniques focus on just one or two variables ● Multivariate analysis techniques allow more than two variables to be analyzed at once ● Useful when there is correlation among the variables ● More information is analyzed simultaneously, giving greater power ● Variable selection (e.g., to give maximum signal/background discrimination)

4. TMVA(Toolkit for Multivariate Analysis) The Toolkit for Multivariate Analysis (TMVA) provides a ROOT-integrated machine learning environment for the processing and parallel evaluation of sophisticated multivariate classification techniques The package includes: Rectangular cut optimization Projective likelihood estimation (PDE approach) Multidimensional probability density estimation (PDE – range-search approach) Multidimensional k-nearest neighbour classifier Linear discriminant analysis (H-Matrix and Fisher discriminants) Function discriminant analysis (FDA) Artificial neural networks (three different implementations) Boosted/Bagged decision trees Predictive learning via rule ensembles (RuleFit) Support Vector Machine (SVM)

5. ● An extremely simplified model of the brain ● Transforms inputs into output to the best of its ability ● The basic computational element is often called a node or unit ● Each input has an associated weight w, which can be modified ● The unit computes some function ‘f ’ of the weighted sum of its inputs:       Input nodes Hidden nodes Output node Neural Networks

6. Bayesian Neural Networks BNN is a model for classification based on Bayesian Statistics The goal of BNN is to approximate the posterior class conditional probabilities BAYES THEOREM P(C k |x) = P(x|C k )P(C k )/P(x) x = (x 1,x 2,....x P ) where P is the no. of variables C k are the classes(signal and background) P(C k |x) = conditional probability, that an object described by x belongs to class C k P(x|C k ) = conditional probability that an object is of the form x given that it is of class C k

7. FBM(Flexible Bayesian Modelling Package) by Radford M. Neal The package is organized in a modular fashion ● UTIL - specify final portion of a probabilistic model that supports reading of numeric input from data files or other sources,& specify sets of training and test cases ● MC - provides support for MCMC methods ● DIST – contains programs for sampling from a distribution specified by formulas for the prior and likelihood for a Bayesian model ● NET – implements Bayesian learning based on MLP networks

8. WORK DONE ● Ported BNN package being used in D0 to a local machine as a standalone package - macros modified to remove D0 dependence - library dependence changed for compilation ● No D0 environment specific dependence ● Tested it for independent samples ● Compared the efficiency of signal/background discrimination of BNN with some of the classifers in TMVA (Thanks to : S. Jain, OSU/D0 for valuable inputs on BNN setup/testing)

9. Make a list of signal and background files and variables for which to be trained Read the root files and write out the variables in a text file Create unweighted events from weighted ones Mix signal and background training files Run FBM code to get the Bayesian probabilities Run Markov Chain Average over the previous iterations BNN filter created, used in final analysis with data STEPS INVOLVED IN MAKING AN ANALYSIS

10. BNN performance using D0 MC data

11. Verification plots

12. BNN performance using a toy example

13. Plots showing efficiency of the BNN

14. Efficiency Plots of the classifiers in TMVA

15. COMPARISON BETWEEN BNN and TMVA CLASSIFIERS 1. BDT 2. 3. 4. 5. 6. LIKELIHOOD PDERS FISHER MLP BNN 61.77 59.80 62.68 63.44 63.49 63.61 -0.14 0.18 0.38 -0.07 -0.22 0.35 CLASSIFIER S / √(S+B) CUT

16. Results and future plan of work ● The efficiency of BNN is comparable to the classifiers in TMVA, therefore good enough to promote BNN for CMS ● Can be used as a cross check to the studies already done using TMVA ● Therefore we plan to test it on data samples available in our group and then use it for physics studies

19. ● The aim is to find an optimum classifier n(x,  ) that minimizes the rate of misclassification ● Minimize the empirical risk function with respect to  E(  )=1/N S ([ti-n(xi,  )]) 2 ● One can show that the error gets minimized when n(x,  )=∫t p(t|x)dt ● This yields one sets of weights  which corresponds to a mapping function n(x,  ) ● Prone to overfitting ● Therefore we will use BNN i.e. training over the entire  space n(x'|t,x) = ∫n(x',  )p(  |t,x)d  ● Will produce a function n that is more stable and is less likely to be overtrained ALGORITHM for NEURAL NETWORKS