1. Probability, Statistics and Errors in High Energy Physics Wen-Chen Chang Institute of Physics, Academia Sinica 章文箴 中央研究院 物理研究所

2. Outline Errors Probability distribution: Binomial, Poisson, Gaussian Confidence Level Monte Carlo Method

3. Why do we do experiments? 1.Parameter determination: determine the numerical value of some physical quantity. 2.Hypothesis testing: test whether a particular theory is consistent with our data.

4. Why estimate errors? We are concerned not only with the answer but also with its accuracy. For example, speed of light 2.998x10 8 m/sec –(3.09  0.15) x10 8 : –(3.09  0.01) x10 8 : –(3.09  2) x10 8 :

5. Source of Errors Random (Statistic) error: the inability of any measuring device to give infinitely accurate answers. Systematic error: uncertainty.

6. Systematic Errors Systematic effects is a general category which includes effects such as background, scanning efficiency, energy resolution, angle resolution, variation of counter efficiency with beam position and energy, dead time, etc. The uncertainty in the estimation of such as systematic effect is called a systematic error Orear Systematic Error: reproducible inaccuracy introduced by faulty equipment, calibration, or technique Bevington Error=mistake? Error=uncertainty?

7. Experimental Examples Energy in a calorimeter E=aD+b a & b determined by calibration expt Branching ratio B=N/(  N T )  found from Monte Carlo studies Steel rule calibrated at 15C but used in warm lab If not spotted, this is a mistake If temp. measured, not a problem If temp. not measured guess  uncertainty Repeating measurements doesn’t help

8. The Binomial n trials r successes Individual success probability p Variance V   = = - 2 =np(1-p) Mean  = =  rP( r ) = np 1-p  p  q A random process with exactly two possible outcomes which occur with fixed probabilities.

9. Binomial Examples n=10 p=0.2 p=0.5p=0.8 p=0.1 n=20 n=50 n=5

10. Poisson ‘ Events in a continuum ’ The probability of observing r independent events in a time interval t, when the counting rate is  and the expected number events in the time interval is. Mean  = =  rP( r ) = Variance V   = = - 2 = =2.5

11. More about Poisson The approach of the binomial to the Poisson distribution as N increases. The mean value of r for a variable with a Poisson distribution is and so is the variance. This is the basis of the well known n  n formula that applies to statistical errors in many situations involving the counting of independent events during a fixed interval. As , the Poisson distribution tends to a Gaussian one.

12. Poisson Examples =25 =10 =5.0 =0.5 =2.0 =1.0

13. Examples The number of particles detected by a counter in a time t, in a situation where the particle flux  and detector are independent of time, and where counter dead-time  is such that   <<1. The number of interactions produced in a thin target when an intense pulse of N beam particles is incident on it. The number of entries in a given bin of a histogram when the data are accumulated over a fixed time interval.

14. Binomial and Poisson From an exam paper A student is standing by the road, hoping to hitch a lift. Cars pass according to a Poisson distribution with a mean frequency of 1 per minute. The probability of an individual car giving a lift is 1%. Calculate the probability that the student is still waiting for a lift (a) After 60 cars have passed (b) After 1 hour a)0.99 60 =0.5472b) e -0.6 * 0.6 0 /0! =0.5488

15. Gaussian (Normal) Probability Density Mean  = =  xP( x ) dx =  Variance V   = = - 2 =  

16. Different Gaussians There’s only one! Normalisation (if required) Location change  Width scaling factor Falls to 1/e of peak at x= 

17. Probability Contents 68.27% within 1  95.45% within 2  99.73% within 3  90% within 1.645  95% within 1.960  99% within 2.576  99.9% within 3.290  These numbers apply to Gaussians and only Gaussians Other distributions have equivalent values which you could use of you wanted

18. Central Limit Theorem Or: why is the Gaussian Normal? If a variable x is produced by the convolution of variables x 1,x 2 … x N I) =  1 +  2 + …  N II)V(x)=V 1 +V 2 + … V N III) P(x) becomes Gaussian for large N

19. Multidimensional Gaussian

20. Chi squared Sum of squared discrepancies, scaled by expected error Integrate all but 1-D of multi-D Gaussian

22. About Estimation Theory Data Statistical Inference Theory Data Probability Calculus Given these distribution parameters, what can we say about the data? Given this data, what can we say about the properties or parameters or correctness of the distribution functions?

23. What is an estimator? An estimator (written with a hat) is a function of the data whose value, the estimate, is intended as a meaningful guess for the value of the parameter. (from PDG)

24. Minimum Variance Bound What is a good estimator? A perfect estimator is: Consistent Unbiassed Efficient minimum One often has to work with less-than- perfect estimators

25. The Likelihood Function Set of data {x 1, x 2, x 3, …x N } Each x may be multidimensional – never mind Probability depends on some parameter a a may be multidimensional – never mind Total probability (density) P(x 1 ;a) P(x 2 ;a) P(x 3 ;a) …P(x N ;a)=L(x 1, x 2, x 3, …x N ;a) The Likelihood

26. Maximum Likelihood Estimation In practice usually maximise ln L as it’s easier to calculate and handle; just add the ln P(x i ) ML has lots of nice properties Given data {x 1, x 2, x 3, …x N } estimate a by maximising the likelihood L(x 1, x 2, x 3, …x N ;a) a Ln L â

27. Properties of ML estimation It ’ s consistent (no big deal) It ’ s biased for small N May need to worry It is efficient for large N Saturates the Minimum Variance Bound It is invariant If you switch to using u(a), then û =u(â) a Ln L â u û

28. More about ML It is not ‘ right ’. Just sensible. It does not give the ‘ most likely value of a ’. It ’ s the value of a for which this data is most likely. Numerical Methods are often needed Maximisation / Minimisation in >1 variable is not easy Use MINUIT but remember the minus sign

29. ML does not give goodness-of-fit ML will not complain if your assumed P(x;a) is rubbish The value of L tells you nothing Fit P(x)=a 1 x+a 0 will give a 1 =0; constant P L= a 0 N Just like you get from fitting

30. Least Squares Measurements of y at various x with errors  and prediction f(x;a) Probability Ln L To maximise ln L, minimise  2 x y So ML ‘proves’ Least Squares. But what ‘proves’ ML? Nothing

31. Least Squares: The Really nice thing Should get  2  1 per data point Minimise  2 makes it smaller – effect is 1 unit of  2 for each variable adjusted. (Dimensionality of MultiD Gaussian decreased by 1.) N degrees Of Freedom =N data pts – N parameters Provides ‘ Goodness of agreement ’ figure which allows for credibility check

32. Chi Squared Results Large  2 comes from 1.Bad Measurements 2.Bad Theory 3.Underestimated errors 4.Bad luck Small  2 comes from 1. Overestimated errors 2. Good luck

33. Fitting Histograms Often put {x i } into bins Data is then {n j } n j given by Poisson, mean f(x j ) =P(x j )  x 4 Techniques Full ML Binned ML Proper  2 Simple  2 x x

34. What you maximise/minimise Full ML Binned ML Proper  2 Simple  2

35. Confidence Level: Meaning of Error Estimates How often we expect to include “the true fixed value of our paramter” P 0, within our quoted range, p  p, for a repeated series of experiments? For the actual value P 0, the probability that a measurement will give us an answer in a specific range of p is given by the area under the relevant part of Gaussian curve. A conventional choice of this probability is 68%.

36. The Straightforward Example Apples of different weights Need to describe the distribution  = 68g  = 17 g 50 100 All weights between 24 and 167 g (Tolerance) 90% lie between 50 and 100 g 94% are less than 100 g 96% are more than 50 g Confidence level statements

37. Confidence Levels Can quote at any level (68%, 95%, 99% … ) Upper or lower or two-sided (x<U x<L L<x<U) Two-sided has further choice (central, shortest … ) U L U’

38. Maximum Likelihood and Confidence Levels ML estimator (large N) has variance given by MVB At peak For large N Ln L is a parabola (L is a Gaussian) a Ln L Falls by ½ at Falls by 2 at Read off 68%, 95% confidence regions

39. Monte Carlo Calculations The Monte Carlo approach provides a method of solving probability theory problems in situations where the necessary integrals are too difficult to perform. Crucial element: random number generator.

40. An Example

41. References Lectures and Notes on Statistics in HEP, http://www.ep.ph.bham.ac.uk//group/locdoc/lectures/stats/index.html http://www.ep.ph.bham.ac.uk//group/locdoc/lectures/stats/index.html Lecture notes of Prof. Roger Barlow, http://www.hep.man.ac.uk/u/roger/ http://www.hep.man.ac.uk/u/roger/ Louis Lyons, “Statistics for Nuclear and Particle Physicists”, Cambridge 1986. Particle Data Group, http://pdg.lbl.gov/2004/reviews/contents_sports.html#mathtoolsetc http://pdg.lbl.gov/2004/reviews/contents_sports.html#mathtoolsetc