1. From Small-N to Large Harrison B. Prosper SCMA IV, June 20061 Bayesian Methods in Particle Physics: From Small-N to Large Harrison B. Prosper Florida State University SCMA IV 12-15 June, 2006

2. From Small-N to Large Harrison B. Prosper SCMA IV, June 20062 Outline  Measuring Zero  Bayesian Fit  Finding Needles in Haystacks  Summary

3. Measuring Zero

4. From Small-N to Large Harrison B. Prosper SCMA IV, June 20064 Measuring Zero – 1 In the mid-1980s, an experiment at the Institut Laue Langevin (Grenoble, France) searched for evidence of neutron antineutron oscillations, a characteristic prediction of certain Grand Unified Theories.

5. From Small-N to Large Harrison B. Prosper SCMA IV, June 20065 CRISP Experiment Institut Laue Langevin Magnetic shield Neutron gas on Field-on: -> B off Field-off: -> N

6. From Small-N to Large Harrison B. Prosper SCMA IV, June 20066 Measuring Zero – 2 Count number of signal + background events N. Suppress putative signal and count background events B, independently. Results: N = 3 B = 7 Results: N = 3 B = 7

7. From Small-N to Large Harrison B. Prosper SCMA IV, June 20067 Measuring Zero – 3 Classic 2-Parameter Counting Experiment N ~ Poisson(s+b) B ~ Poisson(b) Infer a statement of form: Pr[s < u(N,B)] ≥ 0.9

8. From Small-N to Large Harrison B. Prosper SCMA IV, June 20068 Measuring Zero – 4 In 1984, no exact solution existed in the particle physics literature! Moreover, calculating exact confidence intervals is, according to Kendal and Stuart, “a matter of very considerable difficulty”

9. From Small-N to Large Harrison B. Prosper SCMA IV, June 20069 Measuring Zero – 5 Exact in what way? Over some ensemble of statements of the form 0 < s < u(N,B) at least 90% of them should be true whatever the true values of s and b. Neyman (1937)

10. From Small-N to Large Harrison B. Prosper SCMA IV, June 200610 Measuring Zero - 6 Tried a Bayesian approach: NNN f(s, b|N) = f(N|s, b)  (s, b) / f(N) NN = f(N|s, b)  (b|s)  (s) / f(N) Step 1. Compute the marginal likelihood f(N|s) = ∫f(N|s, b)  (b|s) db Step 2. N f(s|N)= f(N|s)  (s) / ∫f(N|s)  (s) ds

11. From Small-N to Large Harrison B. Prosper SCMA IV, June 200611 But is there a signal? 1. Hypothesis testing(J. Neyman) H 0 : s = 0 H 1 : s > 0 2. p-value(R.A. Fisher) H 0 : s = 0 3. Decision theory(J.M. Bernardo, 1999) Discrepancy “Distance” between models

12. Bayesian Fit

13. From Small-N to Large Harrison B. Prosper SCMA IV, June 200613 Bayesian Fit Problem: Given counts Data: N = N 1, N 2,..,N M Signal model: A = A 1, A 2,..,A M Background model: B = B 1, B 2,..,B M where M is number of bins (or pixels) find the admixture of A and B that best matches the observations N.

14. From Small-N to Large Harrison B. Prosper SCMA IV, June 200614 Problem (DØ, 2005) Observations = Background + Signal model model (M)

15. From Small-N to Large Harrison B. Prosper SCMA IV, June 200615 Bayesian Fit - Details Assume model of the form Marginalize over a and b

16. From Small-N to Large Harrison B. Prosper SCMA IV, June 200616 Bayesian Fit – Pr(Model) Moreover,… One can compute f(N|p a, p b ) for different signal models M, in particular, for models M that differ by the value of a single parameter. Then compute the probability of model M Pr(M|N) = ∫dp a ∫dp b f(N|p a, p b, M)  (p a,p b |M)  (M) /  (N)

17. From Small-N to Large Harrison B. Prosper SCMA IV, June 200617 130140150160170180190200210220230 0 0.1 0.2 0.3 Top quark mass hypothesis (GeV) P(M|N) mass= 173.5 ± 4.5 GeV signal = 33 ± 8 background= 50.8 ± 8.3 Bayesian Fit – Results (DØ, 1997)

18. Finding Needles in Haystacks

19. From Small-N to Large Harrison B. Prosper SCMA IV, June 200619 single top quark events 0.88 pb 1.98 pb The Needles

20. From Small-N to Large Harrison B. Prosper SCMA IV, June 200620 W boson events The Haystacks 2700 pb 1 : 1000 signal : noise = 1 : 1000

21. From Small-N to Large Harrison B. Prosper SCMA IV, June 200621 The Needles and the Haystacks

22. From Small-N to Large Harrison B. Prosper SCMA IV, June 200622 Finding Needles - 1 The optimal solution is to compute p(S|x) = p(x|S) p(S) / [p(x|S) p(S) + p(x|B) p(B)] Every signal/noise discrimination method is ultimately an algorithm to approximate p(S|x), or a function thereof.

23. From Small-N to Large Harrison B. Prosper SCMA IV, June 200623 Problem: Given D D = x (= x 1,…x N ),y (= y 1,…y N ) of N labeled events. x are the data, y are the labels. Find A function f(x, w), with parameters w, that approximates p(S|x): www p(w|x, y) = p(x, y|w) p(w) / p(x, y) ww = p(y|x, w) p(x|w) p(w) / p(y|x) p(x) w = p(y|x, w) p(w) / p(y|x) assuming p(x|w) = p(x) Finding Needles - 2

24. From Small-N to Large Harrison B. Prosper SCMA IV, June 200624 Likelihood for classification: www p(y|x, w) =  i f(x i, w) y [1 – f(x i, w)] 1-y where y = 0 for background events y = 1 for signal events ww If f(x, w) flexible enough, then maximizing p(y|x, w) with respect to w yields f = p(S|x), asymptotically. Finding Needles - 3

25. From Small-N to Large Harrison B. Prosper SCMA IV, June 200625 However, in a Bayesian calculation it is more natural to average with respect to the posterior density ww f(x|D) = ∫ f(x, w) p(w|D) dw Questions: w 1. Do suitably flexible functions f(x, w) exist? 2. Is there a feasible way to do the integral? Finding Needles - 4

26. From Small-N to Large Harrison B. Prosper SCMA IV, June 200626 Answer 1: Yes! f(x,w) x1x1 x2x2 u, a v, b A neural network is an example of a Kolmogorov function, that is, a function capable of approximating arbitrary mappings f:R n -> R weights The parameters w = (u, a, v, b) are called weights

27. From Small-N to Large Harrison B. Prosper SCMA IV, June 200627 Answer 2: Yes! Computational Method Generate a Markov Chain (MC) of K points {w}, whose stationary density is p(w|D), and average over the stationary part of the chain. Map problem to that of a “particle” moving in a spatially-varying “potential” and use methods of statistical mechanics to generate states (p, w) with probability ~ exp(-H), where H is the “Hamiltonian” H = p 2 + log p(w|D), with “momentum” p.

28. From Small-N to Large Harrison B. Prosper SCMA IV, June 200628 Hybrid Markov Chain Monte Carlo Computational Method… For a fixed H traverse space (p, w) using Hamilton’s equations, which guarantees that all points consistent with H will be visited with equal probability. To allow exploration of states with differing values of H one introduces, periodically, random changes to the momentum p. Software Flexible Bayesian Modeling by Radford Neal http://www.cs.utoronto.ca/~radford/fbm.software.html

29. From Small-N to Large Harrison B. Prosper SCMA IV, June 200629 Example - Finding SUSY! Transverse momentum spectra Signal: black curve Signal:Noise1:25,000

30. From Small-N to Large Harrison B. Prosper SCMA IV, June 200630 Distribution of f(x|D) beyond 0.9 Assuming L = 10 fb -1 CutSB S/√B 0.991x10 3 2x10 4 7.0 Signal:Noise 1:20 1:20 Example - Finding SUSY!

31. From Small-N to Large Harrison B. Prosper SCMA IV, June 200631 Summary  Bayesian methods have been at the heart of several important results in particle physics.  However, there is considerable room for expanding their domain of application.  A couple of current issues:  Is there a signal? Is the Bernardo approach useful in particle physics?  Fitting: Is there a practical (Bayesian?) method to test whether or not an N-dimensional function fits an N-dimensional swarm of points?