1. Results of combination Higgs toy combination, within and across experiments, with RooStats Grégory Schott Institute for Experimental Nuclear Physics of the Karlsruhe Institute of Technology on behalf of the ATLAS-CMS Higgs groups and statistics forum ATLAS-CMS statistics meeting, 5 th July 2010

2. 2Grégory Schott - ATLAS-CMS statistics meeting - 01.07.2010 Introduction I will present the results of ATLAS, CMS and the combination summary of the results obtained by Kyle Cranmer and myself  thanks to Kyle for the FC, PL and some Hybrid results  in case of PL we both get results that of course agree very well This is on-going work; while a large part of the results are available, not everything has been done yet and not everything has been validated yet Toy study (made with mockup numbers), H→WW at 160 GeV with 1fb -1 don't draw physics conclusion of the results Significance and upper limit computed with RooStats (for a given data) different approaches tested as recommanded by the statistics forums Validation of RooStats (ROOT 5.27.04, June 2009) Conclusion and discussions

3. 3Grégory Schott - ATLAS-CMS statistics meeting - 01.07.2010 The combined model Combination of H→WW→ee, mm, em analyses Parameter of interest: ratio of cross-sections, r = m = s / s SM CMS analysis: 3 observables, 37 nuisance parameters Lognormal distribution of nuisance parameters Uncertainty on control region measurement in the systematics ATLAS analysis: 9 observables, 12 nuisance parameters Truncated gaussian distribution of nuisance parameters Control regions measurements included in the analysis Combined analysis: common parameter interest (thus, 100% correlated) no other correlation (yet) across the experiments Concretely, all the information for the combination is shared via 2 ROOT files (for ATLAS & CMS) containing RooFit/RooStats workspaces graphical representation of the objects in the combinedmodel

4. 4Grégory Schott - ATLAS-CMS statistics meeting - 01.07.2010 Toy-data Data used for significance: generated from the model in the hypothesis that signal is present Data used for exclusion: generated from the model in the background-only hypothesis Check the backups if you're interested by the number of observed events assumed (or the likelihood functions)

5. 5Grégory Schott - ATLAS-CMS statistics meeting - 01.07.2010 Profiled likelihood results significance: reasonably good estimate if large enough 95% CL UL taken for -log l(r) = 1.921 assumption of Wilks's asymptotia that's not always valid! with systematics rUL ATLAS00.79 CMS00.28 COMBI00.25 with systematics rsignif. ATLAS1.132.70 CMS0.984.89 COMBI1.025.58 ^ ^

6. 6Grégory Schott - ATLAS-CMS statistics meeting - 01.07.2010 Hybrid Frequentist-Bayesian run for the 3 test statistics mentionned in the previous talk need to study the advantage and inconvenient of the 3 approaches test statisticssignificance (no syst.) significance (with syst.) ATLAS Q LEP Q TEV l(m) 3.78 - 3.07 ± 0.01 2.8 ± 0.1 - CMS Q LEP Q TEV l(m) 6.22 ± 0.02 - 4.77 ± 0.02 > 4.6 4.3 ± 0.1 COMBI Q LEP Q TEV l(m) ------ > 4.6 > 3.5 - CMS, 477k toys Q LEP l(m)l(m) Q TEV B-toys SB-toys data

7. 7Grégory Schott - ATLAS-CMS statistics meeting - 01.07.2010 CMS model with 'LandS' Hybrid test statistics distributions computing the p-value for significance in this approach is challenging:  speed improvements would be useful  or use importance sampling techniques CMS distribution (and results previous slide) made with a RooFit-independent tool Q LEP

8. 8Grégory Schott - ATLAS-CMS statistics meeting - 01.07.2010 Summary of upper limits techniquetest statrulesamplingUL ATLAS UL CMSUL COMBI Feldman-Cousins (no syst.) l(m)l(m) CL S+B toys0.69 ± 0.05-- Profile LR (Wilks) l(m)l(m) CL S+B asymptotic0.790.280.25 Feldman-Cousins++ l(m)l(m) CL S+B toys0.78 ± 0.050.26 ± 0.020.23 ± 0.02 HybridQ LEP CL S toys~ 0.68 0.29 ± 0.03 (LandS) - HybridQ LEP CL S+B toys~ 0.61-- Bayesiann/a, flat prior on rMCMC*0.720.310.28 ATLAS r posterior (r | data) CMS with BAT [ Caldwell, Kollar, Kroeninger, Comp.Phys.Com. 180, 2197 (2009) ] 95% CL UL 95% CL upper limits: results with systematics (except if indicated otherwise)

9. 9Grégory Schott - ATLAS-CMS statistics meeting - 01.07.2010 Validations Some validation of the analysis performed with independent software (LandS, M. Chen, CMS) Since 'LandS' was developped specifically for this analysis, it is faster than RooStats which is a general tool showing that RooStats still need to improve table of compared results: we still need to work on the missing parts PL and toy-MC results using an earlier version of the ATLAS model were validated with an independent code (Hao Liu, et. al.) further on-going work on validating Bayesian limits: S. Schmitz (CMS) currently using BAT, RooStats providing another MCMCCalculator

10. 10Grégory Schott - ATLAS-CMS statistics meeting - 01.07.2010 Conclusion and outlook A good start, work is on-going on Lessons learned: we achieved in a short time scale to combine the Atlas and CMS analyses -> proof of principle that we can do that proof of principle we can compare different statistical methods to one another  still some weaknesses identified that RooStats need to improve  we also need to continue the validations of RooStats still need to understand those results  discussion has started on statistical methods we want to use in common proper treatment of correlations across experiments will require planning and thoughtful parametrization of systematics activity will continue (other masses, other channels, more thorough correlation studies,...)

11. 11Grégory Schott - ATLAS-CMS statistics meeting - 01.07.2010 End of talk − Backup slides

12. 12Grégory Schott - ATLAS-CMS statistics meeting - 01.07.2010 Toy-data Data used for significance: generated from the model in the hypothesis that signal is present Data used for exclusion: generated from the model in the background-only hypothesis Observables: 1) nobs_bin1 = 4 2) nobs_bin2 = 2 3) nobs_bin3 = 3 4) obs_s_em_0j = 19 5) obs_ww_em_0j = 63 6) obs_tt_em_0j = 3814 7) obs_wj_em_0j = 123 8) obs_s_ee_0j = 6 9) obs_ww_ee_0j = 14 10) obs_wj_ee_0j = 53 11) obs_s_mm_0j = 16 12) obs_ww_mm_0j = 26 Observables: 1) nobs_bin1 = 15 2) nobs_bin2 = 7 3) nobs_bin3 = 13 4) obs_s_em_0j = 36 5) obs_ww_em_0j = 52 6) obs_tt_em_0j = 3650 7) obs_wj_em_0j = 143 8) obs_s_ee_0j = 9 9) obs_ww_ee_0j = 5 10) obs_wj_ee_0j = 49 11) obs_s_mm_0j = 18 12) obs_ww_mm_0j = 28

13. 13Grégory Schott - ATLAS-CMS statistics meeting - 01.07.2010 slide from K. Cranmer, 01.07.2010 ATLAS likelihood model

14. 14Grégory Schott - ATLAS-CMS statistics meeting - 01.07.2010 CMS likelihood model slide from A. Korytov, 24.06.2010

15. 15Grégory Schott - ATLAS-CMS statistics meeting - 01.07.2010 Profile likelihood (zoom)

16. 16Grégory Schott - ATLAS-CMS statistics meeting - 01.07.2010 Bayesian upper limits r posterior (r | data) ATLAS posterior (r | data) COMBI r