Incorporating systematic uncertainties into upper limits

Slides:



Advertisements
Similar presentations
Unified approach to the classical statistical analysis of small signals Gary J. Feldman and Robert D. Cousins.
Advertisements

Sampling: Final and Initial Sample Size Determination
1 LIMITS Why limits? Methods for upper limits Desirable properties Dealing with systematics Feldman-Cousins Recommendations.
Setting Limits in the Presence of Nuisance Parameters Wolfgang A Rolke Angel M López Jan Conrad, CERN.
Statistics In HEP 2 Helge VossHadron Collider Physics Summer School June 8-17, 2011― Statistics in HEP 1 How do we understand/interpret our measurements.
1 Nuisance parameters and systematic uncertainties Glen Cowan Royal Holloway, University of London IoP Half.
Descriptive statistics Experiment  Data  Sample Statistics Experiment  Data  Sample Statistics Sample mean Sample mean Sample variance Sample variance.
Statistical Concepts (continued) Concepts to cover or review today: –Population parameter –Sample statistics –Mean –Standard deviation –Coefficient of.
G. Cowan Lectures on Statistical Data Analysis Lecture 12 page 1 Statistical Data Analysis: Lecture 12 1Probability, Bayes’ theorem 2Random variables and.
G. Cowan 2011 CERN Summer Student Lectures on Statistics / Lecture 41 Introduction to Statistics − Day 4 Lecture 1 Probability Random variables, probability.
Chapter 7 Estimation: Single Population
G. Cowan Lectures on Statistical Data Analysis Lecture 13 page 1 Statistical Data Analysis: Lecture 13 1Probability, Bayes’ theorem 2Random variables and.
Statistical Analysis of Systematic Errors and Small Signals Reinhard Schwienhorst University of Minnesota 10/26/99.
Statistical aspects of Higgs analyses W. Verkerke (NIKHEF)
Regression Analysis Regression analysis is a statistical technique that is very useful for exploring the relationships between two or more variables (one.
Chapter 7 Estimation: Single Population
Basic Business Statistics, 11e © 2009 Prentice-Hall, Inc. Chap 8-1 Confidence Interval Estimation.
QBM117 Business Statistics Estimating the population mean , when the population variance  2, is known.
© 2002 Prentice-Hall, Inc.Chap 6-1 Statistics for Managers using Microsoft Excel 3 rd Edition Chapter 6 Confidence Interval Estimation.
Population All members of a set which have a given characteristic. Population Data Data associated with a certain population. Population Parameter A measure.
G. Cowan 2009 CERN Summer Student Lectures on Statistics1 Introduction to Statistics − Day 4 Lecture 1 Probability Random variables, probability densities,
Estimation in Sampling!? Chapter 7 – Statistical Problem Solving in Geography.
Normal Distributions Z Transformations Central Limit Theorem Standard Normal Distribution Z Distribution Table Confidence Intervals Levels of Significance.
Statistics In HEP Helge VossHadron Collider Physics Summer School June 8-17, 2011― Statistics in HEP 1 How do we understand/interpret our measurements.
Confidence Intervals: The Basics BPS chapter 14 © 2006 W.H. Freeman and Company.
ELEC 303 – Random Signals Lecture 18 – Classical Statistical Inference, Dr. Farinaz Koushanfar ECE Dept., Rice University Nov 4, 2010.
Statistical Methods for Data Analysis Introduction to the course Luca Lista INFN Napoli.
Inferential Statistics 4 Maarten Buis 18/01/2006.
1 Methods of Experimental Particle Physics Alexei Safonov Lecture #25.
1 Methods of Experimental Particle Physics Alexei Safonov Lecture #24.
Study of pair-produced doubly charged Higgs bosons with a four muon final state at the CMS detector (CMS NOTE 2006/081, Authors : T.Rommerskirchen and.
G. Cowan Lectures on Statistical Data Analysis Lecture 4 page 1 Lecture 4 1 Probability (90 min.) Definition, Bayes’ theorem, probability densities and.
G. Cowan Computing and Statistical Data Analysis / Stat 9 1 Computing and Statistical Data Analysis Stat 9: Parameter Estimation, Limits London Postgraduate.
CHAPTER 1 EVERYTHING YOU EVER WANTED TO KNOW ABOUT STATISTCS.
2005 Unbinned Point Source Analysis Update Jim Braun IceCube Fall 2006 Collaboration Meeting.
In Bayesian theory, a test statistics can be defined by taking the ratio of the Bayes factors for the two hypotheses: The ratio measures the probability.
G. Cowan Lectures on Statistical Data Analysis Lecture 12 page 1 Statistical Data Analysis: Lecture 12 1Probability, Bayes’ theorem 2Random variables and.
Lab Chapter 9: Confidence Interval E370 Spring 2013.
G. Cowan RHUL Physics Statistical Issues for Higgs Search page 1 Statistical Issues for Higgs Search ATLAS Statistics Forum CERN, 16 April, 2007 Glen Cowan.
McGraw-Hill/Irwin © 2003 The McGraw-Hill Companies, Inc.,All Rights Reserved. Part Four ANALYSIS AND PRESENTATION OF DATA.
G. Cowan SLAC Statistics Meeting / 4-6 June 2012 / Two Developments 1 Two developments in discovery tests: use of weighted Monte Carlo events and an improved.
Practical Statistics for LHC Physicists Frequentist Inference Harrison B. Prosper Florida State University CERN Academic Training Lectures 8 April, 2015.
Frequentist approach Bayesian approach Statistics I.
Discussion on significance
Status of the Higgs to tau tau
arXiv:physics/ v3 [physics.data-an]
Statistical Methods used for Higgs Boson Searches
MCMC Stopping and Variance Estimation: Idea here is to first use multiple Chains from different initial conditions to determine a burn-in period so the.
Parameter Estimation and Fitting to Data
BAYES and FREQUENTISM: The Return of an Old Controversy
Ex1: Event Generation (Binomial Distribution)
Confidence Intervals and Limits
Discussion 2 1/13/2014.
Grégory Schott Institute for Experimental Nuclear Physics
PSIE Pasca Sarjana Unsri
Lecture 4 1 Probability (90 min.)
Introduction to Inference
Feldman and Cousin Paper
Why does sampling work?.
Likelihood intervals vs. coverage intervals
Lecture 4 1 Probability Definition, Bayes’ theorem, probability densities and their properties, catalogue of pdfs, Monte Carlo 2 Statistical tests general.
Statistical Tests and Limits Lecture 2: Limits
Feldman-Cousins Method
Introduction to Statistics − Day 4
Lecture 4 1 Probability Definition, Bayes’ theorem, probability densities and their properties, catalogue of pdfs, Monte Carlo 2 Statistical tests general.
Statistics for HEP Roger Barlow Manchester University
Interval Estimation Download this presentation.
Copyright © 2015 Elsevier Inc. All rights reserved.
How Confident Are You?.
Uncertainty and confidence
Presentation transcript:

Incorporating systematic uncertainties into upper limits Elton S. Smith, Jefferson Lab Review of Feldman Cousins unified method for constructing confidence limits Integrating over background level and detection efficiency Summary and conclusions

Central Confidence Interval

Upper limits

Probability density Wish to determine parameter m Measure X, with known constant background b Estimate for m is x Function of x/s and m/s only

Ordering principle Likelihood ratio relative to probability with m=max(0,x)

Likelihood Ratio: R(x) = P(x|m)/P(x/mbest) se=0

Ordering Principle: Picking the 90% interval Feldman Cousins Phys Rev D57 (1998) 3873

Probability density, uncertainty in background Background b, estimated at mb with uncertainty smb Integrate over “true” background b Function of x/s and m/s only

Probability density, scale uncertainty Ŝ is the measured detection efficiency S is the “true” detection efficiency Define se = sS/Ŝ, the relative scale uncertainty

Probability distribution

Ordering principle Likelihood ratio relative to probability with m=max(0,x)

90% CL including scale uncertainties

Example 13

Correct Coverage se 90% CL

Dependence on se Rolke NIMA 551 (2005) 493

Scaling from nominal FC limits Simple procedure for scaling FC limits to include systematics

Summary and Conclusions We have investigated how uncertainties in the estimation of background and detection efficiency affect the 90% confidence intervals in the unified approach of Feldman and Cousins. Assumption: Gaussian statistics Systematic uncertainties included using Bayesian approach Confidence intervals have reasonable limiting behavior depend quadratically on se can be obtained by scaling the nominal FC intervals have correct coverage CLAS-NOTE 2007-019 and CLAS-NOTE 2008-20