Class session 7 Screening, validity, reliability

Slides:



Advertisements
Similar presentations
Validity and Reliability of Analytical Tests. Analytical Tests include both: Screening Tests Diagnostic Tests.
Advertisements

Lecture 3 Validity of screening and diagnostic tests
1 Case-Control Study Design Two groups are selected, one of people with the disease (cases), and the other of people with the same general characteristics.
TUTORIAL SCREENING Dr. Salwa Tayel, Dr. A. Almazam, Dr Afzal Mahmood
CRITICAL APPRAISAL Dr. Cristina Ana Stoian Resident Journal Club
GerstmanChapter 41 Epidemiology Kept Simple Chapter 4 Screening for Disease.
1 Comunicación y Gerencia 18/4/2011Dr Salwa Tayel (Screening) بسم الله الرحمن الرحيم.
IE486 Work Analysis & Design II Lab 4 – Upper Limb Assessment Supplementary Notes on Evaluation of Analysis Tools March 23, 2007 Vincent G. Duffy, IE &
Screening for Disease Guan Peng Department of Epidemiology School of Public Health, CMU.
Screening Manish Chaudhary BPH(IOM), MPH(BPKIHS)
Thoughts on Biomarker Discovery and Validation Karla Ballman, Ph.D. Division of Biostatistics October 29, 2007.
What is Screening? Basic Public Health Concepts Sheila West, Ph.D. El Maghraby Professor of Ophthalmology Wilmer Eye Institute Johns Hopkins University.
Statistics in Screening/Diagnosis
BASIC STATISTICS: AN OXYMORON? (With a little EPI thrown in…) URVASHI VAID MD, MS AUG 2012.
Multiple Choice Questions for discussion
Lecture 4: Assessing Diagnostic and Screening Tests
Basic statistics 11/09/13.
Principles and Predictive Value of Screening. Objectives Discuss principles of screening Describe elements of screening tests Calculate sensitivity, specificity.
Basic Ideas and Terminology Ettore Beghi Institute for Pharmacological Research Mario Negri, Milano, Italy.
SCREENING Asst. Prof. Sumattna Glangkarn RN, MSc. (Epidemiology), PhD (Nursing studies)
Reliability of Screening Tests RELIABILITY: The extent to which the screening test will produce the same or very similar results each time it is administered.
CHP400: Community Health Program-lI Mohamed M. B. Alnoor Muna M H Diab SCREENING.
MEASURES OF TEST ACCURACY AND ASSOCIATIONS DR ODIFE, U.B SR, EDM DIVISION.
Likelihood 2005/5/22. Likelihood  probability I am likelihood I am probability.
Chapter 10 Screening for Disease
Screening Puja Myles
SCREENING Dr. Aliya Hisam Community Medicine Dept. Army Medical College, RWP.
Evaluating Results of Learning Blaž Zupan
Screening of diseases Dr Zhian S Ramzi Screening 1 Dr. Zhian S Ramzi.
Screening and its Useful Tools Thomas Songer, PhD Basic Epidemiology South Asian Cardiovascular Research Methodology Workshop.
HSS4303B – Intro to Epidemiology Feb 8, Agreement.
1 Wrap up SCREENING TESTS. 2 Screening test The basic tool of a screening program easy to use, rapid and inexpensive. 1.2.
Diagnostic Tests Studies 87/3/2 “How to read a paper” workshop Kamran Yazdani, MD MPH.
Unit 15: Screening. Unit 15 Learning Objectives: 1.Understand the role of screening in the secondary prevention of disease. 2.Recognize the characteristics.
Screening.  “...the identification of unrecognized disease or defect by the application of tests, examinations or other procedures...”  “...sort out.
10 May Understanding diagnostic tests Evan Sergeant AusVet Animal Health Services.
The Mathematics of Biostatistics Chapter 6 and 7 Copyright Kaplan University 2009.
Timothy Wiemken, PhD MPH Assistant Professor Division of Infectious Diseases Diagnostic Tests.
SCREENING FOR DISEASE. Learning Objectives Definition of screening; Principles of Screening.
Biostatistics Board Review Parul Chaudhri, DO Family Medicine Faculty Development Fellow, UPMC St Margaret March 5, 2016.
CHAPTER 3 Key Principles of Statistical Inference.
Screening Tests: A Review. Learning Objectives: 1.Understand the role of screening in the secondary prevention of disease. 2.Recognize the characteristics.
Critical Appraisal Course for Emergency Medicine Trainees Module 5 Evaluation of a Diagnostic Test.
دکتر حمیدرضا صابری
CHP400: Community Health Program-lI Mohamed M. B. Alnoor Muna M H Diab SCREENING.
Screening for Disease: Part One
Performance of a diagnostic test Tunisia, 31 Oct 2014
…it’s really interesting
DR.FATIMA ALKHALEDY M.B.Ch.B;F.I.C.M.S/C.M
Diagnostic Test Studies
Probability and Statistics
Principles of Epidemiology E
Evaluating Results of Learning
Understanding Results
Class session 6 (AGE) Standardization
Dr. Tauseef Ismail Assistant Professor Dept of C Med. KGMC
Comunicación y Gerencia
EPID 503 – Class 6 Standardization.
بسم الله الرحمن الرحيم Clinical Epidemiology
What is Screening? Basic Public Health Concepts Sheila West, Ph.D.
How do we delay disease progress once it has started?
Diagnosis II Dr. Brent E. Faught, Ph.D. Assistant Professor
What is Screening? Basic Public Health Concepts Sheila West, Ph.D.
Is a Positive Developmental-Behavioral Screening Score Sufficient to Justify Referral? A Review of Evidence and Theory  R. Christopher Sheldrick, PhD,
Screening, Sensitivity, Specificity, and ROC curves
Dr. Hannah Jordan Lecturer in Public Health ScHARR
Patricia Butterfield & Naomi Chaytor October 18th, 2017
Positive predictive value of screening tests
How to assess an abstract
Evidence Based Diagnosis
Presentation transcript:

Class session 7 Screening, validity, reliability Epidemiology 503, Section 2

Announcements Next class is in SPHII-1020 at 10 AM Following class (next Monday) will be review problems – no group work Please complete them BEFORE coming to class Additional problems and a practice exam will be available this week (Wednesday)

Last Class Looked at creating a summary mortality rate for populations with different age distributions Key concept: Mortality rates differ by age group so the number of deaths observed in a population is dependent on the age distribution

Rates from Population A Rates from Population B Because using age-specific death rates from populations typically only used in large groups For each population: Calculate age-specific mortality rates Multiply age-specific rates by the # of people in corresponding age range in standard population Sum expected # of deaths across age groups Divide total # of expected deaths by total standard population Result: Age-adjusted mortality rate for each population of interest Direct Method Rates from Population A Rates from Population B Applied to A standard population e.g. US population in 2000 Choice of standard is somewhat ARBITRARY.

Applied to the age distribution of the study population Useful when I don’t have or trust the group- specific rates (i.e. population is too small) Acquire age-specific mortality rates for standard population Multiply standard population’s age- specific rates by # of people in age range in population of interest Sum expected # of deaths across age groups in study population Divide observed # of deaths by expected # of deaths in population of interest SMR: observed # deaths per year expected # deaths per year >1 more deaths than expected =1 as expected <1 less deaths than expected Indirect Method Rates from the standard population Applied to the age distribution of the study population

This Class Screening: the use of testing to sort out apparently well persons (asymptomatic) who probably have disease from those who probably do not

The Natural History of Disease Lead Time: Interval by which the time of diagnosis is advanced by screening and early detection compared to usual time of diagnosis Critical Point: Point in the natural history of disease before which treatment is effective and/or less difficult to administer Herman CR et al. Screening for Preclinical Disease: Test and Diagnosis. American Journal of Roentgenology. 2002;179:825-831.

Valid Valid Reliable Reliable Valid Valid Reliable Reliable Validity: Ability of a test to distinguish between who has disease and who does not ACCURACY Reliability: Ability to replicate results on same sample if test is repeated PRECISION Valid Reliable Valid Reliable Citation: Hal Morgenstern, Epid 601 Coursepack 2011, Part 3 page 2.

Assessing Validity: Sensitivity Ability of a test to correctly identify those who have the disease Proportion of those who test positive for a disease among those that have disease Specificity Ability of a test to correctly identify those who do not have the disease Proportion of those who test negative among those who do not have disease

Calculating Sensitivity & Specificity Disease Status Screening Test Disease No Disease Positive TP (a) FP (b) Negative FN (c) TN (d) diseased who screen positive a all diseased (a+c) Sensitivity = = non-diseased who screen negative d all non-diseased (b+d) Specificity = =

Example Disease Status Screening Test Disease No Disease Positive 60 TP (a) FP (b) Negative FN (c) TN (d) Disease Status Screening Test Disease No Disease Positive 60 240 Negative 10 290 Sensitivity: 𝑇𝑃 𝑇𝑃+𝐹𝑁 = 𝑎 𝑎+𝑐 = 60 60+10 = 60 70 =0.857 or 85.7% Specificity: 𝑇𝑁 𝐹𝑃+𝑇𝑁 = 𝑑 𝑏+𝑑 = 290 240+290 = 290 530 =0.547 or 54.7%

Trade-Off of Sensitivity and Specificity Ideally we want a test that is 100% sensitive and specific Generally there is a trade-off between sensitivity and specificity As you move the cut point, the sensitivity and specificity will change (no disease) (disease)

Positive predictive value (PPV) Negative predictive value (NPV) Disease Status Screening Test Disease No Disease Positive TP (a) FP (b) Negative FN (c) TN (d) Positive predictive value (PPV) Proportion of people who have the disease among those that tested positive Negative predictive value (NPV) Proportion of people who do not have disease among those that tested negative NPV: 𝑇𝑁 𝐹𝑁+𝑇𝑁 = 𝑑 𝑐+𝑑 PPV: 𝑇𝑃 𝑇𝑃+𝐹𝑃 = 𝑎 𝑎+𝑏

Relationship between PPV and NPV and Prevalence

Why? What happens when the prevalence of a disease goes up? No Disease (-) Totals Screening Test Says Disease (+) A (True Positives) B (False Positives) A+B Screening Test Says No Disease (-) C (False Negatives) D (True Negatives) C+D A + C B + D A+C+B+D What happens when the prevalence of a disease goes up?

Why? PPV: A/(A+B) NPV = D/(C+D) Disease (+) No Disease (-) Totals Screening Test Says Disease (+) A (True Positives) B (False Positives) A+B Screening Test Says No Disease (-) C (False Negatives) D (True Negatives) C+D A + C B + D A+C+B+D PPV: A/(A+B) NPV = D/(C+D)

Implications Because of differences in disease prevalence the same test can have very different predictive values when administered to a high-risk vs. a low-risk group Test interpretation should be done taking into consideration the population In some cases it is appropriate to use different cut- points for different populations or risk groups

Assessing Reliability: Reliability (precision) is influenced by: Intrasubject variation Intraobserver variation Interobserver variation % Agreement: # cells that Agree Total # Observations Kappa Statistic: How much better is the agreement between observers than would be expected by chance alone? Observer 1 + - Total 48 10 58 7 35 42 55 45 100 Observer 2 x100 Example: (48+35)/100= 0.83*100 = 83% agreement between Observer 1 and 2