Diagnostic Test Studies Tran The Trung Nguyen Quang Vinh.

Slides:



Advertisements
Similar presentations
Research Curriculum Session II –Study Subjects, Variables and Outcome Measures Jim Quinn MD MS Research Director, Division of Emergency Medicine Stanford.
Advertisements

Lecture 3 Validity of screening and diagnostic tests
TESTING A TEST Ian McDowell Department of Epidemiology & Community Medicine November, 2004.
Module 6 “Normal Values”: How are Normal Reference Ranges Established?
Critically Evaluating the Evidence: diagnosis, prognosis, and screening Elizabeth Crabtree, MPH, PhD (c) Director of Evidence-Based Practice, Quality Management.
Evaluating Diagnostic Accuracy of Prostate Cancer Using Bayesian Analysis Part of an Undergraduate Research course Chantal D. Larose.
1 Case-Control Study Design Two groups are selected, one of people with the disease (cases), and the other of people with the same general characteristics.
Azita Kheiltash Social Medicine Specialist Tehran University of Medical Sciences Diagnostic Tests Evaluation.
Evaluation of Diagnostic Test Studies
Concept of Measurement
1 Comunicación y Gerencia 18/4/2011Dr Salwa Tayel (Screening) بسم الله الرحمن الرحيم.
Statistics for Health Care
By Dr. Ahmed Mostafa Assist. Prof. of anesthesia & I.C.U. Evidence-based medicine.
Screening Manish Chaudhary BPH(IOM), MPH(BPKIHS)
Thoughts on Biomarker Discovery and Validation Karla Ballman, Ph.D. Division of Biostatistics October 29, 2007.
DIAGNOSTIC TESTS Assist. Prof. E. Çiğdem Kaspar Yeditepe University Faculty of Medicine Department of Biostatistics and Medical Informatics.
Screening and Early Detection Epidemiological Basis for Disease Control – Fall 2001 Joel L. Weissfeld, M.D. M.P.H.
Screening Sherine Shawky, MD, Dr.PH Assistant Professor Public Health King Abdulaziz University College of Medicine
Statistics in Screening/Diagnosis
BASIC STATISTICS: AN OXYMORON? (With a little EPI thrown in…) URVASHI VAID MD, MS AUG 2012.
Multiple Choice Questions for discussion
Lecture 4: Assessing Diagnostic and Screening Tests
Basic statistics 11/09/13.
Diagnostic Testing Ethan Cowan, MD, MS Department of Emergency Medicine Jacobi Medical Center Department of Epidemiology and Population Health Albert Einstein.
Statistics for Health Care Biostatistics. Phases of a Full Clinical Trial Phase I – the trial takes place after the development of a therapy and is designed.
Reliability of Screening Tests RELIABILITY: The extent to which the screening test will produce the same or very similar results each time it is administered.
Vanderbilt Sports Medicine How to practice and teach EBM Chapter 3 May 3, 2006.
Evidence Based Medicine Workshop Diagnosis March 18, 2010.
Dr K N Prasad Community Medicine
Screening and Diagnostic Testing Sue Lindsay, Ph.D., MSW, MPH Division of Epidemiology and Biostatistics Institute for Public Health San Diego State University.
Evaluation of Diagnostic Tests
EVIDENCE ABOUT DIAGNOSTIC TESTS Min H. Huang, PT, PhD, NCS.
1 SCREENING. 2 Why screen? Who wants to screen? n Doctors n Labs n Hospitals n Drug companies n Public n Who doesn’t ?
CHP400: Community Health Program-lI Mohamed M. B. Alnoor Muna M H Diab SCREENING.
Appraising A Diagnostic Test
Likelihood 2005/5/22. Likelihood  probability I am likelihood I am probability.
TESTING A TEST Ian McDowell Department of Epidemiology & Community Medicine January 2008.
Screening of diseases Dr Zhian S Ramzi Screening 1 Dr. Zhian S Ramzi.
Screening and its Useful Tools Thomas Songer, PhD Basic Epidemiology South Asian Cardiovascular Research Methodology Workshop.
HSS4303B – Intro to Epidemiology Feb 8, Agreement.
Overview of Study Designs. Study Designs Experimental Randomized Controlled Trial Group Randomized Trial Observational Descriptive Analytical Cross-sectional.
1 Wrap up SCREENING TESTS. 2 Screening test The basic tool of a screening program easy to use, rapid and inexpensive. 1.2.
Diagnostic Tests Studies 87/3/2 “How to read a paper” workshop Kamran Yazdani, MD MPH.
Unit 15: Screening. Unit 15 Learning Objectives: 1.Understand the role of screening in the secondary prevention of disease. 2.Recognize the characteristics.
Organization of statistical research. The role of Biostatisticians Biostatisticians play essential roles in designing studies, analyzing data and.
Evidence base of clinical diagnosis 邓贝贝 张棣 王迎. Category of diagnostic test laboratory examination medical history and information from physical examination.
Screening.  “...the identification of unrecognized disease or defect by the application of tests, examinations or other procedures...”  “...sort out.
10 May Understanding diagnostic tests Evan Sergeant AusVet Animal Health Services.
EVALUATING u After retrieving the literature, you have to evaluate or critically appraise the evidence for its validity and applicability to your patient.
BIOSTATISTICS Lecture 2. The role of Biostatisticians Biostatisticians play essential roles in designing studies, analyzing data and creating methods.
SCREENING FOR DISEASE. Learning Objectives Definition of screening; Principles of Screening.
Direct method of standardization of indices. Average Values n Mean:  the average of the data  sensitive to outlying data n Median:  the middle of the.
© 2010 Jones and Bartlett Publishers, LLC. Chapter 12 Clinical Epidemiology.
Uses of Diagnostic Tests Screen (mammography for breast cancer) Diagnose (electrocardiogram for acute myocardial infarction) Grade (stage of cancer) Monitor.
Screening Tests: A Review. Learning Objectives: 1.Understand the role of screening in the secondary prevention of disease. 2.Recognize the characteristics.
Critical Appraisal Course for Emergency Medicine Trainees Module 5 Evaluation of a Diagnostic Test.
Diagnostic studies Adrian Boyle.
DR.FATIMA ALKHALEDY M.B.Ch.B;F.I.C.M.S/C.M
Some epidemiological principles and methods
Clinical Epidemiology
Diagnostic Test Studies
Class session 7 Screening, validity, reliability
Dr. Tauseef Ismail Assistant Professor Dept of C Med. KGMC
Comunicación y Gerencia
What is Screening? Basic Public Health Concepts Sheila West, Ph.D.
What is Screening? Basic Public Health Concepts Sheila West, Ph.D.
Dr. Hannah Jordan Lecturer in Public Health ScHARR
PICO model for developing EBM questions
Evidence Based Diagnosis
Presentation transcript:

Diagnostic Test Studies Tran The Trung Nguyen Quang Vinh

Why we need a diagnostic test?  We need “information” to make a decision  “Information” is usually a result from a test  Medical tests: yTo screen for a risk factor (screen test) yTo diagnosse a disease (diagnostic test) yTo estimate a patient’s prognosis (pronostic test)  When and in whom, a test should be done? yWhen “information” from test result have a value.

Value of a diagnostic test  The ideal diagnostic test: yAlways give the right answer: xPositive result in everyone with the disease xNegative result in everyone else yBe quick, safe, simple, painless, reliable & inexpensive  But few, if any, tests are ideal.  Thus there is a need for clinically useful substitutes

Is the test useful ?  Reproducibility (Precision)  Accuracy (compare to “gold standard”)  Feasibility  Effects on clinical decisions  Effects on Outcomes

Determining Usefulness of a Medical Test QuestionPossible DesignsStatistics for Results 1. How reproducible is the test? Studies of: - intra- and inter observer & - intra- and inter laboratory variability Proportion agreement, kappa, coefficient of variance, mean & distribution of differences (avoid correlation coefficient)

Determining Usefulness of a Medical Test QuestionPossible DesignsStatistics for Results 2. How accurate is the test? Cross-sectional, case- control, cohort-type designs in which test result is compared with a “gold standard” Sensitivity, specificity, PV+, PV-, ROC curves, LRs

Determining Usefulness of a Medical Test QuestionPossible Designs Statistics for Results 3. How often do test results affect clinical decisions? Diagnostic yield studies, studies of pre- & post test clinical decision making Proportion abnormal, proportion with discordant results, proportion of tests leading to changes in clinical decisions; cost per abnormal result or per decision change

Determining Usefulness of a Medical Test QuestionPossible Designs Statistics for Results 4. What are the costs, risks, & acceptability of the test? Prospective or retrospective studies Mean cost, proportions experiencing adverse effects, proportions willing to undergo the test

Determining Usefulness of a Medical Test QuestionPossible DesignsStatistics for Results 5. Does doing the test improve clinical outcome, or having adverse effects? Randomized trials, cohort or case-control studies in which the predictor variable is receiving the test & the outcome includes morbidity, mortality, or costs related either to the disease or to its treatment Risk ratios, odd ratios, hazard ratios, number needed to treat, rates and ratios of desirable and undesirable outcomes

Common Issues for Studies of Medical Tests  Spectrum of Disease Severity and Test Results: yDifference between Sample and Population? yAlmost tests do well on very sick and very well people. yThe most difficulty is distinguishing Healthy & early, presymtomatic disease.  Subjects should have a spectrum of disease that reflects the clinical use of the test.

Common Issues for Studies of Medical Tests  Sources of Variation: yBetween patients yObservers’ skill yEquipments => Should sample several different institutions to obtain a generalizable result.

Common Issues for Studies of Medical Tests  Importance of Blinding: (if possible) yMinimize observer bias yEx. Ultrasound to diagnose appendicitis (It is different to clinical practice)

Studies of Diagnostic tests  Studies of Test Reproducibility  Studies of The Accuracy of Tests  Studies of The Effect of Test Results on Clinical Decisions  Studies of Feasibility, Costs, and Risks of Tests  Studies of The Effect of Testing on Outcomes

Studies of Test Reproducibility  The test is to test the precision yIntra-observer variability yInter-observer variability  Design: yCross-sectional design yCategorical variables: Kappa yContinuous variables: coefficient of variance  Compare to it-self (“gold standard” is not required)

Studies of the Accuracy of Tests  Does the test give the right answer?  “Tests” in clinical practice: ySymptoms ySigns yLaboratory tests yImagine tests  To find the right answer.  “Gold standard” is required

How accurate is the test?  Validating tests against a gold standard:  New tests should be validated by comparison against an established gold standard in an appropriate subjects  Diagnostic tests are seldom 100% accurate (false positives and false negatives will occur)

Validating tests against a gold standard  A test is valid if: yIt detects most people with disorder (high Sen) yIt excludes most people without disorder (high Sp) ya positive test usually indicates that the disorder is present (high PV+)  The best measure of the usefulness of a test is the LR: how much more likely a positive test is to be found in someone with, as opposed to without, the disorder

A Pitfall of Diagnostic test A test can separate the very sick from the very healthy does not mean that it will be useful in distinguish patients with mild cases of the disease from others with similar symptoms

Sampling  The spectrum of patients should be representative of patients in real practice.  Example: Which is better? What is the limits? yChest X-ray to diagnose aortic aneurism (AA). Sample are 100 patients with and 100 without AA that ascertained by CT scan or MRI. yFNA to diagnose thyroid cancer. 100 patients with nodule > 3cm and had indication to thyroidectomy (biopsy was the gold standard).

“Gold standard”  “Gold standard” test: often confirm the presence or absence of the disease : D(+) or D(-).  Properties of “Gold standard”: yRuling in the disease (often doing well) yRuling out the disease (maybe not doing well) yFeasible & ethical ? (ex. Biopsy of breast mass) yWidely acceptable.

The test result  Categorical variable: yResult: Positive or Negative yEx. FNA cytology  Continuous variable: yNext step is: find out “cut-off point” by ROC curve yEx. almost biochemical test: pro-BNP, TR-Ab,..

Analysis of Diagnostic Tests  Sensitivity & Specificity  Likelihood ratio: LR (+), LR (-)  Posterior probability (Post-test probability) / Positive, Negative Predictive value (PPV, NPV); given Prior probability (Pre-test probability) How accurate is the test?

Sensitivity and Specificity Test Result Disease D “Gold standard” +- +ab -cd

Positive & Negative Predictive Value  PV (+): positive predictive value  PV (-): negative predictive value Test Result Disease D +- +ab -cd

Posterior odds When combined with information on the prior probability of a disease*, LRs can be used to determine the predictive value of a particular test result: Posterior odds = Prior odds x Likelihood ratio *expressing the prior probability [p] of a disease as the prior odds [p/(1 ‑ p)] of that disease. Conversely, if the odds of a disease are x/y, the probability of the disease is x / (x + y)

Choice of a cut-off point for continuous results Consider the implications of the two possible errors:  If false ‑ positive results must be avoided (such as the test result being used to determine whether a patient undergoes dangerous surgery), then the cutoff point might be set to maximize the test's specificity  If false ‑ negative results must be avoided (as with screening for neonatal phenylketonuria), then the cutoff should be set to ensure a high test sensitivity

Choice of a cut-off point for continuous results  Using receiver operator characteristic (ROC) curves: ySelects several cut-off points, and determines the sensitivity and specificity at each point yThen, graphs sensitivity (true ‑ positive rate) as a function of 1 ‑ specificity (false ‑ positive rate)  Usually, the best cut-off point is where the ROC curve "turns the corner”

RECEIVER OPERATING CHARACTERISTIC (ROC) curve  ROC curves (Receiver Operator Characteristic)  Ex. SGPT and Hepatitis 1-Specificity Sensitivity 1 1 SGPT D + D - Sum < > Sum