Research Rotation Part II B Kanna MD, MPH, FACP Associate Program Director of Internal Medicine Associate Director of Graduate Medical Education for Research

Critique of A randomized controlled clinical trial Section I Critique of A randomized controlled clinical trial

Validity the best available approximation to the truth of a given proposition, inference, or conclusion Accurate & reproducible Person 2 Person 1

Checking Validity Conclusion Validity: Internal Validity: In this study, is there a relationship between the two variables? Internal Validity: Assuming that there is a relationship in this study, is the relationship a causal one? Construct Validity: Assuming that there is a causal relationship in this study, did we implement the program we intended to implement and did we measure the outcome we wanted to measure? External Validity: Assuming that there is a causal relationship in this study between the constructs of the cause and the effect, can we generalize this effect to other persons, places or times? http://www.socialresearchmethods.net/kb/introval.php

Random Error Sample 2 Sample 1 CHANCE Sample 3 CHANCE Universe of subjects Sample 1 CHANCE Sample 3 CHANCE

Checking Internal validity Universe of subjects Sample 2 Confounding Rx Outcome Sample 3 Bias

Applying this to a study/observation Day 1 Red pill Sx improves Day 2 Red pill Sx improves Day 3 Red pill Sx improves

THREE THINGS TO REMEMBER! CHANCE – RANDOM ERROR BIAS – SYSTEMATIC ERROR CONFOUNDING – ADDITIONAL FACTORS

How to apply these concepts during the critique of RCT Learn the general steps in a study design Choose a topic Define a hypothesis Select patients Sample size Randomization Treatment Follow-up Collection of data Analysis Interpretation Conclusion Generalize findings

Steps Statistical methods Principles Reading Choose a topic Likelihood/Plausibility Chance Ethical/feasibility/cost/QOL etc Definition a hypothesis Hypothesis test Null/Alternate hypothesis/ Superiority/non-inferiority designs Selection of patients Sampling Bias/Confounder Single/Multi-center/restriction criteria Sample size Power Errors/Effect size Randomization Allocation Definition/type and methods of randomization/Other study designs Treatment Intervention/controls Types of controls/Intervention /observational study designs Follow-up ITT/Blinding/Comparability Bias/Confounder/chance (power) ITT/Blinding methods/accounting for censored data/changes in allocation/Kaplan Meier analysis Collection of data Measurements/Aggregate data Bias Type of variables/ Distribution of data /measures of central tendency Analysis Statistical testing/p values/CI Tests of significance/p value/CI/ Interpretation Checking for Bias/Confounders Stratified analysis/Regression Conclusion Internal validity Chance/Bias/Confounder Validity ( internal Validity) Generalizability External Validity External validity

Section II Selected topics

Study designs Observational Interventional Case control Cohort Cross sectional Interventional Randomized Non-randomized

Number needed to Rx and harm NNT = 1/ Absolute risk reduction Rx = 0.9% Placebo = 3.2% ARR = 5.0- 2.5 = 2.3% NNT = 1/ 0.023 = 45 NNH = 1 / Absolute risk of adverse events

Sensitivity, Specificity & Post test probability

How to design & start a study   Step 1: Select a topic  Step 2: Choose a mentor and guide  Step 3: Define topic  Step 4: Search literature  Step 5: Define problem  Step 6: Collect back ground information  Step 7: Study Methods Study design; Sample selection; Sample size; Inclusion & Exclusion criteria Follow-up; Measurements; Endpoints; Statistical methods;, P value, CI, Test of significance Confounding factors, Stratification, Regression Results, Discussion , Limitations  Step 8: IRB application along with aggregate data request Step 9: IRB approval Step 10: Start project (Revise protocol if needed and apply for IRB modifications)

How to write an IRB protocol Simplified Guide for residents  1. Title page Title; Authors; Contact information of primary authors; Department  2. Summary of the protocol 3. Introduction Problem definition; Magnitude; Need for study 4. Background Physiologic principles; Cause and effect; Hypothesis 5. Methods Study design; Sample selection; Inclusion & Exclusion criteria; Sample size Data collection (tool); Follow-up( If applicable) ; Measurements ( if applicable); Endpoints  6. Statistical methods Testing hypothesis; P value; CI; Tests of significance; Others analyses (e.g. subgroups)  7. Confounding & Bias Confounding ; Stratification; Regression; Methods used to Limit Bias  8. Confidentiality and Privacy  9. Consent/Waiver ( see Lincoln IRB consent procedures)  10. Reporting of results  11. Significance or Implications of the study  12. References  13. Tables, Figures and legends