1. Sponsored by the Clinical and Translational Science Institute and the Department of Population Health Division of Biostatistics Concepts on the Way from Data to Decisions Prakash Laud, PhD Professor of Biostatistics

2. 6/1/2015 CTSI Biostatistics 2 Speaker Disclosure In accordance with the ACCME policy on speaker disclosure, the speaker and planners who are in a position to control the educational activity of this program were asked to disclose all relevant financial relationships with any commercial interest to the audience. The speaker and program planners have no relationships to disclose.

3. 6/1/2015 CTSI Biostatistics 3 Outline Example Some design issues Hypotheses formulation Study-to-study variation Testing hypotheses Confidence interval Planning a study

4. 6/1/2015 CTSI Biostatistics 4 Data Scenario Outcome: Drop in systolic BP Does Drug A reduce systolic BP? 36 patients treated with Drug A 36 untreated patients

5. 6/1/2015 CTSI Biostatistics 5 Drop in BP, Control and Treated Groups 4691661217 4-30-9316512 1194111211913 663-26111917 555712151410 51-2916787 15186915919 81041145164 115591012174

6. 6/1/2015 CTSI Biostatistics 6 Data Description Control: mean=5.2 sd=5.8 Treated: mean=11.1 sd=4.6

7. 6/1/2015 CTSI Biostatistics 7 Design Issues More precise outcome definition needed Patient recruitment Inclusion of control group Assignment to treatment arm Assignment in matched pairs? Balance via randomization Randomization allows quantification of decision uncertainties

8. 6/1/2015 CTSI Biostatistics 8 Decision Problem Formulation Clear statements of opposing possibilities Drug A reduces systolic BP Drug A does not reduce systolic BP Drug A reduces systolic BP more than does placebo Drug A does not reduce systolic BP more than does placebo

9. 6/1/2015 CTSI Biostatistics 9 Hypotheses and Burden of Proof Research hypothesis carries burden of proof Researcher as prosecutor Research hypothesis like accused is guilty Opposite working assumption is the Null Hypothesis (accused presumed innocent) Researcher’s goal is to establish evidence against the Null Hypothesis

10. 6/1/2015 CTSI Biostatistics 10 Need for Statistical Reasoning Generalizing: from patients in study to a much larger target population of patients Repeatability: what might happen if a different group of patients from the target population were in the study Active, deliberate randomization achieves conceptual equivalence of patient groups in repeated studies

11. 6/1/2015 CTSI Biostatistics 11 Two Kinds of Variability Patient-to-patient variation in response Range, spread, standard deviation of individual levels of drop in BP Study-to-study variation in mean difference between control and treated groups Relevant to repeatability of studies The second has a mathematical relation to the first

12. 6/1/2015 CTSI Biostatistics 12 Back to Data Control: mean=5.2 sd=5.8 Treated: mean=11.1 sd=4.8 Difference in means=5.9

13. 6/1/2015 CTSI Biostatistics 13 SD and SE Patient-to-patient variability in response captured by SD (standard deviation) Study-to-study variability in any study statistic captured by SE (standard error) Formula relating SD to SE depends on study design and statistic chosen

14. 6/1/2015 CTSI Biostatistics 14 The Null Hypothesis Drug A is no better than placebo in reducing systolic BP Mean reduction in treated target population equals mean reduction with placebo given to all in target population

15. 6/1/2015 CTSI Biostatistics 15 Study Outcome Mean(treated) - Mean(placebo)=5.9 Does this mean we have disproved H0? Remember H0 is about target populations, not just patients in study SD(treated)=4.6 SD(placebo)=5.8 SE(mean difference)=1.24

16. 6/1/2015 CTSI Biostatistics 16 Simulation under H0 Statistic used for testing is standardized by dividing study mean difference by SE Website : http://lstat.kuleuven.be/java/http://lstat.kuleuven.be/java/

17. 6/1/2015 CTSI Biostatistics 17 Decision Rule Reject H0 if mean difference is larger than a critical value, not just larger than 0

18. 6/1/2015 CTSI Biostatistics 18 Type I Error Rejecting H0 when it is true α = Probability ( Reject H0 | H0 true ) α = 0.05 for BP study with decision rule: Reject H0 if mean difference > 2.04 Mean difference in study = 5.9 Reject H0 and conclude that Drug A reduces BP more than does the placebo

19. 6/1/2015 CTSI Biostatistics 19 More on Type I Error Having rejected the null hypothesis, we could have made a Type I error. But the decision rule controlled the probability of Type I error. This gives us confidence in our decision Decisions can be made at any set α level To allow others to choose their own α, we report what is called the p-value

20. 6/1/2015 CTSI Biostatistics 20 Some More on Type I Error With mean difference = 5.9, p-value is very small, 0.000001 If mean difference = 1.8, p-value is 0.073 If p-value is less than set α, reject H0 Smaller the p-value, stronger the evidence against the null P-value is NOT the probability that the null hypothesis is true

21. 6/1/2015 CTSI Biostatistics 21 P-value

22. 6/1/2015 CTSI Biostatistics 22 Back to Original Data Mean difference = 5.9, p-value=0.000001 Strong evidence against H0 Strong evidence that Drug A is better than placebo How much better? Achieving 5.9 mmHg higher drop than placebo? What about study-to-study variation?

23. 6/1/2015 CTSI Biostatistics 23 Quantifying Improvement Confidence Interval Mean difference ± 1.96 × SE (mean diff) 5.9 ± 1.96 × 1.24, i.e., 5.9 ± 2.43 3.47 to 8.33 This is a 95% Confidence Interval This interval was made using a procedure that has a 95% probability of capturing the true improvement

24. 6/1/2015 CTSI Biostatistics 24 Simulation of CI’s Website : http://lstat.kuleuven.be/java/http://lstat.kuleuven.be/java/

25. 6/1/2015 CTSI Biostatistics 25 Two Types of Error Null Hypothesis True Null Hypothesis False Rejected Null Hypothesis Type I ErrorCorrect Decision Did not reject Null Hypothesis Correct Decision Type II Error

26. 6/1/2015 CTSI Biostatistics 26 Type II Error and Power In planning a study, setting only a level for probability of Type I Error, like α = 0.05, only protects against this error We also want a small probability of accepting H0 if it is false Equivalently, we want a large probability of rejecting H0 if it is false This is called Power; it depends on effect size

27. 6/1/2015 CTSI Biostatistics 27 Effect Size and Power Need to specify a specific scenario under H1

28. 6/1/2015 CTSI Biostatistics 28 Power and Sample Size

29. 6/1/2015 CTSI Biostatistics 29 Terminology Treatment arm Control arm Randomization Target population Study sample Null hypothesis Alternative /Research hypothesis Standard deviation (patient-to-patient variation) Standard error (study-to- study variation in statistic) Type I error α = probability of Type I error p-value Type II error Power Effect size Sample size Confidence interval

30. 6/1/2015 CTSI Biostatistics 30 Resources The Clinical and Translation Science Institute (CTSI) supports education, collaboration, and research in clinical and translational science: www.ctsi.mcw.eduwww.ctsi.mcw.edu The Biostatistics Consulting Service provides comprehensive statistical support www.mcw.edu/biostatistics.htm www.mcw.edu/biostatistics.htm

31. 6/1/2015 CTSI Biostatistics 31 Free drop-in consulting MCW/Froedtert/CHW: 1 – 3 PM –Monday, Wednesday, Friday @ CTSI Administrative offices (LL772A) –Tuesday, Thursday 1 – 3 PM @ Health Research Center, H2400 VA: 1 st and 3 rd Monday, 8:30-11:30 am –VA Medical Center, Room 6119 Marquette: 2 nd and 4 th Monday, 8:30-11:30 am – Olin Engineering Room 338D