1. Sample Size And Power Warren Browner and Stephen Hulley  The ingredients for sample size planning, and how to design them  An example, with strategies for minimizing sample size

2. Sampling and Inference  A sample is designed to represent a larger population  Therefore, findings in the sample allow inferences about events in the population  Problem: what if the inferences are wrong? Finding something in the sample that isn’t “real” in the population Missing something that is “real”

3. Preventing Wrong Inferences  Difficult when caused by systematic error (bias)  Easier when caused by Random error (chance) Solution: increase sample size Problem: cost, feasibility Goldilocks solution: a sample size that is big enough but not too big

4. Ingredients For Planning Sample Size in an Analytic Study or RCT  Hypothesis Null and alternative One-sided and two-sided  Statistical test Type of variables  Effect size (and its variance)  Power and alpha

5. Research Hypothesis  A clear statement of what you are studying.  Simple:one predictor, one outcome  Specific: who, what, when, where  Stated:in advance

6. Research Hypothesis  In patients with early ALS seen at UCSF in 2007, those randomly assigned to be treated with minocycline will have a lower 1-year mortality than those randomly assigned to placebo.

7. The Null Hypothesis  There’s nothing going on.  Purpose in life: to be rejected in favor of its alternative.  In patients with early ALS seen at UCSF in 2007, those randomly assigned to be treated with minocycline will have the same 1-year mortality as those randomly assigned to placebo.

8. What’s This All About?  A long time ago, statisticians figured out the probability that a sample of a given size would “find something” even if there were nothing going on in the population.

9. This means that...  After a study, we can determine the likelihood that whatever we found in our sample could have occurred by chance... Even if nothing was going on in the population (i.e., the null hypothesis was true)--a “Type I error” If this is very unlikely (say < 1 in 20) we reject the null hypothesis in favor of the alternative hypothesis; we call the finding statistically significant (P <.05)

10. Two-sided Alternative Hypothesis  In patients with early ALS seen at UCSF in 2007, those randomly assigned to be treated with minocycline will have a different 1- year mortality than those randomly assigned to placebo.

11. Two One-sided Alternative Hypotheses  Side A: In patients with early ALS seen at UCSF in 2007, those randomly assigned to be treated with minocycline will have a higher 1-year mortality than those randomly assigned to placebo.  Side B: In patients with early ALS seen at UCSF in 2007, those randomly assigned to be treated with minocycline will have a lower 1-year mortality than those randomly assigned to placebo.

12. If The Null Hypothesis Is True  By chance alone, each of the two one- sided alternative hypotheses is... Possible Wrong Equally likely  Thus a two-sided alternative hypothesis has twice the likelihood of happening by chance alone

13. Next Ingredient: Statistical Test (Types of Variable)  The statistical test determines how the sample size will be calculated  The type of predictor and outcome variable determine which statistical test will be used to analyze the data Both dichotomous: Chi square One dichotomous, one “continuous”: t test Both “continuous”: correlation coeff or t test

14. Statistical Test (Types of Variable)  ALS study Predictor: minocycline vs placebo Outcome: % dead  Both are dichotomous Chi square test

15. Next Ingredient: Effect Sizes (dichotomous variables)  How big an effect you anticipate seeing  Minocycline halves mortality Minocycline = 5%, Placebo = 10%

16. Penultimate Ingredient: Power  The chance of finding something in your sample if it’s really going on in the population (avoiding a Type II error) “Something” = the effect size (or greater)  Usually set at 80% or 90%  = (1 - beta)

17. …and the Final Ingredient: Alpha  The chance of finding something in your sample if there’s nothing going on in the population.

18. Alpha Explained  The level of statistical significance (ie, the p-value that will be considered significant)  The pre-set maximum chance of finding something, if it really isn’t there.  Usually set at 0.05.  May be one-sided or two-sided.

19. Sidedness Of Alpha  With a two-sided alternative hypothesis, you have two chances of finding something that isn’t really there: One (equal) chance for each side.  So a one-sided alpha of 0.05 corresponds to a two-sided alpha of 0.10.

20. SAMPLE SIZE: AN EXAMPLE  Null hypothesis: In patients with early ALS seen at UCSF in 2007, those randomly assigned to be treated with minocycline will have the same 1-year mortality as those randomly assigned to placebo.  Two-sided alternative hypothesis  Dichotomous predictor and outcome  Effect size: 10% mortality + 5%  Power, alpha:90%, 0.05 (two-sided)

21. THE SAMPLE SIZE IS…  Appendix 6.B  Smaller of P1 and P2 = 0.05; power of 90%; alpha of 0.05 (two-sided)  Difference = 0.05 381 473 620  This is per group

22. Sample Size Reduction Strategy #1: Statistical Manipulation  Use a lower power  Use a one-sided alpha Power of 80% One-sided alpha of 0.05

23. The New Sample Size Is…  Appendix 6.B  Smaller of P1 and P2 = 0.05; power of 80%; alpha of 0.05 (one-sided)  Difference = 0.05 381 473 620  This is also per group

24. SS Reduction Strategy #2: Use A More Common Outcome  Change from 1-year mortality to 2-year mortality or loss of independent living  Placebo:40%  Minocycline:20%

25. The New Sample Size Is…  Appendix 6.B  Smaller of P1 and P2 = 0.20; power of 80%; alpha of 0.05 (two-sided)  Difference = 0.20 74 91 118

26. SS Reduction Strategy #3: Use A Continuous Outcome  Change “mortality or loss of independent living” to “muscle strength”  NOTE: Big change in research question and research hypothesis.  New null hypothesis: In patients with early ALS seen at UCSF in 2007, those randomly assigned to be treated with minocycline will have the same grip strength at the end of six months as those treated with placebo.  Two-sided alternative hypothesis

27. Estimate The Mean And Variability Of Grip Strength  Patients with untreated ALS have a (mean ± SD) grip strength of 20 ± 10 kg after 6 months of disease  Minocycline may improve that by 25%

28. Then, at End of Study  Grip strength Placebo:20 kg Minocycline:25 kg (25% more)  Effect size = 5 kg SD = 10 kg  Standardized effect size: E/S = 5/10 = 0.5

29. The New Sample Size Is...  Appendix 6.A  E/S = 0.5  ß = 0.20, Alpha (two-sided) = 0.05  N = 64 per group

30. Ss Reduction Strategy #4: Use A More Precise Outcome  Buy a better instrument to measure grip strength  Use a well-defined protocol  Repeat measurements on two consecutive days  Reduce SD from 10 kg to 8 kg

31. The New Sample Size Is...  New E/S = 5 kg/8 kg= 0.625  ß = 0.20, Alpha (two-sided) = 0.05  N = about 45 per group  This helped quite a bit.

32. SS Reduction Strategy #5: Use Paired Measurements  Most of the variability in grip strength at the end of the study is likely to be due to differences between subjects in grip strength at the beginning of the study.  Switch the outcome to change in grip strength from the beginning to the end of the study.

33. Paired Measurements  Each subject contributes a pair of measurements: (before, after)  The outcome variable is the difference between that pair for each subject.  The SD of the change in a measurement is usually < than the SD of the measurement  SD of change in grip strength is 5 kg  New standardized effect size = 5/5 = 1.0

34. The New Sample Size Is...  E/S = 1.0  ß = 0.20, Alpha (two-sided) = 0.05  N = 17 per group  We now have a potentially do-able study, albeit one that is very different from the original aim.

35. The Bottom Line  Sample size estimation is an integral part of study planning  Almost never the last thing you do  More often, one of your first tasks

36. SAMPLE SIZE PLANNING: REVIEW OF INGREDIENTS  Looking for something in a sample Hypotheses (null and alternative) Will you be able to...  Know it’s there in the population if you find it in your sample ( avoid a Type I error) Test of significance, alpha  Find it in your sample if it’s there in the population (avoid a type II error)? Effect size, power