Core Research Competencies: Sample Size, Why and How? Presented by Lawrence Mbuagbaw, MD, MPH, PhD 19th July 2016

Outline Introduction Why should we estimate a sample size? What information is required to estimate a sample size for a randomized trial? An example of sample size estimation for a randomized trial How do we report a sample size estimation? Other considerations

Introduction How much data should I collect to answer the question of interest and have faith that the answers are correct? Sample size calculation is an important part of the study design to ensure Validity Accuracy Reliability Scientific and ethical integrity of a study Sample Size calculation should be performed prior to conducting the study

Introduction We need samples because we can’t include every body in a study Too cumbersome Too expensive May not be safe Impossible Yet it is possible to estimate a population parameter using statistics from a sample

Economic reasons Undersized Study: Oversized Study: Unnecessary waste of resources for a study that would produce no answers Oversized Study: Unnecessary waste of resources a study that would produce significant results with no practical importance

Scientific reasons Undersized Study: If a study with negative results doesn’t have a sufficient sample size to detect a clinically important effect, then the negative results are interpretable The treatment did not have an effect at least as large as the effect considered to be clinically relevant OR The treatment had no effect Oversized Study: If the study has positive results, we may be detecting statistically significant effects with no clinical importance.

Ethical reasons Undersized Study: Oversized Study: Expose subjects to unnecessary (sometimes potentially harmful or futile) treatments without the capability to advance knowledge Oversized Study: Expose an unnecessarily large number of subjects to potentially harmful or futile treatments

Sample Size requirements (RCT) Study design Study hypothesis Nature of outcome Statistical methods Measure of variation Number of tails Important difference Significance / power Parallel two-arm individually randomised trial Difference Binary outcome Chi-squared test Mostly important for continuous outcomes and will be the standard deviation

Sample Size requirements (RCT) Study design Study hypothesis Nature of outcome Statistical methods Measure of variation Number of tails Important difference Significance / power Two-sided Test One-sided Test *Recommend: two-sided test

Sample Size requirements (RCT) Study design Study hypothesis Nature of outcome Statistical methods Measure of variation Number of tails Important difference Significance / power Comparative Trials Minimal clinically important difference (MCID) *MCID Can be considered as the smallest change or difference in an outcome that is perceived as beneficial and would lead to a change in the patient’s management, assuming an absence of excessive side effects and costs

Sample Size requirements (RCT) Study design Study hypothesis Nature of outcome Statistical methods Measure of variation Number of tails Important difference Significance / power Significance level: 0.05 Power: 80% or 90% Conclusion from trial Reality Interventions different not different Interventions different (true positive) Power=1-beta (false positive) Type 1 error Alpha error Significance level p-value (false negative) Type 2 error Beta error (true negative)

Sample size requirements If you have thought about all these things as they relate to your study, then you are ready to start estimating a sample size

Example: Consider the research question: Do invitation cards compared to usual care increase male partner attendance at antenatal care?

Sample size requirements Study design: parallel group randomized trial Study hypothesis: invitation cards are different from usual practice Nature of outcome: binary (proportion of women who come with their male partners) Statistical methods: Chi-square test Measure of variation: not applicable (we can use a range of values for the important difference) Number of tails: two Important difference: 10% (12% in intervention and 2% in control) Significance/power: alpha=0.05; power =90%

Step 1: Parameters Event rate in treatment group: 12% (p1) Event rate in control group: 2% (p2) Difference: 10% (p1-p2) Relative Risk: 6.0 (p1/p2) Level of significance: 0.05 (two-tailed) Power: 90%

Step 2: Formula or software Schulz KF, Grimes DA. Sample size calculations in randomised trials: mandatory and mystical. Lancet. Apr 9-15 2005;365(9467):1348-1353.

Step two: Formula Sample required in each arm Constant derived from 0.05 and 90% Risk Ratio= P1/P2 Event rate in control group Schulz KF, Grimes DA. Sample size calculations in randomised trials: mandatory and mystical. Lancet. Apr 9-15 2005;365(9467):1348-1353.

Step 3: Numerical application Note that all the existing formulae do not produce identical results! The differences are not great! Total sample size= 132 x 2 =264

Step 4: What to report Sample size total and per group Intervention/control group rate and difference Measure of variance (for continuous outcomes or if you are reporting a range of values for binary outcomes) Source of 2 and 3 Power, level of significance Test used Hypothesis Tails Attrition Software or formula Range of values for different estimates

Step 4: Reporting Sample sizes of 132 per group (264 in total) are required to achieve an 90% power to detect a difference of 10% (derived from previous studies) between the intervention and control groups, assuming an event rate of 12% in the intervention arm and 2% (RR=6.0) in the control arm, at a level of significance set at 0.05. These computations were done based on a chi-squared test of the null hypothesis that there is no difference between the intervention and control groups. After correcting for 15% attrition, a total sample size of 304 (152 per group) is required. The test statistic is two-sided meaning that results in either direction can be interpreted. Computations were done using the formula by Schulz et al.

Step 4: More on reporting: attrition Should be avoided 10-15% may be appropriate Up to 20% usually frowned upon Integrate in SS estimation Method 1: N= (15/100 *n) +n, where N is the final sample size; and n is the initial sample size. N= (15/100 *264) + 96 = 303.4= 304 Method 2: N= n/1-attrition =264/1-0.15= 310.5=311

Step 5: Consult a biostatistician To verify your estimations To learn more Should always be involved in any RCT Will need one for analysis Your paper will be verified by one Schulz KF, Grimes DA. Sample size calculations in randomised trials: mandatory and mystical. Lancet. Apr 9-15 2005;365(9467):1348-1353.

Other considerations Costs Consider a pilot study to obtain parameters for sample size estimation Avoid retrospective planning It is BAD science Attrition or loss to follow-up can be avoided at the design stage High attrition defeats the purpose of sample size estimation

Resources for sample size estimation Online resources: http://www.stat.ubc.ca/~rollin/stats/ssize/ http://statpages.org/proppowr.html https://www.sealedenvelope.com/ Free software: Winpepi: http://www.brixtonhealth.com/pepi4windows.html G*Power http://www.gpower.hhu.de/en.html

Acknowledgments For CTN investigators This work is supported by the CIHR Canadian HIV Trials Network