Sample Size Annie Herbert Medical Statistician Research & Development Support Unit Salford Royal Hospitals NHS Foundation Trust

Timetable TimeTask 60 mins Presentation Presentation Discussion Discussion 15 mins Break 60 mins Demonstration of S.S. calculation Demonstration of S.S. calculation Practical tasks Practical tasks

Outline When are sample size calculations necessary? Single proportions Two proportions Two means Other situations Practical implications Useful references

Why is it important to consider sample size? To have a high chance of detecting a clinically important treatment effect if it exists. To ensure appropriate precision of estimates. To avoid wasting resources and the time of participants. To avoid misleading conclusions.

When is a sample size calculation not necessary? Truly qualitative research. Pilot studies that will be used to inform larger studies (and not make conclusions).

Example 1: Population studies, single proportion (1) What is the prevalence of dysfunctional breathing amongst asthma patients in general practice? (Thomas et al, BMJ 2001) Results: Sample proportion of those suffering from dysfunctional breathing and a confidence interval for this proportion.

Population studies, single proportion (2) Primary outcome variable: Binary Required information: 1) Estimate of what proportion will be, (i (if rate totally unknown pick 50% as most conservative estimate). 2) Size of population if population small, e.g., < 20,000. 3) Acceptable deviation from this population estimate, (half width of confidence interval).

Population studies, single proportion (3)

Statement for Protocol (1) Include all figures that you’ve inputted into the calculation. Possibly add statement about response rate/drop out. Name the person who did the sample size calculation and any software. E.g., ‘A sample of 324 patients will be required to obtain a 95% confidence interval of +/- 5% around a prevalence of approximately 30%. This was calculated by the PI using StatsDirect. We expect that 60% of those who we approach will agree to take part and as this is a questionnaire-based study that there’ll be next to no drop-outs, so we intend to approach 540 patients.’

How sample size varies with precision: Expected Prevalence Acceptable Deviation Sample Size 30% % %14466

Example 2: Comparing two proportions (1) Study: RCT comparing the effectiveness of colony-stimulating factors (CSFs) in reducing sepsis in premature babies. Results: Rate of sepsis at 2 weeks in CSF group and Placebo group, difference between these two proportions, confidence interval for this difference.

Comparing two proportions (2) Primary outcome variable: Binary. Required information: 1) Estimate of proportion in each group (difference is clinically important). 2) Power. 3) Significance level. 4) Treatment:Control ratio (often 1:1).

Definitions (1) ‘Effect Size’ –What do you expect to see? –What has been seen previously? –What is a clinically important difference? ‘Power’ –Probability of detecting a clinically important effect, if it exists. –Typically 80%, 90%.

Definitions (2) ‘Significance Level’ –Cut-off level at which you would say a p-value is significant/non-significant. –Probability of concluding that there is a statistically significant difference in the sample when there is in fact no true difference in the population. –Typically 5%. –Should be set lower if multiple statistical tests have been planned.

Comparing two proportions (3)

Statement for Protocol (2) Comparing two proportions: E.g., ‘A sample size of at least 149 patients per group is required to be able to detect an absolute difference of 16% (50% vs. 34%) in the rate of sepsis between groups with 80% power, at 5% significance level’.

Example 3: Comparing two means (1) A RCT to evaluate a brief psychological intervention in comparison to usual treatment in the reduction of suicidal ideation. (Guthrie et al, BMJ 2001) ‘Suicidal ideation will be measured on the Beck scale; the standard deviation of this scale in a previous study was 7.7, and a difference of 5 points is considered to be of clinical importance.’ Results: Mean reduction in Beck score in the Intervention group and Usual Treatment group, difference between these two means, confidence interval for this difference.

Comparing two means (2) Primary outcome variable: Numerical Required information: 1) Estimate of standard deviation of primary outcome variable. 2) Effect size (difference in means). 3) Power. 4) Significance level. 5) Treatment:Control ratio.

Where to find an estimate of the standard deviation: Pilot study. –Though note standard deviations on very small numbers may be imprecise. Previous studies. In-house data. Rough estimate: Quarter of the range of ‘usual’ values.

Comparing two means (3)

Statement for Protocol (3) Comparing two means: E.g., ‘A sample size of at least 39 patients per group is required to be able to detect a difference in mean Beck score of 5 points or more with 80% power at 5% significance level. This is assuming a standard deviation of 7.7 for the Beck scale.

Things to Note Power is linked to effect size. –All trials have an infinite number of powers! Post-hoc power calculations are pointless. –Power conveyed by confidence interval. If secondary outcomes are important separate sample size calculations should be done for these too. –The largest size resulting from these calculations should be used so powerful enough for all analyses.

Other Situations More than 2 groups. Non-randomised studies, e.g. Case-Control. Equivalence trials. Paired data, e.g., crossover, before/after trial. Time-to-event data. Cluster-randomised studies. Diagnosis studies.

Sample size calculations in practice: Often look at a range of assumptions. –Best case/worst case scenario. Balance between ideal statistical power, resources and time. –Bear in mind only ¼ may consent (or lower). Interim/sub-group analyses. –Seek advice and adjust p-values, etc. accordingly. Issues of power should not overshadow issues of quality.

Useful References Sample size calculations in randomised trials: mandatory and mystical. Schulz & Grimes, The Lancet 2005; 365 An Introduction to Medical Statistics. Bland, M, OUP 2000 Bland, M, OUP 2000 Sample size calculations for clinical studies. Machin, Campbel, Fayers & Pinol, Blackwells