Research Curriculum Session III – Estimating Sample Size and Power Jim Quinn MD MS Research Director, Division of Emergency Medicine Stanford University

Overview Funding Issues -ACEP.org Research Grant Program Overview ACEP.org Research Grant Program OverviewACEP.org Research Grant Program Overview -Kaiser - Mid December Sample Size Calculations -Basic statistical testing -Variables -Assumptions -Strategies for minimizing sample size

Estimating Sample Size Clearly stated simple question One predictor and one outcome measure Ensure that our sample is representative of the population we are basing our hypothesis on.

Hypothesis Testing Null Hypothesis -There is no difference between the predictor and outcome variables in the population -Assuming there is no association, statistical tests estimate the probability that the association is due to chance Alternate Hypothesis -The proposition that there is an association between the predictor and outcome variable -We do not test this directly but accept it by default if the statistical test rejects the null hypothesis

Hypothesis testing Statistical Principles Always use two sided tests Level of statistical significance Type I and II errors Effect Size Variability of the population/sample

Level of Significance Set at 0.05 for alpha and 0.20 for beta “If there is less than a 1/20 chance that difference between two group is due to chance alone we reject the Null hypothesis and accept the Alternate hypothesis that they are different” “If there is less than a 1/20 chance that difference between two group is due to chance alone we reject the Null hypothesis and accept the Alternate hypothesis that they are different” For two sided tests that is in each tail For two sided tests that is in each tail

Type I and II Errors Many types of errors, not just statistical False negative and false positive can occur because of errors due to bias Type I (statistical false positive)- reject the null hypothesis but in fact it is true. (or you think there is a difference but there really isn’t one) Type II (statistical false negative) – accept the null hypothesis but in fact there is a difference

Type I and II Errors Type I and II errors are usually avoidable by having adequate sample size or manipulating the design of the study and measure of outcomes and 0.20 are arbitrary and many believe beta should be 0.10

Effect Size “What is a meaningful difference between the groups” It is truly an estimate and often the most challenging aspect of sample size planning Large difference – small sample size Small differences – large sample size -Find data from other studies -Survey people -Cost/benefit

Variability The greater the variability in the outcome measure the more likely the groups will overlap Less precise measures and measurement error increase the variability Variability is decreased by increasing the sample size For sample size calculations of continuous variables the variability needs to be estimated - Can get from other studies or small pilot study

Sample Size Calculation Comparative Studies State the Null Hypothesis Determine appropriate statistical test (For simplicity use T-test for continuous of chi square for dichotomous) Predict effect size and variability Set α and ß Use the appropriate formula or table

Sample Size Calculation for Descriptive Studies Continuous -Estimate std deviation -Specify precision (width of CI) -Select the confidence level for the interval Dichotomous -Estimate the expected proportion of the variable of interest (if > 50% calculate based on proportion not expected to have the characteristic) -Select the CI width -Select the confidence for the interval

Other Considerations Account for dropouts Ordinal variables especially if 5-6 groups can be treated as continuous Survival analysis Matching Equivalence studies

Strategies for Minimizing Sample Size Use continuous variables Paired measurements (consider measuring the change) Use more precise variables Use unequal group sizes N = [(c+1)/2c] x n (c = controls per cases) Use more common outcome

Errors to Avoid Dichotomous outcomes can appear continuous when expressed as a percentage Sample size is for those who complete the study not those enrolled Tables assume equal numbers in both groups (if in doubt use formulae) For continuous variables use the standard deviation best associated with the outcome Do the calculation before you start your study and use it to plan Cluster data is confusing and needs a statistical consultation

Questions and Answers