1. Power Analysis in Grant Writing Jill Harkavy-Friedman, Ph.D.

2. Is there a difference?

3. Observed Yes Observed No Expected YesTrue Positive Type II Error β Expected No Type I Error α True Negative

4. POWER IS KNOWLEDGE

5. Power: How likely are you to detect an effect? Sample Size: n How many people will you need? Effect Size: σ (e.g., R 2 ) How much of a difference are you trying to detect? Significance Level (Type I Error): α How much of a risk are you willing to take of saying there is a difference when there none? Type II Error: β or (1- α) How much risk there is of saying that there is no difference when there is a difference.

6. Sample Size: n Depends on Nature of the question & statistic approach  Group differences: m1-m2=0  Correlations: r=0  Regression: R2=0 Feasibility  Economic, staffing, recruitment Power needed

7. Effect Size: σ Amount of difference want to detect  Based on previous literature: average SD  Based on pilot data: SD  Based on size of difference Small: d=.20 Medium:d=.50 Large:d=.80 Statistical difference ≠ Clinical difference

8. 3456 0.0 2.5 5.0 7.5 10.0 Count 11%34%2.5%34%11%2.5% Normal Curve and Distribution of Sample Means Significance Level (Type I Error): α

9. The larger the sample size the greater the power The larger the effect size the greater the power The larger the significance level the greater the power

11. Power and Sample Size Table at α 1=.05 ES.30.40.50.60 Power.70108624129.75123704633.80140805237.85163936143.901931107251.952431389063.9935320012991

13. Data Analysis comes first - power second Determine your hypotheses Determine your analyses Determine the parameters for analysis by hypothesis (i.e., power, ES, α) Conduct power analysis

14. Power analysis will require: Type of analysis Sample size Effect size Significance level Number of groups or factors Plug in the numbers

15. Considerations What is your question? What type of data do you have? What are your hypotheses? What are your resources? Clinical vs. statistical significance How will you present your data? What you would like the news headline to be?

16. SAGES Research Committee, August 2006 RE: Suggestions to Assist with Completing a Winning Application Power Analysis:  In order to minimize the reporting of false-negative data, a power analysis should be performed for sample size determination. Power is the capability of a study to detect a difference if the difference really exists. A type II error occurs when a true difference exists between study populations but there are insufficient numbers of subjects to detect this difference. Any grant submitted without one of the items below will not be eligible for review.

17. Power analysis. Please provide the following data: alpha and beta, sample size needed in each group, what difference is expected. (Example: "A power analysis was performed with a beta of.20 and an alpha of 05. Assuming that a 10% difference exists between patient and control groups, 150 subjects will be needed in each arm. Thus the study would provide an 80% chance that a difference would be detected if one exists.") If a power analysis in not appropriate for the submitted project, a statement should be included explaining why a power analysis is not appropriate for the study. Consultation with a statistician is recommended. However, there are many statistical software programs available

18. What to do when you need more power Increase sample size Reduce number of variables Show your data graphically

19. Power Analysis With n=400, alpha 2 =.05, and a medium effect size (.30) the power will be >.99 for analyses of variance and.96 for zero order correlations 120. The power for a regression analysis that includes 11 variables (i.e. sex, ethnicity, positive symptoms, negative symptoms, aggression, impulsivity, depression, premorbid adjustment, gene marker, family history and substance abuse) with n=400, alpha=.05 and a medium effect size (R 2 =.10) will be greater than.90. We do not anticipate that all 11 variables will contribute significantly to the model. With 6 variables, a more likely model, the power will be >.90. For the exploratory regression analyses conducted within the group of attempters (n=200) the power will be at least.80 to detect a medium effect size (R 2 =.10). For correlational studies among the genetic and biochemical measures (approximate n=100) the power will be.80 for a medium effect size (R 2 =.10) and.95 for a larger medium effect size (R 2 =.25). The increased sample size will now provide the power necessary to consider attempters and nonattempters separately. Analyses examining single attempters, multiple attempters and nonattempters (approximate sample sizes: 120, 80, 200) will still maintain adequate power (power>.70). Grant application by Jill M. Harkavy-Friedman, PhD

20. Free Power and Sample Size Calculation Cohen J. Statistical Power Analysis for the Behavioral Sciences (2nd edition). Hillsdale, New Jersey: Lawrence Erlbaum Associates, Publishers, 1988  http://biostat.mc.vanderbilt.edu/twiki/bin/view/Main/PowerSampleS ize http://biostat.mc.vanderbilt.edu/twiki/bin/view/Main/PowerSampleS ize G*Power 2  http://www.psycho.uni-duesseldorf.de/aap/projects/gpower/ (limited) http://www.psycho.uni-duesseldorf.de/aap/projects/gpower/ G*3  http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/ http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/