Power Analysis and Meta-analysis Jaehun Jung 10-27-201 H676

Objectives Discuss the factors that determine power and explore how the value of these factors may change as we move from a primary study to meta-analysis Review the process of power analysis for primary studies, and then show how the same process can be extended for meta- analysis. Answer the question “how many studies do you need?” and retrospective statistical power.

What it is and why it is important Power is the probability of detecting an effect, given that the effect is really there. It is the probability of rejecting the null hypothesis when it is in fact false. Is a good way of making sure that you have thought through every aspect of your study and the statistical analysis before you start conducting reviews.

When is it used? In primary studies In meta-analysis Determine appropriate sample size In meta-analysis Already control over some variables (i.e., sample size and # of studies meet the inclusion criteria) Use it when figure it out how many studies we need to include (Valentine’s article)

Factors that Affect Power in Primary Expected effect size Sample size Alpha Power These four things are related such that each is a function of the other three. If three of these values are fixed, the fourth is completely determined

Example Sample size = 25 Effect size = .30 Alpha = .05 Power? 0.1840 => way low power!

Power Analysis in Meta-analysis Logic of power analysis for meta-analysis is similar to primary The precision reflect both sample size and # of studies Differences in precision whether it is fixed- or random-effect Throughout literature review, you might get sense of the factors (i.e. effect size, sample size and # of studies that meet your criteria)

Factors Affecting Power in Meta-analysis Expected effect size Sample Size and # of Studies Alpha Fixed- or Random-effects Why does it matter?

Power Analysis in Fixed Effect Model The same formula and process, but….. If all studies had the same variance, then Vm = Vy/k So, the Lambda will be different

Power Analysis in Random Effect Model The same formula and process, but….. Within- and between-studies variance Obtain within-studies variance using same procedures as for fixed- Obtain between-studies variance through pilot study, but..... Hedges and Pigott propose a convention Small = 1.33 * the within-study variance Medium = 1.67 * the within-study variance Large = 2 * the within-study variance

Example Sample size = 25 Effect size = .30 Alpha = .05 K = 10 Random-effect Model

Power for a test of homogeneity Asks whether or not the between-studies dispersion is more than would be expected by chance Ratio of between- and within-studies variance Number of studies Alpha

Example (Random-effect Model) # of studies = 6 Large amount of dispersion Alpha = .05 Use function CHINV(alpha, df) for critical value Power=1 – CHIDIST (x, df) Power = 0.3541

Discussion Questions 1 Is there anyone who conduct power analysis on your study? Is there anyone who want to share with your power analysis? What is your expected effect size? What is the average sample size? How many research do you have? What is your desired power in your study?

How many studies are needed? Power = 1 – NORMSDIST(1.64 - Lambda) 80% = 1 – NORMSDIST(-.842) => yield 0.2 * if it one-side Thus….. Lambda = 1.64 + 0.842 = 2.482

Retrospective Power Analysis Retrospective power analysis using observed values can not add information to the analysis already done. However, retrospective power analysis using some other value that is meaning in the context can add information regarding results.

Confidence Interval Estimation Lower bound of the CI tells us whether the result is statistically significant and minimum likely size of the effect Upper bound of the CI gives us to assess the “best case” for the intervention’s impact. The Width of the CI provides the full range of plausible parameters that are consistent with the results of the mata- analysis

Discussion Question 2 Should a synthesis be carried out if power is low? Yes, it should! But why?