1. Power Analysis and Meta-analysis
Jaehun Jung
H676

2. Objectives
Discuss the factors that determine power and explore how the value of these factors may change as we move from a primary study to meta-analysis
Review the process of power analysis for primary studies, and then show how the same process can be extended for meta- analysis.
Answer the question “how many studies do you need?” and retrospective statistical power.

3. What it is and why it is important
Power is the probability of detecting an effect, given that the effect is really there.
It is the probability of rejecting the null hypothesis when it is in fact false.
Is a good way of making sure that you have thought through every aspect of your study and the statistical analysis before you start conducting reviews.

4. When is it used? In primary studies In meta-analysis
Determine appropriate sample size
In meta-analysis
Already control over some variables (i.e., sample size and # of studies meet the inclusion criteria)
Use it when figure it out how many studies we need to include (Valentine’s article)

5. Factors that Affect Power in Primary
Expected effect size
Sample size
Alpha
Power
These four things are related such that each is a function of the other three.
If three of these values are fixed, the fourth is completely determined

6. Example Sample size = 25 Effect size = .30 Alpha = .05
Power? => way low power!

7. Power Analysis in Meta-analysis
Logic of power analysis for meta-analysis is similar to primary
The precision reflect both sample size and # of studies
Differences in precision whether it is fixed- or random-effect
Throughout literature review, you might get sense of the factors
(i.e. effect size, sample size and # of studies that meet your criteria)

8. Factors Affecting Power in Meta-analysis
Expected effect size
Sample Size and # of Studies
Alpha
Fixed- or Random-effects
Why does it matter?

9. Power Analysis in Fixed Effect Model
The same formula and process, but…..
If all studies had the same variance, then Vm = Vy/k
So, the Lambda will be different

10. Power Analysis in Random Effect Model
The same formula and process, but…..
Within- and between-studies variance
Obtain within-studies variance using same procedures as for fixed-
Obtain between-studies variance through pilot study, but.....
Hedges and Pigott propose a convention
Small = 1.33 * the within-study variance
Medium = 1.67 * the within-study variance
Large = 2 * the within-study variance

11. Example Sample size = 25 Effect size = .30 Alpha = .05 K = 10
Random-effect Model

12. Power for a test of homogeneity
Asks whether or not the between-studies dispersion is more than would be expected by chance
Ratio of between- and within-studies variance
Number of studies
Alpha

13. Example (Random-effect Model)
# of studies = 6
Large amount of dispersion
Alpha = .05
Use function CHINV(alpha, df) for critical value
Power=1 – CHIDIST (x, df)
Power =

14. Discussion Questions 1
Is there anyone who conduct power analysis on your study?
Is there anyone who want to share with your power analysis?
What is your expected effect size?
What is the average sample size?
How many research do you have?
What is your desired power in your study?

15. How many studies are needed?
Power = 1 – NORMSDIST( Lambda)
80% = 1 – NORMSDIST(-.842) => yield 0.2 * if it one-side
Thus….. Lambda = = 2.482

16. Retrospective Power Analysis
Retrospective power analysis using observed values can not add information to the analysis already done.
However, retrospective power analysis using some other value that is meaning in the context can add information regarding results.

17. Confidence Interval Estimation
Lower bound of the CI tells us whether the result is statistically significant and minimum likely size of the effect
Upper bound of the CI gives us to assess the “best case” for the intervention’s impact.
The Width of the CI provides the full range of plausible parameters that are consistent with the results of the mata- analysis

18. Discussion Question 2
Should a synthesis be carried out if power is low?
Yes, it should! But why?