1. Meta-analysis statistical models: Fixed-effect vs. random-effects
Ryan Singh, MPH
H676
October 13, 2016

2. Two statistical models for MA
Intent of MA is to get the mean of the effect sizes from each study (summary effect)-2 models:
Fixed-effect model
-one true effect across all studies
Random-effects model
-true effect varies between studies
True effect size (Θ) – effect in underlying population, and the effect size that would be observed if n=∞
Observed effect (Y) – effect size that is actually observed

3. FIXED-EFFECT MODEL RANDOM-EFFECTS MODEL
-one true effect
-true effect varies
-differences b/w studies due to sampling error only
-differences b/w studies more than sampling error
-large studies more influential in ES estimation
-weight between all studies is more balanced
-account for within study uncertainty
-account for within study AND between study uncertainty
-summary effect=common effect
-summary effect=mean effect
-null hypothesis=0 effect in every study
-null hypothesis=0 mean effect
-summary effect limited to identified population
-summary effect more generalizable

4. Fixed-effect model
All factors that could affect ES same across studies
True ES is same
Only variation from random (sampling) error
Assume that if n=∞, then no sampling error (draw)
When to use
All included studies are functionally identical
Goal is to get ES for identified population
*Need to estimate true ES based on observed effects

7. Fixed-effect model To perform FE analysis:
Start with observed effects to estimate population effect
Observed effect is the sum of the true effect and sampling error
Assign appropriate weights to all studies-inverse of each study’s variance
Calculate weighted mean (summary effect) with observed ESs and weights
Calculate all summary statistics (V, SE, CI, Z, p)

8. Precision and weighting
Addresses the accuracy of the predicted/observed effect as an estimate of the true effect
encompasses variance, standard error, & confidence interval
Dependent on sample size, study design, & ?
Weights assigned to studies to minimize within study error

9. Why do some meta-analysts suggest setting w equal to the sample size, rather than the inverse of the variance (according to DeCoster)?
I have no clue.

10. Fixed-effect model
What would happen to observed ES (Y) if n=∞?

11. Fixed-effect model
What would happen to observed ES (Y) if n=∞? sampling error=0, so observed ES=true ES What is the relationship between sample size and variance?

12. Fixed-effect model
What would happen to observed ES (Y) if n=∞? sampling error=0, so observed ES=true ES What is the relationship between sample size and variance? small sample=large variance

13. Random-effects model
Studies not similar enough to assume true ES is same across studies
Studies differ for a variety of reasons
Assume true effects are normally distributed (draw picture)
Goal-use all observed effects to estimate mean of all true ESs
Distance b/w overall mean and observed effect in a study consists of 2 parts:
Sampling error
True variation in ES
Observed effect is the sum of: 1) the overall mean, 2) deviation of true effect from overall mean, & 3) deviation of observed effect from true effect
Variance for both needs to be considered

16. Random-effects model To perform RE analysis:
Start with observed effects to estimate population effect
Observed effect is the sum of the mean and total error
Assign appropriate weights to all studies-inverse of each study’s variance
Calculate weighted mean (summary effect) with observed ESs and weights
Calculate all summary statistics (V, SE, CI, Z, p)

17. Random-effects model To calculate variance, need:
Within study variance=sampling error
Between study variance (τ2)
AKA variance of ESs across studies
Dependent on SD of distribution of true effects across studies
Applies to all studies in MA
Can be estimated
*does not account for within study variance

21. Additional questions
Why are smaller studies less significant in FE analysis? Conversely, why does the weight of each study in RE analysis matter? Thinking back to our readings on study inclusion, what issues arise with including less studies (due to design flaw perhaps) and the limitation of smaller analyses? OR What limitations are there with our (potentially) smaller samples for this class?

22. Additional questions
Why are smaller studies less significant in FE analysis? better information about same ES from larger studies Conversely, why does the weight of each study in RE analysis matter? each study represents a different effect-want to represent each one Thinking back to our readings on study inclusion, what issues arise with including less studies (due to design flaw perhaps) and the limitation of smaller analyses? OR What limitations are there with our (potentially) smaller samples for this class? estimate of between study variance has poor precision

23. FIXED-EFFECT MODEL RANDOM-EFFECTS MODEL
-one true effect
-true effect varies
-differences b/w studies due to sampling error only
-differences b/w studies more than sampling error
-large studies more influential in ES estimation
-weight between all studies is more balanced
-account for within study uncertainty
-account for within study AND between study uncertainty
-summary effect=common effect
-summary effect=mean effect
-null hypothesis=0 effect in every study
-null hypothesis=0 mean effect
-summary effect limited to identified population
-summary effect more generalizable

24. Group Discussion Topic?
How will you go about finding your effect size?
Which model (FE or RE) will you be using?
Questions/issues you have at this point?