1. H676 Week 3 – Effect Sizes Additional week for coding?
Need to cover subgroups (next week)
Need to cover complex data (Oct 27th)
Move submission deadline to 7am Oct 30th
Quick run through ES ideas and formulae
CMA video on data entry
Ryan – fixed- and random-effects models
CMA video on MA with means

2. Strengths of Meta-analysis
increased statistical power
particularly important when the effects are likely to be small and thus likely to be missed by individual studies
ability to investigate reasons for statistical variations between studies
and to determine whether this variation is due to chance
ability to weight information from studies according to the amount of information they contain (weighted averages)
increased precision in estimating the overall effect size
ability to systematically investigate differences between studies and groups of studies
and to explore their causes

3. The main steps of a MA
Accrue studies that meet pre-specified inclusion-exclusion criteria
Calculate a standard ES for each study included in the review
Pool these ESs to produce an overall summary ES
Check for homogeneity/heterogeneity
Try to explain heterogeneity if it is present

4. Effect Size
The second step in MA is to obtain/calculate a common effect size across studies

5. Computing Effect Sizes

6. Standardized Mean Difference
Notes: Cohen’s d and Hedge’s g use pooled SD; Glass’s △ (delta) used SD of control group

7. Two formulae for the pooled SD
For Cohen’s d, the pooled SD is the biased maximum likelihood estimate of the SD:
For Hedge’s g, the pooled SD is the unbiased least squares estimate of the SD:
These notes and formulae are from DeCoster (2009).
Borenstein et al. (2009) use the RH formula for Cohen’s d,
and refer to g only with the correction on the next slide,
which DeCoster and others refer to as g*.

8. Adjusting g for small sample size (g*)
From DeCoster, Use formula 6.8 to adjust g if you calculate g from other test statistics

9. Calculating g from other statistics
See DeCoster (2009), Sections 6.2 – 6.5, for other formulae and details.

10. Correlation as Effect Size

11. Odds Ratio as Effect Size

12. Choosing a Calculation Method
Method depends on what data/statistics are reported/available
Best methods for calculating g are:

13. 2nd and 3rd classes of methods to calculate g
From DeCoster, 2009, Section 7.2

14. Calculating correlation effect sizes

15. Basics of meta-analysis synthesis
Once you have all of those effect sizes, what do you do with them?

16. First, some transformations
(Already covered earlier)

17. Precision of the Mean Effect Size
e.g., those from larger studies

18. Inverse Variance Weights

19. Inverse Variance Weighted Mean ES
Note that this differs for fixed- and random-effects models

20. Other possible adjustments to ESs
Corrections for attenuation. See Hunter & Schmidt (1990) for a more complete list of other biases that might influence effect sizes, and ways for correcting them.
Increase estimated ES because of research design limitations/quality
?????
Increase est ES if correlation of dichotomous variables:
Increase est ES if measures are unreliable:
Or if there is a restriction of range:
where

21. Multiple dependent variables within studies
It’s very common for studies to include different measures of a construct or multiple dependent variables (different constructs)
Three ways of dealing with them:
Usually take the mean or median, but that is usually conservative
Rosenthal & Rubin (1986) formulae for combining correlations
See DeCoster, 2009, Section 7.5.

22. Basic Inferential Statistics

23. Homogeneity Testing

24. Computation of the Homogeneity Q Statistic

25. Alternatives to Q – I2 &T2 T2 = variance of ESs
I2 = proportion of dispersion due to true differences between studies
T2 = estimate of between-study variance

26. Other steps in conducting a MA