H676 Week 3 – Effect Sizes Additional week for coding? Need to cover subgroups (next week) Need to cover complex data (Oct 27th) Move submission deadline to 7am Oct 30th Quick run through ES ideas and formulae CMA video on data entry Ryan – fixed- and random-effects models CMA video on MA with means

Strengths of Meta-analysis increased statistical power particularly important when the effects are likely to be small and thus likely to be missed by individual studies ability to investigate reasons for statistical variations between studies and to determine whether this variation is due to chance ability to weight information from studies according to the amount of information they contain (weighted averages) increased precision in estimating the overall effect size ability to systematically investigate differences between studies and groups of studies and to explore their causes

The main steps of a MA Accrue studies that meet pre-specified inclusion-exclusion criteria Calculate a standard ES for each study included in the review Pool these ESs to produce an overall summary ES Check for homogeneity/heterogeneity Try to explain heterogeneity if it is present

Effect Size The second step in MA is to obtain/calculate a common effect size across studies

Computing Effect Sizes

Standardized Mean Difference Notes: Cohen’s d and Hedge’s g use pooled SD; Glass’s △ (delta) used SD of control group

Two formulae for the pooled SD For Cohen’s d, the pooled SD is the biased maximum likelihood estimate of the SD: For Hedge’s g, the pooled SD is the unbiased least squares estimate of the SD: These notes and formulae are from DeCoster (2009). Borenstein et al. (2009) use the RH formula for Cohen’s d, and refer to g only with the correction on the next slide, which DeCoster and others refer to as g*.

Adjusting g for small sample size (g*) From DeCoster, 2009. Use formula 6.8 to adjust g if you calculate g from other test statistics

Calculating g from other statistics See DeCoster (2009), Sections 6.2 – 6.5, for other formulae and details.

Correlation as Effect Size

Odds Ratio as Effect Size

Choosing a Calculation Method Method depends on what data/statistics are reported/available Best methods for calculating g are:

2nd and 3rd classes of methods to calculate g From DeCoster, 2009, Section 7.2

Calculating correlation effect sizes

Basics of meta-analysis synthesis Once you have all of those effect sizes, what do you do with them?

First, some transformations (Already covered earlier)

Precision of the Mean Effect Size e.g., those from larger studies

Inverse Variance Weights

Inverse Variance Weighted Mean ES Note that this differs for fixed- and random-effects models

Other possible adjustments to ESs Corrections for attenuation. See Hunter & Schmidt (1990) for a more complete list of other biases that might influence effect sizes, and ways for correcting them. Increase estimated ES because of research design limitations/quality ????? Increase est ES if correlation of dichotomous variables: Increase est ES if measures are unreliable: Or if there is a restriction of range: where

Multiple dependent variables within studies It’s very common for studies to include different measures of a construct or multiple dependent variables (different constructs) Three ways of dealing with them: Usually take the mean or median, but that is usually conservative Rosenthal & Rubin (1986) formulae for combining correlations See DeCoster, 2009, Section 7.5.

Basic Inferential Statistics

Homogeneity Testing

Computation of the Homogeneity Q Statistic

Alternatives to Q – I2 &T2 T2 = variance of ESs I2 = proportion of dispersion due to true differences between studies T2 = estimate of between-study variance

Other steps in conducting a MA