Effect Sizes (ES) & Meta-Analyses ES – using Computator ES transformations Meta Analysis

Using Computator page “F or t or means or r -> d” Top is the “definitional formula” using means, std & n for each group Below uses the MSe from an ANOVA, rather than the std for each group.

Using Computator page “F or t or means or r -> d” Top uses “F” while bottom uses “t”. Remember t2 = F

Using Computator page “F or t or d ->r” Top uses “F” while bottom uses “t”. Remember: t2 = F Also: df-error = n1 + n2 -2

Using Computator page “pr_means-> r & d” This can be used for either a 2-group design, or to select any pair of conditions means from a larger design. Make sure you are using the “same conditions” from each study in your meta analysis!

Remember that we can transform “back & forth” between r & d (well…kinda…) Using Computator page “F or t or means or r -> d” Using Computator page “F or t or d ->r” Notice the transformations are not “symmetrical”. When your only way to get “d” is from “r”, you are likely to overestimate that d effect size.

Transformations The most basic meta analysis is to take the average of the effect size from multiple studies as the best estimate of the effect size of the population of studies of that effect. As you know, taking the average of a set of values “works better” if the values are normally distributed! Beyond that, in order to ask if that mean effect size is different from 0, we’ll have to compute a standard error of the estimated mean, and perform a Z-test. The common formulas for both of these also “work better” if the effect sizes are normally distributed. And therein lies a problem! None of d, r & odds ratios are normally distributed!!! So, it is a good idea to transform the data before performing these calculations !!

Transformations -- d d has an upward bias when sample sizes are small the extent of bias depends upon sample size the result is that a set of d values (especially with different sample sizes) isn’t normally distributed a correction for this upward bias & consequent non-normality is available 3 ES = d * 1 - -------- 4N-9 Excel formula is d * ( 1 - (3 / ((4*N) – 9)))

Transformations -- r r is not normally distributed and it has a problematic standard error formula. Fisher’s Zr transformation is used – resulting in a set of ES values that are normally distributed 1 + r ES = .5 * ln ------- 1 - r Excel formula is FISHER(r) all the calculations are then performed using the ES the final estimate of the population ES can be returned to r using another formula (don’t forget this step!!!) e 2ES - 1 r = ------------ e 2ES + 1 Excel formula is FISHERINV(ES)

Steps in a Meta Analysis (getting started) The top row will hold the “name” of the values in each row Let’s work with a meta analysis with 10 ES values Let’s work with “r” values Put the r from each study/analysis in rows 2-11 of column “A” 2. Put the sample size (n) of each rows 2-11 of column “B”

Steps in a Meta Analysis (making the columns) Column “C” is the normal transformation of “A” xls code = FISHER(A2) Column “D” is inverse variance weights xls code =B2-3 5. Column “E” is inverse variance weighted ES xls code =C3*D3 6. Column “F” is inverse variance weighted ES2 xls code = C3*D3^2

Steps in a Meta Analysis (getting the sums) Add a cell to the bottom of column “D” that is the sum of the inverse variance weights xls code =SUM(D2:D11) should be in D12 Add a cell to the bottom of column “E” that is the sum of the weighted ES xls code =SUM(E2:E11) should be in E12 Add a cell to the bottom of column “F” that is the sum of the weighted ES2 xls code =SUM(F2:F11) should be in F12

Steps in a Meta Analysis (getting Meta-analytic “r”) 10. Compute the weighted mean of the normal transformations xls code = E12/D12 (put this in E14) 11. Transform this back to “r” xls code = FISHERINV(E14) (put this in E15) Ta Da !!!!!

Steps in a Meta Analysis (getting significance test of Meta-analytic “r”) 12. Compute the standard error of the estimated ES xls code = SQRT( 1 / D12) (put this in E17) 13. Compute Z for significance test xls code = E14/E17 (put this in E18) 14. Compute p-value for Z-test xls code = NORMDIST(0,ABS(E18),1,TRUE) * 2 (put this in G18) Ta Da !!!!!