1. META-ANALYSIS OF RESEARCH
LECTURE 7
EXPERIMENTAL DESIGN EFFECTS
Victor L. Willson, Instructor

2. MODERATORS
Moderators are typically considered categorical variables for which effects differ across categories or levels
In a limited form, this can be considered a treatment-moderator interaction
Moderator analysis is more general in the sense that any parameters of a within-category analysis may change across categories (multigroup analysis concept in Structural Equation Modeling)

3. Moderator Analysis- QBetween
Analog to ANOVA- split into Qbetween and Qwithin
QB = wiEi2– (wiEi)2 /wi
where Ei is the mean for category i and wi is the total weight function for Ei
Remember that you constructed a mean effect for a study; the weight function for that mean effect is the sum of the weights that made up the mean: Ei = wjgj/wj for J effects in study I
wi = wj

4. Moderator Analysis- QWithin
Analog to ANOVA- split into Qbetween and Qwithin
QW = wj(i)(Ej(i) - MeanEi)2
I j
where MeanEi is the mean for each category i, Ej(i) is an effect j in category i and wj(i) is the weight function for the jth effect in category i
This is analogous to the within-subjects term in ANOVA
Lipsey and Wilson do not give a very good equation for this on p confusing

5. Computational Issues
The excel file “Meta means working COMPUTATIONS” provides a workbook to compute such effects
An exemplar is shown below, is in your set of materials
Computation of QB and QW are done from the summary data of Hedge’s g and sample sizes

7. Moderator Example
For our Storybook reading example, we can break the effect into two design types:
1 = no baseline equivalence
2 = baseline equivalence
Wasik = 2
Coyne = 1
Justice = 2
Fielding = 1

8. Moderator Example Select “Meta means working COMPUTATIONS” excel file
Reduce the number of studies to 2 in Design 1 and 2 in design 2
Insert the Hedge’s g effects, Cntrl N, Trmt N into the correct boxes, all other effects will be correctly computed

9. All computations per Lipsey & Wilson
Storybook Reading Design Moderator effect
STUDY ID
Hedges's G SE W
Ctrl N
Trmt N N
Design
E*W
Sum W(group)
Group Means
Wgp* Mgp
Wgp *Mgp^2
(Wgp*Mgp)^2
Wtd Within GP SS 3
0.6200
0.26
15.21 30 34 64 1
9.4299
0.3085
8.4544
2.6083
1.4757 4
0.29
12.19 23 26 49
-0.976
1.8406
1.4000
0.20
24.89 61 63
124 2
34.852
1.2234
0.7763
0.6100
0.37
7.17 15
4.3717
2.6966
SumEW(gp1)=
SumEW(gp2)=
39.224
3.3163
3.4729
All computations per Lipsey & Wilson
QB=
df=
1.0000
p(QB)=
0.0000
QW=
6.7893
5.0000
p(QW)=
0.2368 Q=
QB sig., two design means are different
QW nonsig., homogeneous effects within the two design categories