Chapter 10 - Part 1 Factorial Experiments

Nomenclature We will use the terms Factor and Independent Variable interchangeably. They mean the same thing. The term “factorial analysis of variance” simply means the analysis of variance when there are multiple factors (multiple independent variables.) I will sometimes use the phrase Factor 1 or 2 interchangeably with Independent Variable 1 or 2. But to prevent confusion, whenever the term is abbreviated I will use IV 1 or IV 2, not F 1 or F 2.

Two-way Factorial Experiments In Chapter 10, we are studying experiments with two independent variables, each of which will have multiple levels. We call each independent variable a factor. The first IV is called Factor 1 or IV 1. The second IV is called Factor 2 or IV 2. So, in this chapter we will study two factor, unrelated groups experiments.

Conceptual Overview In Chapter 9 you learned to do the F test comparing two estimates of sigma 2, MS B and MS W. That is what you do in simple experiments, those with only one independent variable. In the single IV, unrelated groups experiments in Ch. 9, F = MS B /MS W That is one version of a more generic formula. The generic formula tells us what to do in the two (or more) factor case. Here is the basis for the generic formula

Generic formula for the unrelated groups F test F = SAMP.FLUC. + (?) ONE SOURCE OF VARIANCE) (SAMPLING FLUCTUATION) Which can also be written as F = (ID + MP + (?) ONE SOURCE OF VARIANCE) (ID+ MP) Let me explain why.

The denominator of the F test The denominator in the F test reflects variation within each group because of random individual differences (ID) and measurement problems (MP). Since everyone in the same group is treated the same, only ID + MP can contribute to within group variation. So, MS W can not reflect the effect of any independent variable or combination of independent variables. Random sampling fluctuation is based on random individual differences and measurement problems (ID + MP). Thus, in the unrelated groups F and t test, our best index of the random effects of individual differences and measurement problems, the basis of random sampling fluctuation, is MS W, our least squares, unbiased estimate of sigma 2.

Denominator = MS W In the F tests we are doing, F tests for unrelated groups, MS W serves as our best estimate of sigma 2. To repeat, in computing MS W, we compare each score to the mean of its own, specific group. Everyone in each specific group is treated the same. So, the only reasons that scores differ from the other scores in their group and their own group means is that people differ from each other (ID) and there are always measurement problems (MP). MS W = ID +MP

Numerator of the F ratio: Generic formula Numerator of the F ratio is an estimate of sigma 2 that reflects sampling fluctuation + the possible effects of one difference between the groups. In Ch. 9, there was only one difference among the ways the groups were treated, the different levels of the independent variable (IV) MS B reflected the effects of random individual differences (there are different people in each group), random measurement problems, and the effects of the independent variable.In Ch. 9, we could write that as MS B = ID + MP + (?)IV

To repeat, in the single factor analysis of variance F = (ID + MP + ?IV) (ID + MP) Both the numerator and denominator reflect the same elements underlying sampling fluctuation The numerator includes one, and only one, systematic source of variation not found in the denominator.

This allows us to conclude that: IF THE NUMERATOR IS SIGNIFICANTLY LARGER THAN THE DENOMINATOR, THE SIZE DIFFERENCE MUST BE MUST BE CAUSED BY THE ONE ADDITIONAL THING PUSHING THE MEANS APART, the IV. But notice there can’t be more than one thing in the numerator that does not appear in the denominator to make that conclusion inevitable.

Why we can’t use MS B as the numerator in the multifactor analysis of variance In the two factor analysis of variance, the means can be pushed apart by: –The effects of the first independent variable (IV 1 ). –The effects of the second independent variable (IV 2 ) –The effects of combining IV 1 and IV 2 that are above and beyond the effects of either variable considered alone (INT) –Random sampling fluctuation (ID + MP)

So if we compared MS W to MS B in a two factor experiment, here is what we would have. F = (ID + MP + ?IV 1 + ?IV 2 + ?INT) (ID + MP) That’s not an F test. In an F test the numerator must have one and only one source of variation beyond sampling fluctuation. HERE THERE ARE THREE OF THEM! Each of these three things besides sampling fluctuation could be pushing the means apart. So, the F ratio would be meaningless.

To use the F test with a two factor design, we must create 3 numerators to compare to MS W, each comprising ID + MP + one other factor.

HOW? To obtain our 3 numerators for the F test, we divide (analyze) the sums of squares and degrees of freedom between groups (SS B & df B ) into component parts. Each part must contain only one factor along with ID and MP. Then each component will yield an estimate of sigma 2 that can be compared to MS W in an F test.

Write out the answer to these two questions without reading the answer from the slides: Why can’t you compare MS B to MS W in the two factor, unrelated groups F test? What must you do instead?

ANSWER 1: If we compared MS W to MS B in a two factor experiment, here is what we would have. F = (ID + MP + ?IV 1 + ?IV 2 + ?INT) (ID + MP) That’s not an F test. In an F test the numerator must have one and only one source of variation beyond sampling fluctuation. HERE THERE ARE THREE OF THEM! Each of these three things besides sampling fluctuation could be pushing the means apart. So, the F ratio would be meaningless.

ANSWER 2:We must take apart (analyze) the sums of squares and degrees of freedom between groups (SS B & df B ) into component parts. Each part must contain only one factor along with ID and MP. Then each component will yield an estimate of sigma 2 that can be compared to MS W in an F test.

Here is how we divide SS B and df B into their component parts: First, we create a way to study the effects of factor 1 alone. To do that, we combine groups so that the resulting, larger aggregates of participants differ only because they received different levels of the first independent variable, IV 1. Each such combined group will include an equal number of people who received the different levels of IV 2. So the groups are the same in that regard. They differ only on how they were treated on the first independent variable, IV 1.

Computing MS IV1, one of the three numerators in a two factor F test If we find the differences between each person’s combined group mean and the overall mean, square and sum them, we will have a sum of squares for the first independent variable (SS IV1 ). Call the number of levels of an independent variable L. df for the combined group equals the number of levels of its IV minus one (L IV – 1). An estimate of sigma 2 that includes only ID + MP + (?) IV 1 can be computed by dividing this sum of squares by its degrees of freedom, as usual. MS IV1 = SS IV1 /df IV1 = (ID + MP + ?IV 1 )

Once you have MS IV1, you have one of the three F tests you do in a two factor ANOVA F = MS IV1 /MS W

Then you do the same thing to find MS IV2 You combine groups so that you have groups that differ only on IV 2. You compare each person’s mean for this combined group to the overall mean, squaring athe differences for each person and then summing them for the entire sample to get SS IV2. Degrees of freedom = the number of levels of Factor 2 minus 1 (df IV2 = L IV2 – 1). Then MS IV2 = SS IV2 /df IV2 F IV2 = MS IV2 /MS W

What’s left is the interaction. Remember, we are subdividing SS B and df B into their three component parts. We have already computed SS IV1, SS IV2, df IV1, and df IV2. WHAT’S LEFT? The part of SS B and df B that hasn’t been accounted for is the sum of squares and degrees of freedom for the interaction. The interaction involves the means being pushed apart by the two independent variables having a different effect when present together than either has by itself alone. For example, a moderate dose of alcohol can make you intoxicated. A moderate dose of barbituates can make you sleepy. Taken together they multiply each others’ effects and the interaction of the two drugs can easily make you dead.

How to compute the interaction. To compute the sum of squares and df for the interaction, we find that part of the sum of squares between groups and degrees of freedom between groups that are not accounted for by Factor 1 (IV 1 ) and Factor 2 (IV 2 ) THAT IS, Y0U SUBTRACT THE SUMS OF SQUARES AND df YOU’VE ALREADY ACCOUNTED FOR ( SS IV1, SS IV2, df IV1, and df IV2 ) FROM THE SUM OF SQUARES AND DEGREES OF FREEDOM BETWEEN GROUPS (SS B & df B ). WHAT’S LEFT IS SS INT & df INT, THE SUM OF SQUARES AND DEGREES OF FREEDOM FOR THE INTERACTION.

Look at that another way: The whole is equal to the sum of its parts. SS B and df B are the between group sum of squares and degrees of freedom. Each is composed of three parts, SS IV1, SS IV2, SS INT and df IV1, df IV2, and df INT. So if we subtract the SS and df for factors 1 & 2 from SS B and df B, what is left is the sum of squares and df for the interaction. SS INT = SS B -(SS IV1 + SS IV2 )=SS B - SS IV1 - SS IV2 df INT = df B -(df IV1 + df IV2 )=df B - df IV1 - df IV2

Designs of 2 factor studies

Analysis of Variance Each possible combination of IV 1 and IV 2 creates an experimental group. Participants are randomly assigned to each of the treatment groups. Each experimental group is treated differently from all other groups in terms of one or both factors. For example, if there are 2 levels of the first variable (Factor 1or IV 1 ) and 2 of the second (IV 2 ), we will need to create 4 groups (2x2). If IV 1 has 2 levels and IV 2 has 3 levels, we need to create 6 groups (2x3). If IV 1 has 3 levels and IV 2 has 3 levels, we need 9 groups. Etc.

Some nomenclature Two factor designs are identified by simply stating the number of levels of each variable. So a 2x4 design (called “a 2 by 4 design”) has 2 levels of IV 1 and 4 levels of IV 2. A 3x2 design has 3 levels of IV 1 and 2 levels of IV 2. And so on. Which factor is called IV 1 and which is called IV 2 is arbitrary (and up to the experimenter).

Example of 2 x 3 design To make it more concrete, assume we are testing new treatments for Generalized Anxiety Disorder. In a two factor design we examine the effects of cognitive behavior therapy vs. a social support group among GAD patients who receive Ativan, Zoloft or Placebo. Thus, IV 1 has 2 levels (CBT/Social Support) while IV 2 has 3 levels (Ativan/Zoloft/Placebo) So, we would form 2 x 3 = 6 groups to do this experiment. Half the patients would get CBT, the other half get social support. A third of the CBT patients and one-third of the Social Support patients also get Ativan. Another third of those who receive CBT and one-third of those who get social support also receive Zoloft. The final third in each psychotherapy condition get a pill placebo.

A 2 x 3 design yields 6 groups. Let’s say you have 24 participants. Four are randomly assigned to each group. Here are the six treatment groups: –CBT + Ativan –CBT + Zoloft –CBT + Placebo –Social support + Ativan –Social support + Zoloft –Social support + Placebo

Example: Experiment Outline Population: Outpatients with Generalized Anxiety Disorder Subjects: 24 participants divided equally among 6 treatment groups. Independent Variables: –Factor 1: Psychotherapy: CBT or Social Support (SoSp) –Factor 2: Medication: Ativan, Zoloft, or Placebo Groups: 1=CBT + Ativan; 2=CBT + Zoloft; 3=CBT + Placebo; 4=SoSp + Ativan; 5=SoSp + Zoloft; 6=SoSp + Placebo. Dependent variable: Anxiety remaining after treatment. Lower scores equal less anxiety and a better outcome.

A 3X2 STUDY Type of drug Type of therapy Ativan Zoloft Placebo SoSp CBT

Effects We are interested in the main effects of type of psychotherapy and type of drug. Do participants get better with CBT and not with SoSp or the reverse? Do people get better when they get a mild tranquilizer (Ativan) an SSRI (Zoloft) or Placebo. We are also interested in assessing how combining different levels of both factors affect the response in ways beyond those that can be predicted by considering the effects of each IV separately. So, we are interested in the interaction of the independent variables.

MS W Drug Given Type of therapy Ativan Zoloft Placebo SoSp CBT Compare each score to the mean for its group.

Mean Squares Within Groups

MS W SS w = df W = 18 MS W = 132/18 = 7.33

Then we compute a sum of squares and df between groups This is the same as in Chapter 9 The difference is that we are going to subdivide SS B and df B into component parts. Thus, we don’t use SS B and df B in our Anova summary table, rather we use them in an intermediate calculation.

Sum of Squares Between Groups (SS B ) Type of Drug Type of Therapy Ativan Zoloft Placebo SoSp CBT Compare each group mean to the overall mean.

Means for GAD study Type of Drug Diet Type Ativan Zoloft Placebo Social Support Cognitive BT

Sum of Squares Between Groups (M=10.00, SS B =544, df B =5)

Next, we answer the questions about each factor having an overall effect. To get proper between groups mean squares we have to divide the sums of squares and df between groups into components for factor 1, factor 2, and the interaction. Let’s just look at factor 1. Our question about factor 1 was “Do people undergoing different therapies have differential responses to any task?” We can group participants into all those who were treated with CBT and those treated with Social Support.

Forming Groups that Differ only on Factor 1 Pretend that the experiment was a simple, single factor experiment in which the only difference among the groups was the first factor (that is, the type of therapy given each group). Create groups reflecting only differences on Factor 1. So, when computing the main effect of Factor 1 (type of psychotherapy), ignore Factor 2 (type of drug). Divide participants into two groups depending solely on whether they were given CBT or Social Support. Then, find the mean of each of the two combined groups (CBT and Social Support).

Computing SS for Factor 1 Next, find the deviation of the mean of the CBT participants from the overall mean. Then sum and square those differences. Then, find the deviation of the mean of the Social Support participants from the overall mean. Then sum and square those differences. The total of the summed and squared deviations the mean of each of the combined groups from M, the overall mean, is the sum of squares for Factor 1. (SS IV1 ).

df IV1 and MS IV1 Compute a mean square that takes only differences on Factor 1 into account by dividing SS IV1 by df IV1. For example, in this experiment, type of therapy was either CBT or Social Support. The two ways participants are treated are called the two “levels” of Factor 1. df IV1 = L 1 – 1 = 2-1 = 1 [where L 1 equals the number of levels (or different variations) of the first factor (IV 1 )].

SS IV1 : Main Effect of Therapy Type Type of drug Therapy Type Ativan Zoloft Placebo SoSp CBT Compare each score’s therapy mean to the overall mean.

Means for GAD study Type of New Drug Diet Type Ativan Zoloft Placebo Social Support Cognitive BT

CBT/SoSp Means, M=10.00 Means: CBT=8.00; SoSp=12.00 CBT SoSp group 8 group 12

SS IV1 =96.00; df IV1 =2-1=1; MS IV1 =

Factor 2 Then pretend that the experiment was a single experiment with only the second factor. Proceed as you just did for Factor 1 and obtain SS IV2 and MS IV2 where df IV2 =L IV2 – 1=3-1=2.

SS IV1 : Main Effect of Drug Type Drug Type Therapy Type Ativan Zoloft Placebo SoSp CBT Compare each score’s Drug Type mean to the overall mean.

Means for GAD study Type of New Drug Diet Type Ativan Zoloft Placebo Social Support Cognitive BT

Means: Ativan=7.00, Zoloft=7.00, Placebo = 16.00, M= n=8 16 n= n=8

SS IV2 =442; df IV2 =2; MS IV2 =221 Ativan No Emb Mild Emb

Computing the sum of squares and df for the interaction. SS B contains all the possible effects of the independent variables in addition to the random factors, ID and MP. Here is that statement in equation form SS B = SS IV1 + SS IV2 + SS INT Rearranging the terms: SS INT = SS B - (SS IV1 +SS IV2 ) or SS INT = SS B - SS IV1 -SS IV2 SS INT is what’s left from the sum of squares between groups (SS B ) when the main effects of the two IVs are accounted for. So, subtract SS IV1 and SS IV2 from SS B to obtain the sum of squares for the interaction (SS INT ). Then, subtract df IV1 and df IV2 from df B to obtain df INT ).

Means Squares for Interaction MS INT REARRANGE

Testing 3 null hypotheses in the two way factorial Anova

Hypotheses for Therapy Type Null Hypothesis - H 0 : There is no effect of Drug Type. The means for anxiety will be the same whether the GAD patients were given CBT or SocialSupport. Experimental Hypothesis - H 1 : The type of psychotherapy a group gets, considered alone, will affect their anxiety.

Hypotheses for Type of Drug Null Hypothesis - H 0 : The three drug types, including the placebo, will have similar effects on reported anxiety. Experimental Hypothesis - H 1 : The three drug types, including the placebo, will have different effects on reported anxiety.

Hypotheses for the Interaction of Type of Therapy and Type of Drug Null Hypothesis - H 0 : There is no interaction effect. Once you take into account the main effects of type of therapy and type of drug, there will be no differences among the groups that can not be accounted for by sampling fluctuation. Experimental Hypothesis - H 1 : There are effects of combining CBT and Social Support with specific drugs can not be predicted from either IV considered alone.

Computational steps Outline the experiment. Define the null and experimental hypotheses. Compute the Mean Squares within groups. Compute the Sum of Squares between groups. Compute the main effects. Compute the interaction. Set up the ANOVA table. Check the F table for significance. Interpret the results.

Steps so far Outline the experiment. Define the null and experimental hypotheses. Compute the Mean Squares within groups. Compute the Sum of Squares between groups. Compute the main effects. Compute the interaction.

What we know to this point SS IV1 =96.00, df IV1 =1; MS IV1 =96.00 SS IV2 =442.00, df IV2 =2; MS IV2 = SS INT =6.00, df INT =2; MS INT =3.00 SS W =132.00, df W =18; MS W =7.33

Steps remaining Set up the ANOVA table. Check the F table for significance. Interpret the results.

Therapy Type – F(1,18)=13.10, p<.01; Drug Type – F(2,18)=30.14, p<.01 Interaction-F(2,18)=0.41, n.s. Type of Therapy Type of Drug SS df MS F p  Interaction Error n.s.

State Results The main effects for CBT vs. Social Support was significant– F(1,18)=13.10, p<.01. The main effect for Ativan vs. Zoloft vs. Placebo was significant - F(2,18)=30.14, p<.01 The interaction was not significant (F(2,18)=0.41, n.s.)

Interpret Significant Results CBT participants were less anxious than were those who received Social Support. Participants who received the two active medications (Ativan and Zoloft had the same mean anxiety ratings after treatment, while those who received the placebo were much more anxious. Describe pattern of means.

Interpret Significant Results These findings are consistent an additive effect of appropriate medication and CBT for Generalized Anxiety Disorder. Both lower anxiety among these patients. Reconcile statistical findings with the hypotheses.