1. Chapter 10 - Part 1 Factorial Experiments

2. Two-way Factorial Experiments In Chapter 10, we are studying experiments with two independent variables, each of which will have multiple levels. We call each independent variable a factor. The first IV is called Factor 1 or F 1. The second IV is called Factor 2 or F 2. So, in this chapter we will study two factor, unrelated groups experiments.

3. Conceptual Overview In Chapter 9 you learned to do the F test comparing two estimates of sigma 2, MS B and MS W. F = MS B /MS W That is one version of a more generic formula. The generic formula tells us what to do in the two factor case. Here is the generic formula:

4. Generic formula for the unrelated groups F test F = SAMP.FLUC. + (?) ONE SOURCE OF VARIANCE) (SAMPLING FLUCTUATION) Which can also be written as F = (ID + MP + (?) ONE SOURCE OF VARIANCE) (ID+ MP) Let me explain why.

5. The denominator of the F test The denominator in the F test reflects only random sampling fluctuation. It can not reflect the effect of any independent variable or combination of independent variables. In the unrelated groups F and t test, our best index of sampling fluctuation is MS W, our consistent, least squares, unbiased estimate of sigma 2.

6. The denominator of the F test Sigma 2 underlies sampling fluctuation. If we know sigma 2, we can easily tell how far any sample mean should fall from the overall mean (M) and how far multiple means should fall from each other, on the average. Remember, sigma 2 itself is a function of how different each individual score is from the average score. That is, sigma 2 reflects that each score is not identical to the others in the same population (or, in this case, in the same group), because of random individual differences and random measurement problems (ID + MP).

7. Denominator = MS W In the F tests we are doing MS W serves as our best estimate of sigma 2. Everyone in each specific group is treated the same. So the only reasons that scores differ from their own group means is that people differ from each other (ID) and there are always measurement problems (MP). MS W = ID +MP

8. Numerator of the F ratio: Generic formula Numerator of the F ratio is an estimate of sigma 2 that reflects sampling fluctuation + the possible effects of one difference between the groups. In Ch. 9, there was only one difference among the ways the groups were treated, the different levels of the independent variable (IV) MS B reflected the effects of random individual differences (there are different people in each group), random measurement problems, and the effects of the independent variable. We can write that as MS B = ID + MP + (?)IV

9. So, in the single factor analysis of variance F = (ID + MP + ?IV) (ID + MP) Both the numerator and denominator reflect the same elements underlying sampling fluctuation The numerator includes one, and only one, systematic source of variation not found in the denominator.

10. This allows us to conclude that: IF THE NUMERATOR IS SIGNIFICANTLY LARGER THAN THE DENOMINATOR, THE SIZE DIFFERENCE MUST BE MUST BE CAUSED BY THE ONE ADDITIONAL THING PUSHING THE MEANS APART, the IV. But notice there has to be only one thing different in the numerator and the denominator to make that conclusion inevitable.

11. Why we can’t use MS B as the numerator in the multifactor analysis of variance In the two factor analysis of variance, the means can be pushed apart by: –The effects of the first independent variable (F 1 ). –The effects of the second independent variable (F 2 ) –The effects of combining F 1 and F 2 that are above and beyond the effects of either variable considered alone (INT) –Sampling fluctuation (ID + MP)

12. So if we compared MS W to MS B in a two factor experiment, here is what we would have. F = (ID + MP + ?F 1 + ?F 2 + ?INT) (ID + MP) That’s not an F test. In an F test the numerator must have one and only one source of variation beyond sampling fluctuation. HERE THERE ARE THREE OF THEM!Each of these three things besides sampling fluctuation could be pushing the means apart. So, in the multifactor F test, a ratio between MS B and MS W is meaningless. We must create 3 numerators to compare to MS W, each comprising ID + MP + one other factor

13. What can we do???? We must take apart (analyze) the sums of squares and degrees of freedom betwee groups (SS B and df B ) into component parts. Each part must contain only one factor along with ID and MP. Then each component will yield an estimate of sigma 2 that can be compared to MS W in an F test

14. We start with SS B and df B and subdivide them – slide 1 First, we create a way to study the effects of the first factor, ignoring the presence of Factor 2. We combine groups so that the resulting, larger aggregates of participants differ only because they received different levels of F 1. Each such combined group will include an equal number of people who received the different levels of F 2. So the groups are the same in that regard. They differ only in terms of how they were treated on the first independent variable, F 1.

15. Each combined group has different people than the other combined group(s). So the the groups differ because of random individual differences (ID). Different random measurement problems will appear in each group (MP). Each combined group received a specific level of F 1 and each group has a mean. Thus the groups will differ from each other because of ID + MP + the possible effects of Factor 1.

16. Computing MS F1, one of the three numerators in a two factor F test So, if we find the differences between each person’s mean for his/her combined group and the overall mean, then square and sum the differences, we will have a sum of squares for the first independent variable (SS F1 ). Call the number of levels of an independent variable L. df for the combined group equals the number of levels of a factor minus one (L F1 – 1). An estimate of sigma 2 that includes only ID + MP + (?) F 1 can be computed by dividing this sum of squares by its degrees of freedom, as usual. MS F1 = SS F1 /df F1 = (ID + MP + ?F 1 )

17. Once you have MS F1, you have one of the three F tests you do in a two factor ANOVA F = MS F1 /MS W

18. Then you do the same thing to find MS F2 You combine groups so that you have groups that differ only on F 2. You compare each person’s mean for this combined group to the overall mean, squaring and summing the differences for each person to get SS F1. Degrees of freedom = the number of levels of Factor 2 minus 1 (df F2 = L F2 – 1). Then MS F2 = SS F2 /df F2 F F2 = MS F2 /df F2

19. The wholes are equal to the sum of their parts. Remember, SS B and df B are the total between group sum of squares and degrees of freedom. Each is composed of three parts, SS F1, SS F2, SS INT and df F1, df F2, and df INT. We are subdividing SS B and df B into their 3 component parts. We have already computed SS F1, SS F2, df F1, and df F2. If we subtract the parts of SS B that we have already accounted for (SS F1 & SS F2 ), the remainder will be THE SUM OF SQUARES FOR THE INTERACTION (SS INT ). If we subtract the parts of df B that we have already accounted for (df F1 & df F2 ), the remainder will be the degrees of freedom for the interaction (df INT ).

20. Here’s the formulae SS B = SS F1 + SS F2 + SS INT So SS INT = SS B – (SS F1 + SS F2 ) df B = df F1 + df F2 + df INT So df INT = df B – (df F1 + df F2 )

21. MS INT = SS INT /df INT Once you find the sum of squares and degrees for the interaction, you can compute a mean square, as usual, by dividing SS by df. The mean square for the interaction reflects ID + MP plus the possible effects of combining two variables that are not accounted for by the effects of either one alone or by simply adding the effects of the two.

22. For example, For example, alcohol and tranquilizers both can make you intoxicated. Combine them and you don’t just get more intoxicated. You can easily wind up dead. Their effects multiply each other, they don’t just add together. So the effect of the two variables together is different from the effect of either considered alone. That’s an interaction of two variables.

23. What’s next? We have two things to do: 1. Learn to compute the ANOVA and test three null hypotheses. 2. Learn more theory

24. Like Ch. 9, we’ll learn to compute ANOVA first. Many students find that it begins to make sense, when you see how the numbers come together. After the computation, we’ll return to theory.

25. A two factor experiment Introductory Psychology students are asked to perform an easy or difficult task after they have been exposed to a severely embarrassing, mildly embarrassing, non-embarrassing situation. The experimenter believes that people use whatever they can to feel good about themselves. Therefore, those who have been severely embarrassed will welcome the chance to work on a difficult task. The experimenter hypothesizes that in a non- embarrassing situation, participants will enjoy the easy task more than the difficult task.

26. Example: Experiment Outline Population: Introductory Psychology students Subjects: 24 participants divided equally among 6 treatment groups. Independent Variables: –Factor 1: Embarrassment levels: severe, mild, none. –Factor 2: Task difficulty levels: hard, easy Groups: 1=severe, hard task; 2=severe, easy task; 3=mild, hard task; 4=mild, easy task; 5=none, hard; 6=none, easy. Dependent variable: Subject rating of task enjoyment, where 1 = hating the task and 9 = loving it.

27. Effects We are interested in the main effects of embarrassment or task difficulty. Do participants like easy tasks better than hard ones? Do people like tasks differently when embarrassed or unembarrassed. We are also interested in assessing how combining different levels of both factors affect the response in ways beyond those that can be predicted by considering the effects of each IV separately. This is called the interaction of the independent variables.

28. Testing the null hypotheses. Do people undergoing different levels of embarrassment have differential responses to any task? H 0-F1 : Levels of embarrassment will not cause differences in liking for the task above and beyond those accounted for by sampling fluctuation. Do people like easy tasks better than hard ones (or the reverse) irrespective of how embarrassed they are? H 0-F2 : Differences in task difficulty will not cause differences in liking for the task above and beyond those accounted for by sampling fluctuation..

29. Testing the null hypotheses. Do embarrassment and task difficulty interact such that unembarrassed participants prefer easy tasks while embarrassed ones prefer hard tasks? H 0-INT : There will be no differences in liking for the task caused by combining task difficulty and embarrassment that can not be attributed to the effects of each independent variables considered alone or to sampling fluctuation.

30. Computational steps Outline the experiment. (Done) Define the null and experimental hypotheses. (Done) Compute the Mean Squares within groups. Compute the Sum of Squares between groups. Compute the main effects. Compute the interaction. Set up the ANOVA table. Check the F table for significance. Interpret the results.

31. A 3X2 STUDY Embarrassment Task Difficulty Severe Mild None Easy Hard

32. MS W Embarrassment Task Difficulty Severe Mild None Easy Hard Compare each score to the mean for its group.

33. Mean Squares Within Groups 1.1 1.2 1.3 1.4 2.1 2.2 2.3 2.4 3.1 3.2 3.3 3.4 4.1 4.2 4.3 4.4 5.1 5.2 5.3 5.4 6.1 6.2 6.3 6.4 648643453575648643453575 544733645676544733645676 044001014040044001014040 011411401010011411401010 666644445555666644445555 555544446666555544446666 0 -2 2 0 0 1 -2 0 2 0 2 2 0 0 1 0

34. Then we compute a sum of squares and df between groups This is the same as in Chapter 9 The difference is that we are going to subdivide SS B and df B into component parts. Thus, we don’t use SS B and df B in our Anova summary table, rather we use them in an intermediate calculation.

35. Sum of Squares Between Groups (SS B ) Embarrassment Task Difficulty Severe Mild None Easy Hard Compare each group mean to the overall mean.

36. 666644445555666644445555 555544446666555544446666 Sum of Squares Between Groups (SS B ) 555555555555555555555555 555555555555555555555555 111111110000111111110000 000011111111000011111111 1 0 1

37. Next, we answer the questions about each factor having an overall effect. To get proper between groups mean squares we have to divide the sums of squares and df between groups into components for factor 1, factor 2, and the interaction. Let’s just look at factor 1. Our question about factor 1 was “Do people undergoing different levels of embarrassment have differential responses to any task?” We can group participants into all those who were severely embarrassed, all those who were mildly embarrassed, and all those who were not embarrassed.

38. SS F1 : Main Effect of Embarrassment Embarrassment Task Difficulty Severe Mild None Easy Hard Compare each score’s Embarrassment mean to the overall mean.

39. Calculate Embarrassment Means 1.1 1.2 1.3 1.4 2.1 2.2 2.3 2.4 5.1 5.2 5.3 5.4 6.1 6.2 6.3 6.4 6486434564864345 3364567633645676 3.1 3.2 3.3 3.4 4.1 4.2 4.3 4.4 3575544735755447

40. Sum of squares and Mean Square for Embarrassment (F 1 ) Severe 1.1 1.2 1.3 1.4 2.1 2.2 2.3 2.4 Emb. 5 5555555555555555 0000000000000000 0000000000000000 No 5.1 5.2 5.3 5.4 6.1 6.2 6.3 6.4 Emb. 5 5555555555555555 0000000000000000 0000000000000000 Mild 3.1 3.2 3.3 3.4 4.1 4.2 4.3 4.4 Emb. 5 5555555555555555 0000000000000000 0000000000000000

41. Factor 2 Then proceed as you just did for Factor 1 and obtain SS F2 and MS F2 where df F2 =L F2 - 1.

42. SS F2 : Main Effect of Task Difficulty Embarrassment Task Difficulty Severe Mild None Easy Hard Compare each score’s difficulty mean to the overall mean.

43. Calculate Difficulty Means Hard 1.1 1.2 1.3 1.4 3.1 3.2 3.3 3.4 5.1 5.2 5.3 5.4 Easy 2.1 2.2 2.3 2.4 4.1 4.2 4.3 4.4 6.1 6.2 6.3 6.4 task 6 4 8 6 4 3 4 5 3 6 4 task 3 5 7 5 4 7 5 6 7 6

44. 555555555555555555555555 Sum of squares and Mean Square – Task Difficulty 555555555555555555555555 000000000000000000000000 000000000000000000000000 555555555555555555555555 555555555555555555555555 000000000000000000000000 000000000000000000000000

45. Computing the sum of squares and df for the interaction. SS B contains all the possible effects of the independent variables in addition to the random factors, ID and MP. Here is that statement in equation form SS B = SS F1 + SS F2 + SS INT Rearranging the terms: SS INT = SS B - (SS F1 +SS F2 ) or SS INT = SS B - SS F1 -SS F2 SS INT is what’s left from the sum of squares between groups (SS B ) when the main effects of the two IVs are accounted for. So, subtract SS F1 and SS F2 from overall SS B to obtain the sum of squares for the interaction (SS INT ). Then, subtract df F1 and df F2 from df B to obtain df INT ).

46. Means Squares - Interaction REARRANGE

47. Testing 3 null hypotheses in the two way factorial Anova

48. Hypotheses for Embarrassment Null Hypothesis - H 0 : There is no effect of embarrassment. Except for sampling fluctuation, the means for liking the task will be the same for the severe, mild, and no embarrassment treatment levels. Experimental Hypothesis - H 1 : Embarrassment considered alone will affect liking for the task.

49. Hypotheses for Task Difficulty Null Hypothesis - H 0 : There is no effect of task difficulty. The means for liking the task will be the same for the easy and difficult task treatment levels except for sampling fluctuation. Experimental Hypothesis - H 1 : Task difficulty considered alone will affect liking for the task.

50. Hypotheses for the Interaction of Embarrassment and Task Difficulty Null Hypothesis - H 0 : There is no interaction effect. Once you take into account the main effects of embarrassment and task difficulty, there will be no differences among the groups that can not be accounted for by sampling fluctuation. Experimental Hypothesis - H 1 : There are effects of combining task difficulty and embarrassment that can not be predicted from either IV considered alone. Such effects might be that: –Those who have been severely embarrassed will enjoy the difficult task more than the easy task. –Those who have not been embarrassed will enjoy the easy task more than the difficult task.

51. Theoretically relevant predictions In this experiment, the investigator predicted a pattern of results specifically consistent with her theory. The theory said that people will use any aspect of their environment that is available to avoid negative emotions and enhance positive ones. In this case, she predicted that the participants would like the hard task better when it allowed them to avoid focusing on feelings of embarrassment. Otherwise, they should like the easier task better.

52. Computational steps Outline the experiment. Define the null and experimental hypotheses. Compute the Mean Squares within groups. Compute the Sum of Squares between groups. Compute the main effects. Compute the interaction. Set up the ANOVA table. Check the F table for significance. Interpret the results.

53. Steps so far Outline the experiment. Define the null and experimental hypotheses. Compute the Mean Squares within groups. Compute the Sum of Squares between groups. Compute the main effects. Compute the interaction.

54. What we know to this point SS F1 =0.00, df F1 =2 SS F2 =0.00, df F2 =1 SS INT =16.00, df INT =2 SS W =32.00, df W =18

55. Steps remaining Set up the ANOVA table. Check the F table for significance. Interpret the results.

56. ANOVA summary table Embarrassment Task Difficulty 32 18 1.78 0 2 0 0 n.s SS df MS F p  Interaction Error 0 1 0 0 n.s 16 2 84.50.05

57. Means for Liking a Task Embarrassment Task Difficulty Severe Mild None Easy Hard 65 5 4 46 555 5 5

58. To interpret the results, always Plot the Means Severe Mild None Embarrassment Task Task Enjoy -ment Easy Hard

59. State Results Consistent with the experimenters theory, neither the main effect of embarrassment nor of task difficulty were significant. The interaction of the levels of embarrassment and of levels of the task difficulty was significant,

60. Interpret Significant Results Examination of the group means, reveals that subjects in the hard task condition most liked the task when severely embarrassed, and least liked it when not embarrassed at all. Those in the easy task condition liked it most when not embarrassed and least when severely embarrassed. Describe pattern of means.

61. Interpret Significant Results These findings are consistent with the hypothesis that people use everything they can, even adverse aspects of their environment, to feel as good as they can. Reconcile statistical findings with the hypotheses.

62. Back to theory Notice some things about the experiment

63. Design of the experiment Each possible combination of F 1 and F 2 creates an experimental group that is treated differently from all other groups, in terms of one or both factors. Participants are randomly assigned to each of the treatment groups. For example, if there are 2 levels of the first variable (Factor 1or F 1 ) and 2 of the second (F 2 ), we will need to create 4 groups (2x2). If F 1 has 2 levels and F 2 has 3 levels, we need to create 6 groups (2x3). If F 1 has 3 levels and F 2 has 3 levels, we need 9 groups. Etc.

64. Identifying experimental designs Two factor designs are identified by simply stating the number of levels of each variable. So a 2x4 design (called “a 2 by 4 design”) has 2 levels of F 1 and 4 levels of F 2. A 3x2 design has 3 levels of F 1 and 2 levels of F 2. Etc. Which factor is called F 1 and which is called F 2 is arbitrary (and up to the experimenter).

65. Example of 2 x 3 design To make it more concrete, assume we are testing new treatments for Generalized Anxiety Disorder. In a two factor design we examine the effects of cognitive behavior therapy vs. a social support group among GAD patients who receive Ativan, Zoloft or Placebo. Thus, F 1 has 2 levels (CBT/SocSup) while F 2 has 3 levels (Ativan/Zoloft/Placebo) So, we would form 2 x 3 = 6 groups to do this experiment. Half the patients would get CBT, the other half get social support. A third of the CBT patients and one-third of the Social Support patients also get Ativan. Another third of those who receive CBT and one-third of those who get social support also receive Zoloft. The final third in each psychotherapy condition get a pill placebo.

66. A 2 x 3 design yields 6 groups. Let’s say you have 48 participants. Eight are randomly assigned to each group. Here are the six treatment groups: –CBT + Ativan –CBT + Zoloft –CBT + Placebo –Social support + Ativan –Social support + Zoloft –Social support + Placebo

67. Remember, each group is randomly chosen from an (already) random sample. So, all the groups have similar means and variances on variables measuring any kind of potential difference. Thus, the group means on all variables will differ at the start of a two factor experiment solely because of random sampling fluctuation. The question is whether that will still be true after we treat the groups differently is a systematic, preplanned manner.

68. Again, on what measures will the groups tend to resemble each other at the beginning of the experiment ? Answer: On every conceivable and inconceivable measure. If we were to make a bridge of playing cards from here to Pluto and put one measure on each card, our cards could only constitute a small percentage of the measurements that could be made. And on each and every one we would expect the group means to be fairly close to each other and to the population mean.

69. Here’s an example of a 2 x 2 study We are looking at the effects of stress and communication on liking for others. Participants were placed for half an hour in a crowded or uncrowded environment. So, half the participants were stressed, half were not. (Crowding is a simple, nonharmful way to stress people), Half of each group then had communication encouraged, the other half had communication discouraged. Here is a diagram of the study.

70. Again, each combination of the two independent variables becomes a group, all of whose members get the same level of both factors. Stressed, Communication encouraged Not stressed, Communication discouraged Not stressed, Communication encouraged Stressed, Communication discouraged This is a 2X2 study. STRESS LEVEL COMMUNICATION

71. REVIEW

72. Analysis of Variance In our statistical analysis, we will determine whether differences among the treatment groups’ means are consistent with the null hypothesis that they differ solely because of random sampling fluctuation. As we did in Chapter 9, we will compare two estimates of sigma 2, One estimate will be based on the variation of group means of around the overall mean. The other estimate is MS W, derived from the variation of scores around their own group means.

73. As you know… Sigma 2 is estimated either by comparing a score to a mean (the within group estimate) or by comparing a mean to another mean. This is done by –Calculating a deviation or –Squaring the deviations. –Summing the deviations. –Dividing by degrees of freedom

74. H 0 vs. H 1 Again, the null will predict that a ratio of the two estimates of sigma 2 will be about 1.00. Again, the experimental hypothesis says the ratio will be greater than one. But we can’t just compare MS B to MS w as we did in Chapter 9. We have a problem that needs to be solved before any estimates of sigma 2 can be compared.

75. The Problem Unlike the one-way ANOVA of Chapter 9, we now have two variables that may push the means of the experimental groups apart. Moreover, combining the two variables may have effects beyond those that would occur were each variable presented alone. We call such effects the interaction of the two variables. Such effects can be multiplicative as opposed to additive. Example: Moderate levels of drinking can make you high. Barbiturates can make you sleep. Combining them can make you dead. The effect (on breathing in this case) is multiplicative.

76. The difference between analysing single and multifactor designs – a review If we simply did the same thing as in Ch. 9, our between group term could be effected by Factor 1, Factor 2, and/or their interaction as well as by sampling fluctuation. An F test requires two estimates of sigma 2, one that indexes only sampling fluctuation, the other indexes sampling fluctuation plus one other variable or interaction of variables. So we are going to have to do something different to get appropriate between groups terms/

77. Next, we create new between groups mean squares by combining the original experimental groups. To get proper between groups mean squares we have to divide the sums of squares and df between groups into components for factor 1, factor 2, and the interaction. We calculate sums of squares and df for the main effects of factors 1 and 2 first. We obtain the sum of squares and df for the interaction by subtraction.

78. To analyze the data we will again estimate the population variance (sigma 2 ) with mean squares and compute F tests The denominator of the F ratio will be the mean square within groups (MS W ) where MS W =SS W / n-k. (AGAIN!) In the multifactorial analysis of variance, the problem is obtaining proper mean squares for the numerator. We will study the two way analysis of variance for independent groups.

79. Computing SS for Factor 1 Pretend that the experiment was a simple, single factor experiment in which the only difference among the groups was the first factor (that is, the degree to which a group is embarrassed). Create groups reflecting only differences on Factor 1. So, when computing the main effect of Factor 1 (level of embarrassment), ignore Factor 2 (whether the task was hard or easy). Divide participants into three groups depending solely on whether they not embarrassed, mildly embarrassed, or severely embarrassed. Next, find the deviation of the mean of the severely, mildly, and not embarrassed participants from the overall mean. Then sum and square those differences. Total of the summed and squared deviations of each person’s group’s score from the overall mean in the severely, mildly, and not embarrassed conditions is the sum of squares for Factor 1. (SS F1 ).

80. df F1 and MS F1 Compute a mean square that takes only differences on Factor 1 into account by dividing SS F1 by df F1. df F1 = L – 1 where L equals the number of levels (or different variations) of the first factor (F 1 ). For example, in this experiment, embarrassment was either absent, mild or severe. These three ways participants are treated are called the three “levels” of Factor 1.

81. Then Do the same for factor 2 Subtract the sums of squares and df for factors 1 & 2 from the sum of squares and degrees of freedom between group to obtain SS INT & df INT. Once you have all the sums of squares and degrees of freedom, compute ANOVA and determine significance with the F table. Finally, plot the means and carefully interpret the results.