Chapter 12 A Priori and Post Hoc Comparisons Multiple t-tests Linear Contrasts Orthogonal Contrasts Trend Analysis Bonferroni t Fisher Least Significance Difference Studentized Range Statistic Dunnett’s Test

Fisher Least Significance Difference Simple 1) Do a normal omnibus ANOVA 2) If there it is significant you know that there is a difference somewhere! 3) Do individual t-test to determine where significance is located

Fisher Least Significance Difference Problem You may have an ANOVA that is not significant and still have results that occur in a manner that you predict! If you used this method you would not have “permission” to look for these effects.

Remember

Remember

Studentized Range Statistic Which groups would you likely select to determine if they are different?

Studentized Range Statistic Which groups would you likely select to determine if they are different? This statistics controls for Type I error if (after looking at the data) you select the two means that are most different.

Studentized Range Statistic Easy! 1) Do a normal t-test

Studentized Range Statistic Easy! 2) Convert the t to a q

Studentized Range Statistic 3) Critical value of q (note: this is a two-tailed test) Figure out df (same as t) Example = 20 Figure out r r = the number of groups

Studentized Range Statistic 3) Critical value of q (note: this is a two-tailed test) Figure out df (same as t) Example = 20 Figure out r r = the number of groups Example = 4

Studentized Range Statistic 3) Critical value of q Page 744 Example q critical = +/- 3.96

Studentized Range Statistic 4) Compare q obs and q critical same way as t values q = -5.61 q critical = +/– 3.96

Practice You collect axon firing rate scores from rates in one of four conditions. Condition 1 = 10 mm of Zeta inhibitor Condition 2 = 20 mm of Zeta inhibitor Condition 3 = 30 mm of Zeta inhibitor Condition 4 = 40 mm of Zeta inhibitor Condition 5 = 50 mm of Zeta inhibitor You are simply interested in determining if any two groups are different from each other – use the Studentized Range Statistic

Studentized Range Statistic Easy! 1) Do a normal t-test

Studentized Range Statistic Easy! 2) Convert the t to a q

Studentized Range Statistic 3) Critical value of qnote: this is a two-tailed test) Figure out df (same as t) Example = 20 Figure out r r = the number of groups Example = 5

Studentized Range Statistic 3) Critical value of q Page 744 Example q critical = +/- 4.23

Studentized Range Statistic 4) Compare q obs and q critical same way as t values q = -4.34 q critical = +/– 4.23

Dunnett’s Test Used when there are several experimental groups and one control group (or one reference group) Example: Effect of psychotherapy on happiness Group 1) Psychoanalytic Group 2) Humanistic Group 3) Behaviorism Group 4) Control (no therapy)

Psyana vs. Control Human vs. Control Behavior vs. Control

Psyana vs. Control = 47.8 – 51.4 = -3.6 Human vs. Control = 50.8 – 51. 4 = -0.6 Behavior vs. Control = 59 – 51.4 = 7.6

Psyana vs. Control = 47.8 – 51.4 = -3.6 Human vs. Control = 50.8 – 51. 4 = -0.6 Behavior vs. Control = 59 – 51.4 = 7.6 How different do these means need to be in order to reach significance?

Dunnett’s t is on page 753 df = Within groups df / k = number of groups

Dunnett’s t is on page 753 df = 16 / k = 4

Dunnett’s t is on page 753 df = 16 / k = 4

Psyana vs. Control = 47.8 – 51.4 = -3.6 Human vs. Control = 50.8 – 51. 4 = -0.6 Behavior vs. Control = 59 – 51.4 = 7.6* How different do these means need to be in order to reach significance?

Practice As a graduate student you wonder what undergraduate students (freshman, sophomore, etc.) have different levels of happiness then you.

Dunnett’s t is on page 753 df = 25 / k = 5

Fresh vs. Grad = -17.5* Soph vs. Grad = -21.5* Jun vs. Grad = -31.5* Senior vs. Grad = -8.5

What if. . . You were asked to determine if psychology and sociology majors have significantly different class attendance (i.e., the number of days a person misses class) You would simply do a two-sample t-test two-tailed Easy!

But, what if. . . You were asked to determine if psychology, sociology, and biology majors have significantly different class attendance You would do a one-way ANOVA

But, what if. . . You were asked to determine if psychology majors had significantly different class attendance than sociology and biology majors. You would do an ANOVA with contrast codes

But, what if. . . You were asked to determine the effects of both college major (psychology, sociology, and biology) and gender (male and female) on class attendance You now have 2 IVs and 1 DV You could do two separate analyses Problem: “Throw away” information that could explain some of the “error” Problem: Will not be able to determine if there is an interaction

Factorial Analysis of Variance Factor = IV Factorial design is when every level of every factor is paired with every level of every other factor Psychology Sociology Biology Male X Female

Factorial Analysis of Variance Currently Different people in each cell Equal n in each cell Psychology Sociology Biology Male X Female

Factorial Analysis of Variance 2 X 3 Factorial “2” is because one IV has 2 levels (male and female) “3” because one IV has 3 levels (psychology, biology, sociology) Psychology Sociology Biology Male X Female

Factorial Analysis of Variance 2 X 3 4 X 5 2 X 2 X 7

Notation One factor is A and the other is B Psychology Sociology Biology Male Female

Notation One factor is A and the other is B B B1 B2 B3 A1 A2 A

Notation One factor is A and the other is B Any combination of A and B is called a cell B B1 B2 B3 A1 A2 A

Notation One factor is A and the other is B Any combination of A and B is called a cell The number of subjects in each cell = n B B1 B2 B3 A1 A2 A

Notation One factor is A and the other is B Any combination of A and B is called a cell The number of subjects in each cell = n Each subject in a cell = Xij i = level of A; j = level of B B B1 B2 B3 A1 A2 A

Notation One factor is A and the other is B Any combination of A and B is called a cell The number of subjects in each cell = n Each subject in a cell = Xij i = level of A; j = level of B B B1 B2 B3 A1 A2 A

Notation One factor is A and the other is B Any combination of A and B is called a cell The number of subjects in each cell = n Each subject in a cell = Xijk i = level of A; j = level of B; k = observation in that cell B B1 B2 B3 A1 X111,X112 X113 X121,X122 X123 X131,X132 X133 A2 X211,X212 X213 X221,X222 X223 X231,X232 X233 A

Notation n = 3; N = nab = 18 The means for each of the cells are: B1 X111,X112 X113 X121,X122 X123 X131,X132 X133 A2 X211,X212 X213 X221,X222 X223 X231,X232 X233 A

Notation n = 3; N = nab = 18 The means for each of the cells are: B1 X11 X12 X13 A2 X21 X22 X23 A

Notation n = 3; N = nab = 18 The means for row (level of A): B1 B2 B3 X11 X12 X13 X1. A2 X21 X22 X23 X2.

Notation n = 3; N = nab = 18 The means for column (level of B): B1 B2 X11 X12 X13 X1. A2 X21 X22 X23 X2. X.1 X.2 X.3

Notation n = 3; N = nab = 18 The grand mean: B1 B2 B3 A1 X11 X12 X13

Sociology Psychology Biology Female 2.00 1.00 3.00 .00 Males 4.00 n = 3 N = 18

Sociology Psychology Biology Female 2.00 1.00 3.00 .00 Males 4.00 n = 3 N = 18

Sociology Psychology Biology Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 Males 4.00 Mean2j 3.67 0.33

Sociology Psychology Biology Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33

Sociology Psychology Biology Mean Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 1.78 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33 2.33

Sociology Psychology Biology Mean Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 1.78 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33 2.33 2.06

Sociology Psychology Biology Mean Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 1.78 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33 2.33 2.06 Main effect of gender

Sociology Psychology Biology Mean Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 1.78 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33 2.33 2.06 Main effect of major

Sociology Psychology Biology Mean Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 1.78 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33 2.33 2.06 Interaction between gender and major

Sum of Squares SS Total Computed the same way as before The total deviation in the observed scores Computed the same way as before

Sociology Psychology Biology Mean Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 1.78 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33 2.33 2.06 SStotal = (2-2.06)2+ (3-2.06)2+ . . . . (1-2.06)2 = 30.94 *What makes this value get larger?

Sociology Psychology Biology Mean Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 1.78 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33 2.33 2.06 SStotal = (2-2.06)2+ (3-2.06)2+ . . . . (1-2.06)2 = 30.94 *What makes this value get larger? *The variability of the scores!

Sum of Squares SS A Represents the SS deviations of the treatment means around the grand mean Its multiplied by nb to give an estimate of the population variance (Central limit theorem) Same formula as SSbetween in the one-way

Sociology Psychology Biology Mean Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 1.78 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33 2.33 2.06 SSA = (3*3) ((1.78-2.06)2+ (2.33-2.06)2)=1.36 *Note: it is multiplied by nb because that is the number of scores each mean is based on

Sociology Psychology Biology Mean Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 1.78 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33 2.33 2.06 SSA = (3*3) ((1.78-2.06)2+ (2.33-2.06)2)=1.36 *What makes these means differ? *Error and the effect of A

Sum of Squares SS B Represents the SS deviations of the treatment means around the grand mean Its multiplied by na to give an estimate of the population variance (Central limit theorem) Same formula as SSbetween in the one-way

Sociology Psychology Biology Mean Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 1.78 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33 2.33 2.06 SSB = (3*2) ((3.17-2.06)2+ (2.00-2.06)2+ (1.00-2.06)2)= 14.16 *Note: it is multiplied by na because that is the number of scores each mean is based on

Sociology Psychology Biology Mean Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 1.78 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33 2.33 2.06 SSB = (3*2) ((3.17-2.06)2+ (2.00-2.06)2+ (1.00-2.06)2)= 14.16 *What makes these means differ? *Error and the effect of B

Sum of Squares SS Cells Represents the SS deviations of the cell means around the grand mean Its multiplied by n to give an estimate of the population variance (Central limit theorem)

Sociology Psychology Biology Mean Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 1.78 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33 2.33 2.06 SSCells = (3) ((2.67-2.06)2+ (1.00-2.06)2+. . . + (0.33-2.06)2)= 24.35

Sociology Psychology Biology Mean Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 1.78 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33 2.33 2.06 SSCells = (3) ((2.67-2.06)2+ (1.00-2.06)2+. . . + (0.33-2.06)2)= 24.35 What makes the cell means differ?

Sum of Squares SS Cells What makes the cell means differ? 1) error 2) the effect of A (gender) 3) the effect of B (major) 4) an interaction between A and B

Sum of Squares Have a measure of how much cells differ SScells Have a measure of how much this difference is due to A SSA Have a measure of how much this difference is due to B SSB What is left in SScells must be due to error and the interaction between A and B

Sum of Squares SSAB = SScells - SSA – SSB 8.83 = 24.35 - 14.16 - 1.36

Sum of Squares SSWithin SSWithin = SSTotal – (SSA + SSB + SSAB) The total deviation in the scores not caused by 1) the main effect of A 2) the main effect of B 3) the interaction of A and B SSWithin = SSTotal – (SSA + SSB + SSAB) 6.59 = 30.94 – (14.16 +1.36 + 8.83)

Sum of Squares SSWithin

Sociology Psychology Biology Mean Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 1.78 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33 2.33 2.06 SSWithin = ((2-2.67)2+(3-2.67)2+(3-2.67)2) + . .. + ((1-.33)2 + (0-.33)2 + (0-2..33)2 = 6.667

Sociology Psychology Biology Mean Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 1.78 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33 2.33 2.06 SSWithin = ((2-2.67)2+(3-2.67)2+(3-2.67)2) + . .. + ((1-.33)2 + (0-.33)2 + (0-2..33)2 = 6.667 *What makes these values differ from the cell means? *Error

Compute df Source df SS A 1.36 B 14.16 AB 8.83 Within 6.59 Total 30.94

Source df SS A 1.36 B 14.16 AB 8.83 Within 6.59 Total 17 30.94 dftotal = N - 1

Source df SS A 1 1.36 B 2 14.16 AB 8.83 Within 6.59 Total 17 30.94 dftotal = N – 1 dfA = a – 1 dfB = b - 1

Source df SS A 1 1.36 B 2 14.16 AB 8.83 Within 6.59 Total 17 30.94 dftotal = N – 1 dfA = a – 1 dfB = b – 1 dfAB = dfa * dfb

Source df SS A 1 1.36 B 2 14.16 AB 8.83 Within 12 6.59 Total 17 30.94 dftotal = N – 1 dfA = a – 1 dfB = b – 1 dfAB = dfa * dfb dfwithin= ab(n – 1)

Compute MS Source df SS A 1 1.36 B 2 14.16 AB 8.83 Within 12 6.59 Total 17 30.94

Compute MS Source df SS MS A 1 1.36 B 2 14.16 7.08 AB 8.83 4.42 Within 12 6.59 .55 Total 17 30.94

Compute F Source df SS MS A 1 1.36 B 2 14.16 7.08 AB 8.83 4.42 Within 12 6.59 .55 Total 17 30.94

Test each F value for significance Source df SS MS F A 1 1.36 2.47 B 2 14.16 7.08 12.87 AB 8.83 4.42 8.03 Within 12 6.59 .55 Total 17 30.94 F critical values (may be different for each F test) Use df for that factor and the df within.

Test each F value for significance Source df SS MS F A 1 1.36 2.47 B 2 14.16 7.08 12.87 AB 8.83 4.42 8.03 Within 12 6.59 .55 Total 17 30.94 F critical A (1, 12) = 4.75 F critical B (2, 12) = 3.89 F critical AB (2, 12) = 3.89

Test each F value for significance Source df SS MS F A 1 1.36 2.47 B 2 14.16 7.08 12.87* AB 8.83 4.42 8.03* Within 12 6.59 .55 Total 17 30.94 F critical A (1, 12) = 4.75 F critical B (2, 12) = 3.89 F critical AB (2, 12) = 3.89

In SPSS

In SPSS Main Effects Easy – just like a one-way ANOVA

Sociology Psychology Biology Mean Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 1.78 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33 2.33 2.06

In SPSS Interaction Does the effect of one IV on the DV depend on the level of another IV?

Sociology Psychology Biology Mean Female 2.00 1.00 3.00 .00 Mean1j 2.67 1.67 1.78 Males 4.00 Mean2j Mean.j 3.67 3.17 0.33 2.33 2.06 Want to plot out the cell means

Sociology Psychology Biology

Practice 2 x 2 Factorial Determine if 1) there is a main effect of A 2) there is a main effect of B 3) if there is an interaction between AB

Practice A: NO B: NO AB: NO

Practice A: YES B: NO AB: NO

Practice A: NO B: YES AB: NO

Practice A: YES B: YES AB: NO

Practice A: YES B: YES AB: YES

Practice A: YES B: NO AB: YES

Practice A: NO B: YES AB: YES

Practice A: NO B: NO AB: YES

Practice These are sample data from Diener et. al (1999). Participants were asked their marital status and how often they engaged in religious behavior. They also indicated how happy they were on a scale of 1 to 10. Examine the data

Frequency of religious behavior Never Occasionally Often Married 6 3 7 2 8 4 5 9 Unmarried 1