1. Chapter 9 - Lecture 2 Some more theory and alternative problem formats. (These are problem formats more likely to appear on exams. Most of your time in lecture will be spent on solving similar problems.)

2. Review of one-way, unrelated groups F and t tests

3. Simple Experiments Simple experiments have only a single independent variable with multiple levels. In simple experiments, participants are chosen independently. So participants in one group are unrelated to those in other groups.

4. To calculate the F test, zWe are going to look at two different ways of calculating mean squares to estimate the population variance and then compare the mean squares. zOne way is based on the difference between each score and its group mean. This estimate of sigma 2 is called MS W and it is familiar. zThe other way is based on the difference between the group and overall means: called MS B, it is new.

5. Mean square within groups Since everyone in a group is treated the same way, differences between scores and their own group mean can only reflect random individual differences and random measurement problems. This is the mean square within groups (MS W ) and it is always a good estimate of sigma 2, the population variance. MS W can index only individual differences and measurement problems (ID + MP).

6. Mean square between groups Groups are treated differently in a deliberate, planned way. These different treatments are the different levels of the independent variable. Therefore, differences between each group’s mean and the overall mean can reflect the effects of the independent variable (as well as the effects of random individual differences and random measurement problems). Thus MS B = ID + MP + (?)IV This is the mean square between groups (MS B ). If the independent variable pushes the group means apart, MS B will overestimate sigma 2 and be larger than MS W.

7. Testing the Null Hypothesis (H 0 ) H 0 says that the groups differ from each other and from the overall mean only because of random individual differences and measurement problems. These are the same things that make scores differ from their own group means. So, according to H 0, MS B and MS W are two ways of measuring the same thing (ID + MP). Two measurements of the same thing should be about equal to each other and a ratio between them should be about equal to 1.00. We could establish a 95% confidence interval around 1.00 for each pair of degrees of freedom. The F table does it for us, showing us the value of F just outside the 95% confidence interval

8. The Experimental Hypothesis (H 1 ) The experimental hypothesis says that the groups’ means will be made different from each other (pushed apart) by the IV, the independent variable (as well as by random individual differences and measurement problems). If the means are pushed apart, MS B will increase, reflecting the effects of the independent variable (as well as of the random factors). MS W will stay the same. Sot, if the IV has an effect, MS B will be larger than MS W Therefore, H 1 suggests that a ratio comparing MS B to MS W should be larger than 1.00.

9. Ratio of mean squares = F ratio Mean Squares within groups. Mean Squares between groups. Possibly effected by independent variable. Not effected by independent variable. If the independent variable causes differences between the group means, then MS B will be larger than MS W. If the effect is large enough and/or there are enough degrees of freedom, the result may be a statistically significant F ratio.

10. The F Table F table tells us whether the F ratio is significant. The null hypothesis predicts an F of about 1.00. We establish a 95% confidence interval around 1.00. We only pay attention to the high end of the distribution. Any F ratio smaller than the critical value at p<.05 is in the confidence interval around 1.00. F ratios smaller than the critical value are therefore consistent with the null hypothesis (no differences that can’t be explained as simply sampling fluctuation)

11. Statistical Significance p<.05 means that we have found an F ratio that occurs with 5 or fewer samples in 100 when the null is true.The null predicts that we will find an F ratio close to 1.00, not an unusually large F ratio. If we find a larger F ratio than the null predicts, we have shown H 0 to predict badly and reject it. Results are statistically significant when you equal or exceed the critical value of F at p<.05. If your F ratio equals or exceeds the critical value at p<.01, you get to brag.

12. 3 10.13 9.55 9.28 9.12 9.01 8.94 8.88 8.84 34.12 30.82 29.46 28.71 28.24 27.91 27.67 27.49 Degrees of freedom in Numerator 1 2 3 4 5 6 7 8 Df in denominator 4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 21.20 18.00 16.69 15.98 15.52 15.21 14.98 14.80 5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 16.26 13.27 12.06 11.39 10.97 10.67 10.45 10.27 6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 13.74 10.92 9.78 9.15 8.75 8.47 8.26 8.10 7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 12.25 9.55 8.45 7.85 7.46 7.19 7.00 6.84 8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 11.26 8.65 7.59 7.01 6.63 6.37 6.19 6.03 9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 10.56 8.02 6.99 6.42 6.06 5.80 5.62 5.47 These are related to the number of different treatment groups. They relate to the Mean Square between groups. k-1

13. 3 10.13 9.55 9.28 9.12 9.01 8.94 8.88 8.84 34.12 30.82 29.46 28.71 28.24 27.91 27.67 27.49 Degrees of freedom in Numerator 1 2 3 4 5 6 7 8 Df in denominator 4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 21.20 18.00 16.69 15.98 15.52 15.21 14.98 14.80 5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 16.26 13.27 12.06 11.39 10.97 10.67 10.45 10.27 6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 13.74 10.92 9.78 9.15 8.75 8.47 8.26 8.10 7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 12.25 9.55 8.45 7.85 7.46 7.19 7.00 6.84 8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 11.26 8.65 7.59 7.01 6.63 6.37 6.19 6.03 9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 10.56 8.02 6.99 6.42 6.06 5.80 5.62 5.47 These are related to the number of subjects. They relate to the Mean Square within groups. n-k

14. 3 10.13 9.55 9.28 9.12 9.01 8.94 8.88 8.84 34.12 30.82 29.46 28.71 28.24 27.91 27.67 27.49 Degrees of freedom in Numerator 1 2 3 4 5 6 7 8 Df in denominator 4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 21.20 18.00 16.69 15.98 15.52 15.21 14.98 14.80 5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 16.26 13.27 12.06 11.39 10.97 10.67 10.45 10.27 6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 13.74 10.92 9.78 9.15 8.75 8.47 8.26 8.10 7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 12.25 9.55 8.45 7.85 7.46 7.19 7.00 6.84 8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 11.26 8.65 7.59 7.01 6.63 6.37 6.19 6.03 9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 10.56 8.02 6.99 6.42 6.06 5.80 5.62 5.47 The critical values in the top rows are alpha =.05.

15. 3 10.13 9.55 9.28 9.12 9.01 8.94 8.88 8.84 34.12 30.82 29.46 28.71 28.24 27.91 27.67 27.49 Degrees of freedom in Numerator 1 2 3 4 5 6 7 8 Df in denominator 4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 21.20 18.00 16.69 15.98 15.52 15.21 14.98 14.80 5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 16.26 13.27 12.06 11.39 10.97 10.67 10.45 10.27 6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 13.74 10.92 9.78 9.15 8.75 8.47 8.26 8.10 7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 12.25 9.55 8.45 7.85 7.46 7.19 7.00 6.84 8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 11.26 8.65 7.59 7.01 6.63 6.37 6.19 6.03 9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 10.56 8.02 6.99 6.42 6.06 5.80 5.62 5.47 The critical values in the bottom rows are for bragging rights (p<.01).

16. An experiment Population: Depressed patients in a psychiatric hospital Number of participants: 9 Number of groups: 3 Design: Single factor, unrelated groups. Independent variable: Type of treatment –Level 1: Medication –Level 2: Psychotherapy –Level 3: ECT Dependent variable: HAM-D scores. (Lower = better.) H 0 : Treatments do not differ in effectiveness. H 1 : Treatments differ in effectiveness.

17. First Alternative Format Assume that you were given the following set of calculations on which to base a choice among treatments. (That is, instead of asking you to compute an F test from scratch, you might be given the information on the next slide.) Set up an Anova summary table and find the results.

18. MS W and MS B : Compute Anova 1.1 1.2 1.3 2.1 2.2 2.3 3.1 3.2 3.3 5 10 15 19 12 8 11 15 22 999000999999000999 -3 0 3 10 13 16 13 25 0 25 36 1 25 1 36 -5 0 5 6 -5 6 10 13 16

19. ANOVA summary table Between Groups Stress level Within Groups Error 54 2 27.00 174 6 29.00 0.93 SS df MS F p  ? Do we need to look at the F table to determine significance? Your answer should be “no.” Why? Divide MS B by MS W to calculate F.

20. 3 10.13 9.55 9.28 9.12 9.01 8.94 8.88 8.84 34.12 30.82 29.46 28.71 28.24 27.91 27.67 27.49 Degrees of freedom in Numerator 1 2 3 4 5 6 7 8 Df in denominator 4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 21.20 18.00 16.69 15.98 15.52 15.21 14.98 14.80 5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 16.26 13.27 12.06 11.39 10.97 10.67 10.45 10.27 6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 13.74 10.92 9.78 9.15 8.75 8.47 8.26 8.10 7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 12.25 9.55 8.45 7.85 7.46 7.19 7.00 6.84 8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 11.26 8.65 7.59 7.01 6.63 6.37 6.19 6.03 9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 10.56 8.02 6.99 6.42 6.06 5.80 5.62 5.47 5.14 10.92 H 0 says F should be about 1.00. 0.93 is less than 1.00, never mind the critical value at the.05 alpha level (5.14). So it is not statistically significant. F (2,6)=0.93, n.s.

21. 3 10.13 9.55 9.28 9.12 9.01 8.94 8.88 8.84 34.12 30.82 29.46 28.71 28.24 27.91 27.67 27.49 Degrees of freedom in Numerator 1 2 3 4 5 6 7 8 Df in denominator 4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 21.20 18.00 16.69 15.98 15.52 15.21 14.98 14.80 5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 16.26 13.27 12.06 11.39 10.97 10.67 10.45 10.27 6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 13.74 10.92 9.78 9.15 8.75 8.47 8.26 8.10 7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 12.25 9.55 8.45 7.85 7.46 7.19 7.00 6.84 8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 11.26 8.65 7.59 7.01 6.63 6.37 6.19 6.03 9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 10.56 8.02 6.99 6.42 6.06 5.80 5.62 5.47 Notice the usual effect of increased df The more df B and df W,... the better our estimates of sigma 2,... the closer F should be to 1.00 when the null is true. H 0 says that F should be about 1.00.

22.    Degrees of freedom in Numerator 1 2 3 4 5 6 7 8 Df in denominator 36 4.41 3.26 2.86 2.63 2.48 2.36 2.28 2.21 7.39 5.25 4.38 3.89 3.58 3.35 3.18 3.04 40 4.08 3.23 2.84 2.61 2.45 2.34 2.26 2.19 7.08 4.98 4.13 3.65 3.34 3.12 2.95 2.82 60 4.00 3.15 2.76 2.52 2.37 2.25 2.17 2.10 7.08 4.98 4.13 3.65 3.34 3.12 2.95 2.82 NOTE: Most critical values of F at.05 are between 2.5 and 6.0 Most critical values of F at.01 are between 4 and 14 400 3.86 3.02 2.62 2.39 2.23 2.12 2.03 1.96 6.70 4.66 3.83 3.36 3.06 2.85 2.69 2.55  3.84 2.99 2.60 2.37 2.21 2.09 2.01 1.94 6.64 4.60 3.78 3.32 3.02 2.80 2.64 2.51

23. The t test

24. The t Test: t for 2, F for More The t test is a special case of the F ratio. If there are only two levels (groups) of the independent variable, then

25. t table and F table when there are only two groups: df in the numerator is always 1

26. Relationship between t and F tables Because there is always 1 df between groups, the t table is organized only by degrees of freedom within group (df W ). By the way, the values in the t table are the square root of the values in the first column of the F table.

27. df 12345678.05 12.7064.3033.1822.7762.5712.4472.3652.306.01 63.6579.9255.8414.6044.0323.7073.4993.355 df 910111213141516.05 2.2622.2282.2012.1792.1602.1452.1312.120.01 3.2503.1693.1063.0553.0122.9972.9472.921 df 1718192021222324.05 2.1102.1012.0932.0862.0802.0742.0692.064.01 2.8982.8782.8612.8452.8312.8192.8072.797 df 2526272829304060.05 2.0602.0562.0522.0482.0452.0422.0212.000.01 2.7872.7792.7712.7632.7562.7502.7042.660 df 1002005001000200010000.05 1.9841.9721.9651.9621.9611.960.01 2.6262.6012.5862.5812.5782.576 t table from Chapter 6 Let’s look at these values.

28. df 3 4 5.05 3.1822.7762.571.01 5.8414.6044.032 t table from Chapter 6 F table. 3 10.13 34.12 4 7.71 21.20 5 6.61 16.26 1

29. More Conceptual Format – Final Exam? A researcher compared responses to four doses (placebo, low dose, moderate dose, high dose) of Thinkright, a new drug for schizophrenia. There were 6 participants in each group. She measures responses on a scale of normal cognition. Higher scores equal more normal functioning. The sum of squares within group is 200.00. The sum of squares between groups was 100.00.

30. ANOVA summary table We must fill in the blanks for SS & df. Between Groups Dose level Within Groups Error 100.00 ??? 200.00 ??? SS df MS F p 

31. Finding df B & df W. There are four groups. k=4 Six participants/group. 6 x 4=24 n=24. df B = k – 1 = 3 df W =n-k=24-4=20

32. ANOVA summary table Between Groups Dose level Within Groups Error 100.00 3 200.00 20 SS df MS F p 

33. Once you have these four (SS B, df B, SS W, df W ) all the rest is simple division and using the F table.

34. MS B =SS B /df B =100.00/3=33.33 MS W =SS W /df W =200.00/20=10.00 F=MS B /MS W =33.33/10.00=3.33

35. ANOVA summary table Between Groups Dose level Within Groups Error 100 3 33.33 200 20 10.00 3.33 SS df MS F p  ? We need to look at the F table to determine significance. Divide MS B by MS W to calculate F.

36. Critical value of F with 3, 20 df =3.10 for p<.05 F(3,20)=3.33, p<.05

37. ANOVA summary table Between Groups Stress level Within Groups Error 100 3 33.33 200 20 10.00 3.33 SS df MS F p .05

38. Toughest Format-Final Exam? A researcher compared three types of psychotherapy for Generalized Anxiety Disorder: Interpersonal Therapy (IPT), Cognitive Behavior Therapy (CBT) and Mindfulness Therapy (MT). Results were measured with an index of relaxation. Higher scores equaled greater relaxation. The five participants in the IPT groups averaged 23.00 The five participants in the CBT group averaged 31.00. The seven participants in the MT group averaged 27.00. The overall mean was 27.00. The sum of squares within group was 280.00.

39. ANOVA summary table We must fill in the blanks for SS & df. Between Groups Therapy type Within Groups Error ??? 280.00 ??? SS df MS F p 

40. Finding df B & df W. There are three groups. k=3 5 participants in two groups 7 in the remaining one. 5+5+7=17 n=17. df B = k – 1 = 2 df W =n-k=17-3=14

41. ANOVA summary table Still missing SS B. Between Groups Therapy type Within Groups Error ??? 2 280.00 14 SS df MS F p 

42. Finding SS B one group at a time. First group: Distance of group mean from overall mean for the 5 participants in this group ( X b ar -M) is (23.00-27.00) = -4.00. Each of the five participants in this group contributes that distance squared to SS B (–4.00 x –4.00 = 16.00) Total contribution of this group to SS B = 5x16.00=80.00

43. Finding SS B one group at a time. Second group: Distance of group mean from overall mean for the 7 participants in this group ( X bar -M) is (27.00-27.00) = 0.00. Each of the seven participants in this group contributes that distance squared to SS B (0.00 x 0.00 = 0.00) Total contribution of this group to SS B = 7x0.00=0.00

44. Finding SS B one group at a time. Third group and total SS B. Distance of group mean from overall mean for the 5 participants in this group (X bar -M) is (31. 00-27.00) = -4.00. Each of the five participants in this group contributes that distance squared to SS B (4.00 x 4.00 = 16.00) Total contribution of this group to SS B = 5x16.00=80.00 SS B =80.00 + 0.00 + 80.00 = 160.00

45. ANOVA summary table. Between Groups Therapy type Within Groups Error 160.00 2 280.00 14 SS df MS F p 

46. Once you have these four (SS B, df B, SS W, df W ) all the rest is simple division and using the F table.

47. Computing Mean Squares & F MS B =SS B /df B =160.00/2=80.00 MS W =SS W /df W =280.00/14=20.00 F=MS B /MS W =80.00/20.00=4.00

48. ANOVA summary table Between Groups Therapy type Within Groups Error 160 2 80.00 280 14 20.00 4.00 SS df MS F p  ? We need to look at the F table to determine significance. Divide MS B by MS W to calculate F.

49. Critical value of F with 2,14 df =3.74 for p<.05 F(2,14)=4.00, p<.05

50. ANOVA summary table Between Groups Therapy type Within Groups Error 160 2 80.00 280 14 20.00 4.00 SS df MS F p .05

51. The only hard part was computing SS B. Remember how we did it. We found how much each individual contributed to SS B by finding his or her group’s mean difference from the overall mean (X-M) and then squared it (X-M) 2. Then, to find each group’s contribution to SS B, we multiplied (X-M) 2 by the number of participants in the group. We summed the contributions of all the groups to obtain SS B.

52. One more thing to always remember

53. Correlation vs Experimetation You must remember the basic difference between the two designs and what that means about drawing casual inferences (saying that changing one thing causes another to change in response).

54. Pre-existing differences among participants always provide alternative explanations in correlational studies. Correlational research is based on the comparison of pre-existing differences. SCIENTIFICALLY, IT IS ALWAYS POSSIBLE THAT ONE OF THE MYRIAD PRE-EXISTING DIFFERENCES (OR COMBINATION OF DIFFERENCES) that you are not studying IS THE REASON(S) UNDERLYING A CORRELATION BETWEEN the 2 VARIABLES YOU ARE STUDYING.

55. Therefore: WE CAN NOT ELIMINATE THE POSSIBILITY THAT OTHER, UNMEASURED DIFFERENCES AMONG PARTICIPANTS ARE CAUSING THE RELATIONSHIP YOU FOUND BETWEEN THE VARIABLES IN A CORRELATIONAL STUDY. THEREFORE, YOU CAN’T KNOW THAT ONE OF THE TWO VARIABLES YOU HAPPENED TO STUDY IS THE FACTOR CAUSING CHANGES IN THE OTHER - NO MATTER HOW PLAUSIBLE AN EXPLANATION IT SEEMS. THEREFORE, YOU CAN’T SAY HOW TO EFFECT CHANGE AT ALL BASED ON CORRELATIONAL RESEARCH

56. At the start of an experiment Participants are randomly selected from the population and randomly assigned to different experimental groups. Since the groups are randomly selected, we assume that each is representative of the population. That is, in each case the sample mean should be close to the population mean. Same for the variance. So the means and variances of all the samples should be similar on all pre-existing differences. While the people remain different from each other, within and between groups, THERE ARE NO PRE-EXISTING DIFFERENCES AMONG THE GROUPS: THE GROUPS ARE THE SAME, NOT DIFFERENT

57. Therefore: If, at the end of the experiment groups that were previously the same wind up more different than they should be if only sampling fluctuation is at work, we can say we caused the change by the different ways we treated the groups. We can then generalize that ability to change Y by changing X to the population represented by the random sample that participated in the experiment.