1. ANOVA: PART II

2. Last week  Introduced to a new test:  One-Way ANOVA  ANOVA’s are used to minimize family-wise error:  If the ANOVA is statistically significant:  One or more groups are significantly different in your test  Follow-up testing required to determine which groups  If the ANOVA is not statistically significant:  All groups are statistically similar (can’t be confident enough that any differences aren’t simply random sampling error)  No follow-up testing is needed, all will also be non-significant These ‘rules’ are true for all ANOVA’s

3. Tonight…  We’ll finish testing for ‘group differences’ with:  Factorial ANOVA Any ANOVA including more than one IV Introduce interactions  ‘Hint at’:  ANCOVA ANOVA’s including a co-variate  Repeated Measures ANOVA ANOVA’s for dependent, paired, or related groups

4. Something to think about…  One-Way ANOVA’s ‘look’ very similarly to t-tests  They compare individual groups, one by one  Compare means between the levels of one variable  Factorial ANOVA’s, ANCOVA’s, and RM ANOVA’s are actually more closely related to multiple linear regression  All of these tests use the General Linear Model Y = mX + b  We will be comparing groups, but we’ll be doing it using the General Linear Model behind the scenes  This should make more sense later in the lecture…

5. Factorial ANOVA example  Example dataset contains patients that were diagnosed with breast cancer  Key Variables: Survival Time in Months – Survival time after diagnosis/treatment (our dependent variable) Tumor Size – Subjects grouped by initial tumor size ( 5 cm, our independent variable)  We want to know if Tumor Size effects the cancer Survival Time

6. Picture of our analysis  If we ran a One-way ANOVA, it would look something like this:  Will compare mean survival time between each of the three groups and determine if they are significantly different Tumor Size < 2 cm Mean Survival 1 2-5 cm Mean Survival 2 > 5 cm Mean Survival 3

7. However…  This is a good analytical plan – but in reality more variables influence cancer survival than just tumor size  We would like to examine several different IV’s  For example, lymph nodes are essentially a means for cancer to travel around the body. If the cancer spreads, survival time decreases  Lymph node involvement may also influence survival time  Instead of 1 – we could run 2 one-way ANOVA’s…

8. New Plan  ANOVA #1:  ANOVA #2: Tumor Size < 2 cm Mean Survival 1 2-5 cm Mean Survival 2 > 5 cm Mean Survival 3 Lymph Node Involvement? No Mean Survival 1 Yes Mean Survival 2 This will tell us if Tumor Size impacts survival This will tell us Lymph Node involvement impacts survival

9. But…  Now our two tests examine how two independent variables influence our dependent variable but…  …what if Tumor Size and Lymph Node involvement are related to each other? Remember collinearity? This is a similar concept Called Interaction in ANOVA (more on this later) By running two separate tests we are ignoring this potential interaction of the variables

10. Factorial ANOVA’s  Running a factorial ANOVA allows us to run one ANOVA that:  1) Uses two (or more) independent variables  2) Examines the potential for interaction between our IV’s  Again, notice here that Factorial ANOVA’s are very similar to multiple regression  We’re using two or more independent variables and one dependent variable  The only difference is the Factorial ANOVA tests for group differences while the Regression predicts

11. Terminology  In Factorial ANOVA’s the independent variables are called ‘main effects’  Tumor Size is a main effect  Lymph Node involvement is a main effect  This is just the lingo, think of it as ‘main’ variable  Factorial ANOVA’s are usually described by how many levels each main effect has  How many levels does our Tumor Size variable have?  How many levels does our Lymph Node variable have?  We have a 3 x 2 Factorial ANOVA

12. New Analysis: 6 Groups  Instead of two tests – everything is tested at once  Let’s run this model in SPSS… Tumor Size < 2 cm2-5 cm> 5cm Lymph Node Involvement? No Mean Survival 1 Mean Survival 2 Mean Survival 3 Yes Mean Survival 4 Mean Survival 5 Mean Survival 6 Main Effect #1 Main Effect #2 Two-Way ANOVA Or 3 X 2 Factorial ANOVA

13.  Factorial ANOVA use the General Linear Model tab  The “Univariate” description refers to the number of dependent variables you have – not independent  Again you can see a clear separation of One-Way ANOVA’s (that are like t-tests) and all the other ANOVA’s (that are like multiple regression). SPSS puts them under different options.

14.  Dependent Variable is Survival Time  Fixed Factors = IV’s  Tumor Size  Lymph Node  Post-Hoc Tests same as before  Options similar (can get means, SD, for each group, etc…)

15.  For Post-Hoc options now I can select all of my independent variables (not just 1)

16.  Use the “Plots” option every time you run a Factorial ANOVA – we’ll discuss why later  It really doesn’t matter which IV you enter first, but I tend to “Separate Lines” by the IV with the fewest levels, in this case Lymph Nodes

17. Estimated Marginal Means  Factorial ANOVA’s have another new addition under the ‘Options’ button  The Post-Hoc tab will use Pairwise Comparisions to compare the levels of our independent variables (Tumor Size and Lymph Node) and look for differences  However, the Estimated Marginal Means are required when you want to make comparisons using BOTH of your two main effects at the same time  Lets use our picture…

18. What Post-Hoc Tests do… Tumor Size < 2 cm Mean Survival 1 2-5 cm Mean Survival 2 > 5 cm Mean Survival 3  Post-Hoc tests will compare Tumor Size between ONLY these three groups (ignoring the effect of Lymph Node Involvement)  And the post-hoc test will compare Lymph Nodes, ignoring Tumor Size Lymph Node Involvement? No Mean Survival 1 Yes Mean Survival 2 That is NOT the only thing we are interested in!

19. Estimated Marginal Means vs Post-Hoc  In other words:  Post-Hoc tests: Test for group difference one independent variable at a time Adequate for One-Way ANOVA’s Like a t-test  Estimated Marginal Means: Test for group differences while considering all variables in your model More appropriate for Factorial ANOVA’s

20. How do they Work?  This option will ‘adjust’ the group means of Tumor Size for Lymph Node Involvement  If everyone had the same amount of lymph node involvement – what effect would Tumor Size have?  These are ‘estimated’ means because they do NOT exist in real life – SPSS is saying, “What if…?” Tumor Size < 2 cm2-5 cm> 5cm Lymph Node Involvement? No Mean Survival 1 Mean Survival 2 Mean Survival 3 Yes Mean Survival 4 Mean Survival 5 Mean Survival 6 X1X1 X2X2 X3X3

21. Estimated Marginal Means  These are ‘estimated’ means because they do NOT exist in real life – SPSS is saying, “What if…?”  Has SPSS lost it’s mind?  How can we ‘adjust’ for another variable?  How can we ‘estimate’ how an independent variable will effect the dependent variable?  SPSS is using the General Linear Model to ‘predict’

22. The General Linear Model in ANOVA  What SPSS is really doing:  Survival Time = TumorSizeX 1 + LymphX 2 + b  Estimated Marginal Means function by assuming a constant value for one of the X’s – in this case it is Lymph Node Involvement:  Survival Time1 = (<2cm)X 1 + (Constant)X 2 + b  Survival Time 2 = (2-5cm)X 1 + (Constant)X 2 + b  Survival Time 3 = (>5cm)X 1 + (Constant)X 2 + b  The constant for the IV is similar to the mean lymph node involvement for the sample

23. Estimated Marginal Means  This option will ‘adjust’ the group means of Tumor Size for Lymph Node Involvement  If everyone had the same amount of lymph node involvement – what effect would Tumor Size have? Tumor Size < 2 cm2-5 cm> 5cm Lymph Node Involvement? No Mean Survival 1 Mean Survival 2 Mean Survival 3 Yes Mean Survival 4 Mean Survival 5 Mean Survival 6 X1X1 X2X2 X3X3 Estimated Survival Time for everyone with a Tumor Size < 2cm (adjusting for Lymph Node Involvement) Initial Questions on Estimated Marginal Means?

24.  Move over all factors into the box on the right  Display Estimated Means for…  Check the box that says “Compare Main Effects”  This will generate p-values for the groups – adjusted for the other variables

25. Factorial Outputs  Once you run the test the Output file can be overwhelming since there is so much to look at.  Move through one output box at a time  For this 1 analysis, I requested:  Descriptive statistics  A Means plot  Estimated Marginal Means for Tumor Size, Lymph Node Involvement, and our Interaction  Post-Hoc tests and pair-wise comparisons  …And the actual ANOVA

26. Ready for the results…?

27. Sample Sizes  You want to try and have all groups be equal in size, but it’s almost impossible to do!  How many subjects in our study? 826 + 274 + 42= 1142= 871+ 271

28. Descriptives for all 6 groups Main Effect #1: Tumor Size Main Effect #2 Lymph Node?

29. ANOVA Results  Take note of df, F, and Significance for each Main Effect (we have 2) and each Interaction (we have 1)  Also, you will report the Error (or ‘overall’) df = 1136

30.  Initial interpretation:  Survival Time is significantly impacted by Tumor Size p < 0.001  Survival Time is not significantly impacted by Lymph Node Involvement p = 0.207  Survival Time is significantly impacted by the interaction of Tumor Size and Lymph Node Involvement p = 0.016 Next Step? Estimated Marginal Means to determine WHERE the differences are!

31. EMM for Tumor Size  Estimated Marginal Means/Pairwise Comparisons for Tumor Size (adjusting for Lymph Node)  Looks just like a Post-Hoc test – except it’s based on the Estimated Marginal Means!

32. EMM for Tumor Size  Interpretation:  Those with Tumors 5 cm group (p = 0.006). There was not a significant difference between the 2-5 cm and > 5 cm group (p = 0.254).

33. ?  Do I need to look at the Estimated Marginal Means for Lymph Node involvement?  Was the Main Effect of Lymph Node statistically significant?  NO! But you can verify that for yourself:

34. Interactions  Aside from the Main Effects, we did have a statistically significant interaction we need to consider  Interactions:  A combination of Main Effects. Indicates that two IV’s are related to each other Similar to: collinearity, confounders, etc…  If you have a significant interaction:  At least one of your Main Effects is influenced by another main effect

35. Interactions  Think of it this way…  When considered alone: As tumor size increased survival time decreased Lymph node involvement had no effect on survival time  An statistically significant interaction means… The effect that tumor size has on survival time is MODIFIED by lymph node involvement Interactions also known as ‘effect modification’ The two variables interact to cause a different effect on survival The effect of tumor size is dependent on lymph node involvement!

36. Interactions  When you have a statistically significant interaction, you have to ignore any statistically significant main effects  Why? Because the interaction ‘overrides’ them  So, in writing you would report the significant main effect, but realize the interaction is MORE important  This is why we requested the means plot earlier. A plot of the means (including both the interaction terms) makes it easier to see…

37.  If there is no interaction, these lines would be parallel  The effect of increasing tumor size would be the same for both lines You can see clearly that Lymph Node does have an effect – but only for those with tumors larger than 2 cm No difference Big difference

38. Interactions  In general, if the lines on your plot are NOT parallel, you probably have a significant interaction  Without effect modification/interactions, you would expect the variables to have the same impact no matter what their combination  Examples…

39. Example: Interaction

40. Example: NO Interaction

41. Example: Interaction

42. The interaction is significant…and we can see it on the plot…what now…?

43. Significant Interactions  In truth, it’s much easier if you do NOT find significant interactions  Report only the statistically significant main effects Use the Post-Hoc tests or Estimated Marginal Means to see which groups are different  Ignore the non-significant main effects  Ignore the non-significant interaction  When you do find a significant interaction, you must now treat each group separately (in our case, consider all 6 groups)

44. Estimated Marginal Means Interaction Table  When you request it, you will get a pairwise comparison table for the interaction using the estimated marginal means  You have to use the estimated marginal means here because we have ‘combined’ two variables  This table will tell you which groups are significantly different – but you have to use the 95% confidence intervals!

45.  When 95% confidence intervals for 2 groups do NOT overlap, they are significantly different (p < 0.05)  Which groups are statistically different here?  Compare the two groups WITHIN each category of Tumor Size Estimated Marginal Means Interaction

46. Reporting Your Findings  A 3 x 2 Factorial ANOVA revealed a statistically significant main effect for Tumor Size (F(2, 1136) = 7.719, p 5cm group, lymph node involvement further decreased survival time*.  *You could list the means/SD for each group and reveal the results of the pairwise comparisons. But most of the time this information is place in a table (with 95% CI’s).

47. Checklist for Factorial ANOVA’s  Look at group descriptives and sample size  Examine the ANOVA output box (Tests of Between Subjects Effects) to see df, F ratio’s, and p-values for the Main Effects and Interactions(s)  If you have significant Main Effects Use Estimated Marginal Means + Pair-wise Comparisons to see which groups are different (adjusting for the other variables in your model)  If you have a significant Interaction: Ignore the Main effects! The interaction is what is REALLY going on! Use the means Plot to see how your Main Effects work together Use the estimated marginal means interaction table and the 95% confidence intervals to see which groups are different Questions on the Checklist?

48. Factorial ANOVA’s  Our example here includes a 3-level variable and a 2-level variable.  As you add levels – or variables – these things will really grow quickly  Example: Besides tumor size and lymph node involvement, progesterone receptor status (2-level; positive or negative) might also be relevant to cancer survival. I’ll add it to our ANOVA:  DV = Survival Time  IV = Tumor Size, Lymph Node Involvement, Progesterone Receptor Status  What kind of Factorial ANOVA is this now? 3 x 2 x 2 ANOVA

49.  Note my NEW main effects and interactionS  What would I need to do now?

50. QUESTIONS ON FACTORIAL ANOVA’S?

51. Upcoming…  In-class activity  Homework:  Cronk complete 6.6  Cronk READ 6.7 – you are not required to DO this section, just check it out  Holcomb Exercises 51 and 52 (on Factorial ANOVA)  We’ll change things up next week by talking about 2 statistical concepts (rather than ‘new’ tests)  Power and Effect Size  ANCOVA