Smith/Davis (c) 2005 Prentice Hall Chapter Fifteen Inferential Tests of Significance III: Analyzing and Interpreting Experiments with Multiple Independent Variables PowerPoint Presentation created by Dr. Susan R. Burns Morningside College

Smith/Davis (c) 2005 Prentice Hall Before Your Statistical Analysis While you are carefully planning your experiment, you also will need to choose the experimental design that best enables you to ask your questions of interest. It is important to not collect data for which you have no design or test with which to analyze it.

Smith/Davis (c) 2005 Prentice Hall Analyzing Factorial Designs We analyze factorial designs with the same type of statistical test that we used for analyzing the multiple- group designs (ANOVA). Labels you may hear that refer to the size of the design include: – Factorial ANVOA as a general term – Two-way ANOVA for two IVs – Three-way ANOVA for three IVs Also, researchers indicate the size of the design as X by Y, where X and Y represent the number of levels of the two factors.

Smith/Davis (c) 2005 Prentice Hall Analyzing Factorial Designs Other labels that are used to describe factorial designs that use random assignment for all IVs include: – Independent groups – Completely randomized – Completely between-subjects – Completely between-groups – Totally between-subjects – Totally between-groups Designs that use matching or repeated measures may be called: – Randomized block – Completely within-subjects – Completely within-groups – Totally within-subjects – Totally within-groups. Designs that use a combination of the two are referred to as mixed or split-plot factorial.

Smith/Davis (c) 2005 Prentice Hall Planning Your Experiment When designing an experiment, there are many possible ways that you can expand to include additional IVs. The authors recommend that you keep things simple.

Smith/Davis (c) 2005 Prentice Hall Rational of Factorial ANOVA Similar to the one-way ANOVA, we still use ANOVA to partition (divide) the variability into two scores – treatment variability and error variability. However, with factorial designs, the sources of treatment variability increase. That is, instead of having one IV as the sole source of treatment variability, factorial designs have multiple IVs and interactions as sources of treatment variability.

Smith/Davis (c) 2005 Prentice Hall Rationale of Factorial ANOVA The equations used to separately evaluate the effects of each of the two IVs as well as their interaction are as follows: If you use a larger factorial design, you would end up with an F ratio for each of the IVs and each interaction

Smith/Davis (c) 2005 Prentice Hall Understanding Interactions A significant interaction means that the effects of the various IVs are not straightforward and simple. Thus, we basically ignore the main effects of our independent IVs when there is a significant interaction. A good way to understand interactions is to graph them. – By graphing your DV on the y axis and one IV on the x axis, you can depict your other IV lines on the graph. When you have a significant interaction, you will notice that the lines of the graph cross or converge. – This pattern is a visual indication that the effects of one IV change as the second IV is varied. Non-significant interactions typically show lines that are close to parallel.

Smith/Davis (c) 2005 Prentice Hall Two-Way ANOVA for Independent Groups In two-way ANOVAs, the variability is broken down into three sources: the first IV, the second IV, and the interaction between the two IVs. The variability associated with each effect is referred to as the sum of squares.

Smith/Davis (c) 2005 Prentice Hall Two-Way ANOVA for Independent Groups Sums of Squares for Two-Way ANVOA for Independent Groups – Total Sum of Squares is similar to the formula used to calculate standard deviations: SS tot = ΣX 2 – (ΣX) 2 /N – Main Effect Sums of Squares are analogous to the treatment sum of squares (must be computed for each IV): SS IV = Σ[ΣX 2 /n] – (ΣX) 2 /N

Smith/Davis (c) 2005 Prentice Hall Example Calculations

Smith/Davis (c) 2005 Prentice Hall Two-Way ANOVA for Independent Groups Sums of Squares for Two-Way ANVOA for Independent Groups – Interaction Sum of Squares are more time consuming to compute because you have to work with four means for each of the four cells. You must work out a formula for each of the four cells: begin with the cell mean, subtract the two overall means for its column and row, and then add the overall mean. In each case, you square the total, sum the cells, and the multiply by the number of participants per cell. – Error Sum of Squares represents all the variability from sources other than the IVs and their interaction (e.g., individual difference, errors in measurement, and extraneous variation). To find the error sum of squares, use the formula that you used for the total sum of squares and apply it to each of the four cells. To check your computational accuracy, the mean squares for the two IVs, their interaction, and the error term should, when added together, equal the total sum of squares.

Smith/Davis (c) 2005 Prentice Hall Two-Way ANOVA for Independent Groups Mean Squares for each source of variation are calculated by dividing the SS for each source by its appropriate degrees of freedom. – Total df = N – 1 – df IV = number of groups (levels of the IV) minus 1 – df int = the df for the two IVs multiplied together – df error = the total N – the number of groups for each IV multiplied together

Smith/Davis (c) 2005 Prentice Hall Two-Way ANOVA for Independent Groups F ratios are computed by dividing the mean square for each source of variation by the error mean square. You will have Fs for each of the IVs and the interaction. Again, you will need to use the F table using the appropriate degrees of freedom for each F to determine significance. When you have a significant interaction, you have to refer to both IVs to understand the findings. Again, the best way to understand an interaction is to graph it.

Smith/Davis (c) 2005 Prentice Hall

Graphing Interactions Did we find a significant interaction? That is, was the pattern of clerks’ responses to customers dressed in different clothing the same for customers of both sexes?

Smith/Davis (c) 2005 Prentice Hall Post Hoc Comparison Post hoc Comparisons may not be necessary for a variety of reasons: – If your interaction is significant, it supersedes the main effects. If the interaction renders the main effect moot, then there would be no need to use the post hoc tests. – You might not need to use post hoc because your experimental design may make them unnecessary. For example, with a 2 x 2 design, if you have a significant main effect, without a significant interaction, you only have two levels to compare. – If your multiple-level IV is not significant, then you do not need to use post hoc comparisons. – The only situation when you would need post hoc analysis in a two-way factorial is when you have no significant interaction, but a significant main effect for an IV with three or more levels.

Smith/Davis (c) 2005 Prentice Hall Source Table A Source Table again should be used to summarize your results. A source table for a two-way ANOVA will have the following sources: – Variability from each of the two IVs – The interaction – The error variability

Smith/Davis (c) 2005 Prentice Hall Effect Size Eta squared (η 2 ) is still an appropriate measure of the proportion of variance accounted for by an effect. When you have a significant interaction, you only need to report the η 2 for the interaction.

Smith/Davis (c) 2005 Prentice Hall Computer Analysis of Two-Way ANOVA It would be unusual for a researcher to calculate a two-way ANOVA by hand. The printout for the two- way ANOVA will be similar to printout seen with the one-way ANOVA.

Smith/Davis (c) 2005 Prentice Hall Two-Way ANOVA for Correlated Groups Two-Way ANOVA for Correlated Groups requires that you have two IVs with correlated groups for both IVs. Total Sum of Squares remain the same as the total sum of squares for two-way ANOVA for independent groups. The same is true for the main effects and interaction sum of squares. Participant Sum Squares are calculated by summing across the participants rather than across the groups as you do to calculate the variability due to the main effects and interaction.

Smith/Davis (c) 2005 Prentice Hall Two-Way ANOVA for Correlated Groups Other than this difference, the formula is the same as the one used for finding the sum of squares for either main effect. Error sum of squares can be calculated two ways using subtraction. – SS Tot - SS A – SS B – SS int – SS part = SS error – Or you can use the approach of calculating the error sum of squares as you did for the independent samples analysis. However, you still need to remove the participant sum of squares from that figure.

Smith/Davis (c) 2005 Prentice Hall

Two-Way ANOVA for Correlated Groups Mean Squares are still calculated by dividing the SS for each source of variance by the appropriate degrees of freedom. Degrees of freedom for the total sum of squares, the main effects, and the interaction are calculated using the same equation as before. There is a change, however, for the error term by taking out the df for the participants effect.

Smith/Davis (c) 2005 Prentice Hall Two-Way ANOVA for Correlated Groups F ratios are calculated by dividing the MS for each effect by the MSerror term. – Again, you would use the appropriate table and degrees of freedom to determine significance. Post hoc tests are not necessary for this analysis (as was the case for the two-way ANOVA for independent groups). – Both the significant interaction and the fact that our IVs have only two levels tells us that we need no further analyses.

Smith/Davis (c) 2005 Prentice Hall Two-Way ANOVA for Correlated Groups The Source Table for this analysis is similar to that of the independent-groups analysis, with one important difference. – Because we’ve measured variability due to participants, we will need to include that factor in our source table. If it was not included, our total variability would not add up. Effect size can be calculated again using η 2, calculated in the same manner as with the independent groups ANOVA.

Smith/Davis (c) 2005 Prentice Hall Computer Analysis of Two-Way ANOVA for Correlated Groups Again, it would be unusual for researchers to calculate this analysis by hand. The computerize results you see for this should be similar to the results that you would obtain from the independent-groups ANOVA.

Smith/Davis (c) 2005 Prentice Hall Two-Way ANOVA for Mixed Groups Two-Way ANOVA for Mixed Groups requires that we have two IVs with independent groups for one IV and correlated groups for the second IV. Sum of Squares calculation has not changed from the earlier analyses. – However, in a mixed design, we divide some of the error variability into between-subjects variability, and some into within-subjects variability.

Smith/Davis (c) 2005 Prentice Hall Calculation of the Two-Way ANOVA for Mixed Groups Error Sum of Squares is calculated by having two error terms; one for the between-subjects variable, one for the within-subjects variable. – The method of calculating the error term for the within variable and the interaction entails working with the four different cell totals, each of which represents one of the combinations of the two variables. – When the two error terms are added equal the total error sum of squares that was found using the independent- samples analysis.

Smith/Davis (c) 2005 Prentice Hall Calculation of the Two-Way ANOVA for Mixed Groups Mean Squares are calculated again using the appropriate sum of squares and degrees of freedom. – Because we have two error terms, we need to find the degrees of freedom for each of them. – The df for the between-subjects variable equals the number of groups minus 1 multiplied by the number of participants per group minus 1. – The df for the within-subjects effects (the within-subjects IV and the interaction) equals the number of groups for the between-subjects variable multiplied by the number of groups for the within-subjects variable minus 1. – In a 2 x 2 design, the error terms’ df will be equal.

Smith/Davis (c) 2005 Prentice Hall Calculation of the Two-Way ANOVA for Mixed Groups F ratios are calculated by dividing the appropriate MS by the appropriate error term. – Once again, you need to use the appropriate table to determine whether the results are significant. – Remember having a significant interaction renders the main effects meaningless. Post hoc tests again are not necessary for a 2 x 2 design. The Source Table differs from those before because of the division of the error terms. – The source table is now divided into two parts, with the between- subjects effect toward the top of the table and the within-subjects effects near the bottom of the table. Again, η 2 is calculated as a measure of effect size. – Remember, if the interaction is significant, you only need to calculate the effect size for it.

Smith/Davis (c) 2005 Prentice Hall

Calculation of the Two-Way ANOVA for Mixed Groups

Smith/Davis (c) 2005 Prentice Hall Computer Analysis of Two-Way ANOVA for Mixed Groups. The mixed-group computer printout will be similar to that seen for the other two-way ANOVAs. However, like the source table, the between and within- subjects variables are separated. This division is because of the different error terms for each of these variables.

Smith/Davis (c) 2005 Prentice Hall The Continuing Research Problem The authors of your text note that these designs are still fairly simple, and at this point, you may consider pursuing a line of programmatic research. That is, pursuing, a line of research.