1 Analysis of Variance (ANOVA) Educational Technology 690

2 Introduction Scenario Number of years of experience in baseball and coping skill  Type of Study?  Participants: 3 groups (6, 7, 10 years)  Instrument: Athletic Coping Skills Inventory (peaking under pressure subscale)

3 Analysis T test? NO! More than 2 means. Analysis of Variance (ANOVA) Compare if the means are statistically different F test F=13.08, P<.01 Conclusion?  Differences are due to years of experience rather than chance occurrence of scores

4 More On ANOVA ANOVA tests the difference between the means of more than two groups Simple ANOVA: on one treatment factor (IV) or dimension  Years of experience Factorial ANOVA: on more than one treatment factor or dimension (IV)  Years of experience; pitch type

5 Examples Simple ANOVA Years of experience ( 3) Coping skills of professional baseball players Factorial ANOVA Grouping factors (IVS): years of experience/pitch type 3 (levels) X 2 (levels ) factorial design computes a separate F ratio for each independent variable and the interaction between the variables  F for years of experience;  F for pitch type;  F for experience*pitch interaction

6 Example: 3 X 2 Factorial Design Professional Baseball Players Experiment Group 1Group 2 Group 3Group 4 CurveballFastball Group 5Group 6 > 7 Years Years < 3 Years Experience Level Pitch Type

7 Example: 2 X 2X2 Factorial Design operating canal lock Text, Interaction, Sound Effects Narration, No Interaction, No Sound Effects Text, No Interaction, No Sound Effects Interactivity/NonSound/VisualText/Narration Narration, Interaction, Sound Effects Narration, No Interaction, Sound Effects Text, No Interaction, Sound Effects Narration, Interaction, No Sound Effects Text, Interaction, No Sound Effects Factorial ANOVA

8 Why called ANOVA How much of the variability in the DV can be explained by the IV IV: nominal (categories; limited no. of values) DV: continuous (numerical) Sports example coping skills (variability) Group 1 Group 2 Group 3 10 (2 years) 10 (6 years) 10 (8 years) S1…S10 S1…S10 S1….S10

9 Understanding the F Formula F= Msbetween/Mswithin MS: mean sum of squares Mswithin: residual mean Sums of Square MS=Sum of Squares/df Df (between-group): k (group)-1 Df (within-group): n (total no. of sample)-k Sports example: Msbetween groups (treatment) F(variability of coping skills)= Mswithin groups (chance)

10 Activities: What Type of ANOVA? There are no difference between people’s typing accuracy and their hours of training (2, 4, and 6 hours) Treatment (Grouping) factor: levels of training Test variable: typing accuracy There are differences between male and female’s typing accuracy and hours of training (2, 4, and 6 hours) Grouping factor: hours of training; gender Test V: typing accuracy

11 Group Activity: What Type of ANOVA (p. 238 of Salkind) 1. Using the table, provide 3 examples of a simple ANOVA, 1 two-factor ANOVA, and 1 three-factor ANOVA. Sharing of examples 2. Hypothesis testingHypothesis testing

12 Computing the F Value 1. Statement of null or non-null hypotheses No directions (more than two means) 2. Setting the level of significance (a) or Type I error associated with the hypothesis Recalling type I error?  Null-H is true, but we reject it. a: An estimate of the probability that we are wrong when we reject the null hypothesis (NH)  Usually use a= 0.05, when p<0.05, rejecting NH  5% of being wrong in rejecting NH

13 Computing the F Value 3. Selecting test statistics Using the chart from Salkind book 4. Computation of F value Degree of freedom  An approximation of the sample or group size  DF for between-group (numerator): K-1 (K: no. of groups)  DF for within group (denominator): n-k (N: no. of sample size--participants)  Example: 3 groups, 30 subjects—F (DFnumerator, DFdemonitor) F (2, 27)

14 Interpreting F Value 5a. Computing by hand  See formula on Gay: a. Determining the value needed for rejection of null hypothesis F (2, 27) obtained value = 8.80 Critical value on F test table at 0.05 = 3.36 The F value needed for rejecting null hypothesis 7a. Comparison of obtained value and critical value 8a. Decision:  Rejecting the null hypothesis?

15 Interpreting F Value 5b. Computing by software Car data There is no difference between weight of the car and the country it is made. Analyze->ANOVA 6b. Determining the value needed for rejection of null hypothesis F (2, 113), a = 5%, p= P needs to < 0.05 (mean difference has a less than 5% probability due to chance) 7b. Decision

16 Factorial Analysis of Variance two or more independent variables are considered as well as the interaction between them computes a separate F ratio for each independent variable and the interaction between the variables

17 Factorial Analysis of Variance If a new interface to your computerized training program helps employees learn the material quicker than the old interface. You are also interested if it affects both high and low IQ employees the same 2 levels X 2 levels Dependent variable? Amount of knowledge gained Independent variables? Type of Interface and IQ Three separate F ratios F of type of interface F of IQ F of interface*IQ

18 Exploring StatView ANOVA To discover if there is an overall difference between the means ANOVA post hoc (or multiple comparison) test If ANOVA shows an overall mean difference Run Post Hoc to discover where the difference lie. Each mean is compared to each other (remembering t test? Judge by P value) Fisher’s PLSD; Schdffe’s F; Bonferroni

19 Group Activities Choosing one set of data in StatView Generating a hypothesis Testing your hypothesis using ANOVA and Post Hoc test Decision and interpretation Class Sharing of results

20 Example 1 shows that the new interface worked well for both those with low and high IQ, and that high IQ employees did better than low IQ employees with both interfaces. A factorial analysis of variance indicated a statistically significant difference in the interface and IQ, but not on an interaction.

21 Example 2 shows that the new interface worked best for those with high IQ, and that high IQ employees did better than low IQ employees with both interfaces. A factorial analysis of variance indicated a statistically significant difference in IQ, but not in interface. It also showed a significant difference in the interaction of IQ and method.

22 What does F Value Mean? F= Msbetween (treatment)/Mswithin (chance) F=1? Difference between groups not significant F >1 Group difference increases->numerator increases->F increase the larger the F, the larger the difference due to treatment