1. Education 793 Class Notes Presentation 10 Chi-Square Tests and One-Way ANOVA

2. 2 Review: Crosstabs VIEW8612 MARRIED WOMEN BEST IN HOME by SEX86 STUDENT'S SEX SEX86 Page 1 of 1 Count | Row Pct |MALE FEMALE Col Pct | Row | 1 | 2 | Total VIEW8612 --------+--------+--------+ 1 | 762 | 1590 | 2352 DISAGREE STRONG | 32.4 | 67.6 | 55.1 | 36.4 | 73.1 | +--------+--------+ 2 | 889 | 343 | 1232 DISAGREE SOME | 72.2 | 27.8 | 28.9 | 42.5 | 15.8 | +--------+--------+ 3 | 343 | 170 | 513 AGREE SOME | 66.9 | 33.1 | 12.0 | 16.4 | 7.8 | +--------+--------+ 4 | 97 | 72 | 169 AGREE STRONG | 57.4 | 42.6 | 4.0 | 4.6 | 3.3 | +--------+--------+ Column 2091 2175 4266 Total 49.0 51.0 100.0 How do we assess whether the relationship we observe in the table is statistically significant? We need a sampling distribution

3. 3 Chi-Square The Chi-Square statistic is used to assess whether or not the observed frequencies in the table are significantly different from the expected frequencies. It is used in the form of counts (not percents). Observations must be mutually exclusive, one individual can only fall into one cell

4. 4 Chi-Square Chi square is a nonparametric test. It does not require the sample data to be more or less normally distributed (as parametric tests like z and t-tests do), although it relies on the assumption that the variable is normally distributed in the population from which the sample is drawn. But chi square, while forgiving, does have some requirements: –The sample must be randomly drawn from the population –Data must be reported in raw frequencies (not percentages) –Values/categories on independent and dependent variables must be mutually exclusive and exhaustive –Observed frequencies cannot be too small

5. 5 Three Types of Tests Goodness of Fit: Often called a One- Way  2 test. Used for data in one row and =>2 columns Two-way test: Used for data in =>2 rows and =>2 columns –Test of Independence (One population) –Test of Homogeneity (Two or more populations)

6. 6 Lucky US!! All three tests are computed the exact same way.

7. 7 Example Academic Major SexPhysicsEnginrEnglishDesignTotal Male108 (A)345 (B)94 (C)17 (D)564 Female8 (E)12 (F)253 (G)60 (H)333 Total11635734777897 1.Compute the Expected Cell Frequencies 1.Example: A = (Row Total X Column Total) / Grand Total = (564 X 116) / 897

8. 8 Example Academic Major SexPhysicsEnginrEnglishDesignTotal Male108 (A)345 (B)94 (C)17 (D)564 Female8 (E)12 (F)253 (G)60 (H)333 Total11635734777897 After we compute all expected cell counts: Use formula to compute Chi-Square statistic: df=(R-1)(C-1)

9. 9 Details: Sample Sizes In general, the greater the degrees of freedom (i.e., the more values/categories on the independent and dependent variables), the more lenient the minimum expected frequencies threshold Some recommend 5 or more per cell, some recommend more than 5, and others require 10 or more A common rule is 5 or more in all cells of a 2- by-2 table, and 5 or more in 80% of cells in larger tables, but no cells with zero count

10. 10 Unix Example: Married women at home get file cirp8690.sav. crosstab view8612 by sex86 /cells=count row column /statistics=chisq. VIEW8612 MARRIED WOMEN BEST IN HOME by SEX86 STUDENT'S SEX SEX86 Page 1 of 1 Count | Row Pct |MALE FEMALE Col Pct | Row | 1 | 2 | Total VIEW8612 --------+--------+--------+ 1 | 762 | 1590 | 2352 DISAGREE STRONG | 32.4 | 67.6 | 55.1 | 36.4 | 73.1 | +--------+--------+ 2 | 889 | 343 | 1232 DISAGREE SOME | 72.2 | 27.8 | 28.9 | 42.5 | 15.8 | +--------+--------+ 3 | 343 | 170 | 513 AGREE SOME | 66.9 | 33.1 | 12.0 | 16.4 | 7.8 | +--------+--------+ 4 | 97 | 72 | 169 AGREE STRONG | 57.4 | 42.6 | 4.0 | 4.6 | 3.3 | +--------+--------+ Column 2091 2175 4266 Total 49.0 51.0 100.0

11. 11 Actual vs. Expected crosstab view8612 by sex86 /cells=count expected /statistics=chisq. VIEW8612 MARRIED WOMEN BEST IN HOME by SEX86 STUDENT'S SEX SEX86 Page 1 of 1 Count | Exp Val |MALE FEMALE | Row | 1 | 2 | Total VIEW8612 --------+--------+--------+ 1 | 762 | 1590 | 2352 DISAGREE STRONG |1152.8 |1199.2 | 55.1% +--------+--------+ 2 | 889 | 343 | 1232 DISAGREE SOME | 603.9 | 628.1 | 28.9% +--------+--------+ 3 | 343 | 170 | 513 AGREE SOME | 251.4 | 261.6 | 12.0% +--------+--------+ 4 | 97 | 72 | 169 AGREE STRONG | 82.8 | 86.2 | 4.0% +--------+--------+ Column 2091 2175 4266 Total 49.0% 51.0% 100.0% Chi-Square Value DF Significance -------------------- ----------- ---- ------------ Pearson 594.08275 3.00000

12. 12 Analysis of Variance The most basic type of Analysis of Variance is one in which there is only one continuous, dependent variable and more than two treatment levels (One-Way); the more complex types use similar logic Unlike a t-test that can only be used to compare two means, ANOVA can be used to compare two or more means: Null Hypothesis

13. 13 ANOVA Example: We want to look at whether or not there is a statistically significant difference in SAT prep programs. One outcome (SAT score). Three levels in type of program: Kaplan, Sylvan, Mitron If the variability between groups is greater than the variability within groups, this is evidence of a treatment effect.

14. 14 How it Works Analysis of Variance works by comparing two estimates of variance (  2 ). –One estimate (Mean Square Error or "MSE") is based on the variances within the samples. The MSE is an estimate of  2 whether or not the null hypothesis is true. –The second estimate (Mean Square Between or "MSB" for short) is based on the variance of the sample means. The MSB is only a good estimate of  2 if the null hypothesis is true. If the null hypothesis is false then MSB estimates something larger than  2.

15. 15 How it Works If the null hypothesis is true, then MSE and MSB should be about the same since they are both estimates of the same quantity (  2 ). This would give a ratio of the two around 1. However, if the null hypothesis is false then MSB can be expected to be larger than MSE since MSB is estimating a quantity larger then . Thus, if MSB is sufficiently larger than MSE, then the ratio of the two would be greater than 1, suggesting evidence to reject the Null hypothesis that there is no difference between the groups.

16. 16 Computations, Definitions, Distributions F = MSB / MSE Two degrees of freedom are involved: dfnumerator = a - 1 dfdenominator = N - a a is the number of groups N is the total number of subjects in all groups

17. 17 An Interactive Demonstration Simulation of ANOVA

18. 18 Design Requirements There is one dependent variable with two or more treatment levels The levels of the dependent variables differ either quantitatively or qualitatively A subject may appear in only one group

19. 19 Assumptions The subjects scores are independent The scores within each treatment (level) are normally distributed The variances across treatment groups (levels) are equal (homogeneity) When cell sizes are equal, ANOVA is robust to violation of the homogeneity

20. 20 Other Details In this course we will consider only fixed effect ANOVA The F-test is an omnibus test –It tests only if there is A difference among the means, we don’t know where the differences exist. In more advanced applications it is possible to do post-hoc comparisons to find out which groups means differ

21. 21 Next Week Analysis of Covariance –Chapter 17 p. 504-516