Multivariate Analysis: Analysis of Variance Sociology 680 Multivariate Analysis: Analysis of Variance

1) Analysis of Variance Models 2) Structural Equation Models A Typology of Models IV DV Category Quantity 1) Analysis of Variance Models (ANOVA) 2) Structural Equation Models (SEM) Linear Models 3) Log Linear Models (LLM) 4) Logistic Regression Models (LRM) Category Models

Examples of the Four Types 1. The effects of sex and race on Income 2. The effects of age and education on income 3. The effects of sex and race on union membership 4. The effects of age and income on union membership

The General Linear Model Recall that the bi-variate Linear Regression model focuses on the prediction of a dependent variable value (Y), given an imputed value on a continuous independent variable (X). The variation around the mean of Y less the variation around the regression line (Y’) is our measure of r2 Y (Weight) .. .… . . …. . … . … . . .. .. . .. . …. …. . .. … . ... . X (Height) Y’

The General Linear Model (cont.) Fixing a value of (X) and predicting a value of (Y) allows us to use the layout of points, under an assumption of linearity, to determine the effect of the IV on the DV. We do this by calculating the Y’ value in conjunction with the standard error of that value (Sy’) Where: and Y (Weight) Y’ .. .… . . …. . … . … . . .. .. . .. . …. …. . .. … . ... . { Y’ X (Height)

An Example of Simple Regression Given the following information, what would you expect a student’s score to be on the final examination, if his score on the midterm were 62? Within what interval could you be 95% confident the actual score on the Final would fall (i.e. what is the standard error)? Midterm (X)   Final (Y) = 70 = 75 Sx = 4 Sy = 8   r = 0.60 Y’ = 75 + (0.6)(8/4)(62-70) = 65.4 = 8 (.8) = 6.4

The Test of Differences But now assume that the goal is not prediction, but a test of the difference in two predictions (e.g. “are people who are 5’8” significantly heavier than those who are 5’4”). That difference hypothesis could just as easily be recast as “Are taller people significantly heavier than shorter people, where taller and shorter connote categories. Y (Weight) .. .… . . …. . … . … . . .. .. . .. . …. …. . .. … . ... . X (Height) Y’ Y’1 Y’2

The t-test If there are simply two categories, we would be doing an ordinary t-test for the difference of means where: Y (Weight) Y’ . .. ... …. ... . .. .. . …. … .. . Shorter Taller | | Y’1 Y’2 X (Height)

Analysis of Variance If we were to have three categories, the test of significance becomes a simple one-way analysis of variance (ANOVA) where we are assessing the variance between means (Y’s) of the categories in relation to the variation within those categories, or: Variance Between Categories Variance Within Categories Y (Weight) . .. ... …. ... . .. .. . …. . .. .. . .... .. . Short Med Tall | | | Y’ Y’1 Y’2 Y’2 X (Height)

Three Types of Analysis of Variance One Way Analysis of Variance - ANOVA (Factorial ANOVA if two or more - IVs) Analysis of Covariance - ANCOVA (Factorial ANCOVA if two or more - IVs) Multiple Analysis of Variance (MANOVA) (Factorial MANOVA if two or more 2IVs)

Simple One Way ANOVA Concept: When two or more categories of a non-quantitative IV are tested to see if a significant difference exists between those category means on some quantitative DV, we use the simple ANOVA where we are essentially looking at the ratio of the variance between means / variance within categories. As an F-ratio: F-ratio = Bet SS/df divided by Within SS/df. As a formula it is:

Example of a simple ANOVA Suppose an instructor divides his class into three sub-groups, each receiving a different teaching strategies (experimental condition). If the following results of test scores were generated, could you assume that teaching strategy affects test results? In Class At Home Both C+H 115 125 135 145 155 140 150 160 165 175 185 140 150 160 Grand Mean = 150

Example of a simple ANOVA (cont.) Step 1: State hypotheses: Ho: 1 = 2 = 3; Step 2: Specify the distribution: (F-distribution) Step 3: Set alpha (say .05; therefore F = 3.68) Step 4: Calculate the outcome: Step 5: Draw the conclusion: Retain or Reject Ho: Type of instruction does or does not influence test scores.

Example of a simple ANOVA (cont.) In Class At Home Both C+H 115 125 135 145 155 140 150 160 165 175 185 Bet SS = ((5(140-150)2 + 5(150-150)2 +5 (160-150)2)) = 1000 Bet df = 3-1 = 2 W/in SS = (115-140)2 + (135-140)2 + (140-140)2 + (145-140)2+ (165-140)2 + (125-150)2 + (145-150)2 + (150-150)2 + (155-150)2 + (175-150)2 + (135-160)2 + (155-160)2 + (160-160)2 + 165-160)2 + (185-160)2 = 3900 W/in df = 15 – 3 = 12 Source SS df MS F Bet 1000 2 500 1.54 Within 3900 12 325

SPSS Input for One-way ANOVA

SPSS Output from a simple ANOVA

Two Way or Factorial ANOVA Concept: When we have two or more non-quantitative or categorical independent variables, and their effect on a quantitative dependent variable, we need to look at both the main effects of the row and column variable, but more importantly, the interaction effects.

Example of a Factorial ANOVA In Class At Home Both C+H 115 125 135 145 155 140 150 160 165 175 185 Working 135 Not Working 160 Means 140 150 160 150

SPSS Input for 2x3 Factorial ANOVA

SPSS Output from a 2x3 ANOVA

Analysis of Covariance (ANCOVA) Concept: Not unlike a 2-way ANOVA, ANCOVA introduces a second independent variable. However, it is not always subject to experimental control (as would be the case in a 2-way ANOVA) and is typically quantitative in nature. Therefore we treat the second IV as a “covariate” of the DV. Example: to study the effect of race and education (IVs) on income (DV), we would adjust the racial differences by the correlation between education (the covariate) and income. This reduces the residual / error variance (which is the denominator in the F-ratio for the main effect of racial differences).

ANCOVA (cont.) In this example, we are essentially subtracting the covariance of X&Y from both the Bet SS and Within SS of the racial categories:

SPSS Input from an ANCOVA

SPSS Output from an ANCOVA

Multiple Analysis of Variance (MANOVA) Concept: MANOVA tests whether significant differences among means of multiple (k) categories exist on a combination of dependent variables. Example: to study the effects job satisfaction (IV) on hours worked and income earned (DVs). In essence we do a linear combination of the DV and then perform the equivalent of a simple ANOVA for the IV job satisfaction.

Input for MANOVA

Output for MANOVA