1. Department of Cognitive Science
Inferential Statistics III: ANCOVA
Michael J. Kalsher
Department of Cognitive Science
MGMT PSYCHOMETRICS © 2014 Kalsher

2. Analysis of Covariance
When and Why do we use ANCOVA?
Partitioning Variance
Doing ANCOVA using SPSS
Interpretation
Main Effects
Covariates

3. What is ANCOVA?
ANOVA can be extended to include one or more continuous variables, termed covariates, that are not part of the main experimental manipulation, but have an influence on the DV.
ANCOVA tests whether manipulated factors (IVs) have a significant effect on the DV after removing-- or partialing out--the variance accounted for by the covariates.

4. Accounting for Variance: With and Without a Covariate
SSM
SSR =
Systematic Variance
Unsystematic Variance
SSR without Covariate
SSR with Covariate
SSM
SSM
SSR
SSR
SSCov
Variance explained by the covariate reduces the overall amount of unexplained (error) variance.

5. Purpose of Covariates in ANOVA
Reduce error variance
The F-ratio in ANOVA compares the amount of variability explained by the experimental manipulation (MSM), against the variability that it cannot explain (MSR). What we hope is that the covariate explains some of the variance that was previously unexplained. This has the effect of reducing the unexplained variance (i.e., MSR becomes smaller) and so our F-ratio gets bigger. In real terms this means we obtain a more sensitive measure of the experimental effect.
Eliminate Confounds
In any experiment, there may be unmeasured variables that confound the results (i.e., a variable that varies systematically with the experimental manipulation). If any variables are known to influence the dependent variable being measured, then ANCOVA can help to remove the bias of these variables. Once a possible confounding variable has been identified, it can be measured and entered into the analysis as a covariate.
ANCOVA is like an ANOVA on the values of the DV, after removing the influence of the covariate, rather than on the original values.

6. Example 1: Using ANCOVA to evaluate the relative effectiveness of competing math textbooks
An educational psychologist is interested in evaluating the relative effectiveness of two textbooks for improving math skills.
A literature review reveals that general IQ is related to math skills.
She uses this information to make the test of the association between math skills and textbook type more sensitive (i.e., more powerful).
Procedure
In each one of the two textbook groups, we compute the correlation coefficient between IQ and math skills.
We then estimate the amount of variation in math skills that is accounted for by IQ, and the amount of residual variation (the variance in math skills not explained by IQ).
We use the residual variance in an ANOVA as an estimate of the true unexplained variance (error) after controlling for IQ.

7. Example 2: Evaluating the effectiveness of the types of treatment for treating a fatal disease
A researcher compares the effectiveness of three treatments ("Placebo", "Drug 1", and "Drug 2”) for treating a fatal disease in terms of average survival time.
ANOVA could be used to analyze these data. But, suppose that supplementary information is available to him (i.e., each patients’ age).
Age in this case is a "covariate" - it is not related to treatment, but it might be related to survival time.
He uses the information to make the test of the association between survival time and treatment type more sensitive (more powerful).
In each of the treatment groups, he computes the correlation coefficient between age and survival time.
He estimates the amount of variation in survival time accounted for by age, and the amount of residual variation that is not explained by age.
He uses the residual variance in an ANOVA as an estimate of the true error after controlling for age.

8. Choosing Covariates
In general, it is preferable to have a small number of covariates.
Covariates should be correlated with the DV, but NOT with each other.
When covariates are significantly correlated:
they contribute very little to error reduction
they can cause computational difficulties such as multicollinearity.
one or the other should be removed since they are statistically redundant.

9. Assumptions of ANCOVA
ANCOVA has the same basic assumptions of all of the parametric tests, but also has two additional requirements:
independence of the covariate and treatment effect.
homogeneity of regression slopes.

10. Assumptions of ANCOVA: Independence of the covariate and treatment effect
We assume the covariate is correlated with the outcome variable (DV), so that as scores on the covariate change, scores on the DV change by a similar amount.
Important to ensure that:
the covariate explains previously unexplained variance in the DV—not variance explained by the treatment.
the effect that the covariate has on the outcome should be the same for all of the experimental groups.
Randomizing participants to experimental groups or matching experimental groups on the covariate can help.
Important to test this before running the ANCOVA.

11. Basic ANOVA showing that the total variance in the DV can be partitioned into treatment variance (SSM) and error or unexplained variance (SSR).
Ideal situation for ANCOVA in which the covariate shares variance with the currently “unexplained” variance, but is independent from the treatment effect (no overlapping variance).
Situation in which ANCOVA should NOT be used since the effect of the covariate overlaps with the experimental effect. Thus, the experimental effect is confounded with the effect of the covariate.

12. Assumptions of ANCOVA: Homogeneity of Regression Slopes
What ANCOVA actually does is to create a single regression equation, ignoring groups, to predict dependent variable (Y) scores using the covariate(s) (X1, X2, X3). Then, in essence, an ANOVA, not ignoring groups, is performed using the residualized scores of each person computed by subtracting each person’s predicted score based on the single equation, from the participants’ actual scores.
These computations are legitimate if the regression equations that predict the Y scores, computed separately in the ANOVA groups, have parallel slopes. This is the homogeneity of regression assumption, which requires that the “b” weight applied to the covariates are reasonably equal across each group.

13. Assumptions of ANCOVA: Homogeneity of Regression Slopes
The slope of the line predicting the DV from the CV should be relatively the same for each level of the IV.
In other words the regression coefficient (bi) relating a particular CV to the DV should be the same for each group.
In still other words, this means no IV x DV interaction.

14. Sample ANCOVA Problem: Viagra and Libido
Previously, we considered an example looking at the effects of Viagra on libido. Viagra was shown to influence libido, but other factors could also play a role in explaining a person’s libido, including their health, mood, and their partner’s libido.
If one or more of these variables is measured, it is possible to control for the influence they have on the dependent variable by including them in the analysis.
In the context of hierarchical regression, we would enter one or more of these variables into the model in the first block, followed by entry of the “dummy variables” representing the experimental manipulation in the second block. As such, we “partial out” the effect of the covariates (e.g., their partner’s libido) to see what effect the independent variable has after the effect of the covariate.

15. ViagraCovariate.sav Dose Participant’s Libido Partner’s Libido Placebo
3.22 (1.79)
3.44 (2.07)
Low Dose
4.88 (1.46)
3.12 (1.73)
High Dose
4.85 (2.12)
2.00 (1.63)
ViagraCovariate.sav

16. How Does ANCOVA Work? Imagine we had just two groups:
Placebo
Low Dose
This paradigm can be expressed as a regression equation using a dummy coding variable:

17. Dummy Coding Placebo = 0, Low Dose = 1
When Dose = Placebo, Predicted Libido = mean of placebo group:
When Dose = Low Dose, Predicted Libido = Difference between the means of the Placebo and Low Dose groups:

18. ANCOVA and the GLM
We can run a regression with Libido as the outcome and the Dose (Placebo or Low) as the predictor, Note:
Intercept is the mean of Placebo group
b for the Dummy Variable is the difference between the means of the placebo and low dose group ( = 1.66)

19. ANCOVA as Regression
The covariate can be added to the regression model of the ANOVA.
To evaluate the effect of the experimental manipulation controlling for the covariate we enter the covariate into the model first (think back to hierarchical regression), then next the dummy coded IV (dose of Viagra).
Yi = b0 + b3Covariatei + b2Highi + b1Lowi + εi
Yi = b0 + b3Partner’s Libidoi + b2Highi + b1Lowi + εi

20. SPSS Output: ANCOVA as Regression

21. SPSS Output: ANCOVA as Regression

22. ANCOVA Initial Considerations: Testing Independence of the IV and Covariate

23. Initial Considerations: One-Way ANOVA using Partner’s Libido as the DV
The non-significant F-test (p>.05) tells us that the treatment groups do not differ significantly on the covariate (Partner’s libido).

24. ANCOVA: Main Analysis

25. ANCOVA: Contrasts
Note that the “Post-Hoc” option isn’t available for analyses using covariates.
Be sure to “click” on the “Change” button!

26. ANCOVA: Options You can perform “Post-Hoc” tests through “Options”.
Move the IV, “Dose”, into the “Display Means for” box.
“Click” on the “Compare main effects” box.
The “Confidence interval adjustment” option becomes available.
The resulting output will be a table of estimated marginal means (the means adjusted for the effect of the covariate).

27. SPSS Output: Descriptive Statistics and Homogeneity test

28. Testing for Equal Variance: Different Approaches
Levene’s Test
If Levene’s test is significant, a more stringent alpha can be used (.01) or the variable can be dropped from the analysis.
This is a very conservative test and is not necessarily the best way to judge whether variances are unequal enough to cause problems.
Alternative Procedure: Highest and lowest variances
Obtain standard deviation values for each of the groups; square these values to obtain the variances.
Take the largest value and divide it by the smallest value. If the resulting value is less than 2, then we shouldn’t worry too much. If it is larger than 2, then we do (see next slide).

29. The ratio of the highest variance (4. 49) to lowest variance (2
The ratio of the highest variance (4.49) to lowest variance (2.13) = Look up the critical value for 10 subjects, and 3 variances.
Critical value = 5

30. SPSS Output: Main Analysis
ANCOVA
ANOVA

31. The Main Effect
F(2, 26) = 4.14, p < .05

32. SPSS Output: Parameter Estimates
Dummy Coded Variables
Dose = 1: Represents the difference between the Placebo and High Dose groups.
Dose = 2: Represents the difference between the Low Dose and High Dose groups.
Dose = 3: The High Dose reference group.

33. SPSS Output: Planned Contrasts
Low dose vs. Placebo
High dose vs. Placebo
Adjusted Means

34. SPSS Output: Post-Hoc Tests
Note: Difference between Placebo and Low Dose groups is no longer significant (p>.05).

35. Assumption of Homogeneity of Regression Slopes
We assume the relationship between the DV and covariate is the same in each treatment group.

36. Re-run the ANCOVA: Custom Model
The purpose: To test the Covariate by Outcome Interaction If the interaction is significant, the homogeneity of regression slopes has been violated.

37. Step 1
Step 2
Step 3

38. SPSS Output: Testing the Dose*Partner_Libido Interaction

39. Calculating Effect Size for ANCOVA
Eta squared (2 ) is essentially r2. Derived by dividing SSM by SST. Assesses the proportion of total variance explained by a variable.
Partial eta squared (partial 2 ) assesses the proportion of variance that a variable explains that is not explained by other variables in the analysis.
2 = SSEffect
SSTotal
Partial 2 = SSEffect
SSEffect + SSResidual

40. Calculating Effect Size for ANCOVA
Partial 2 Dose = SSDose = =
SSDose + SSResidual
Partial 2 Partner Libido = SSPartnerLibido = =
SSPartnerLibido + SSResidual
These values show that Dose explained a bigger proportion of the variance not attributable to other variables than Partner_Libido

41. (see pp. 416-417 for specifics of results reporting).
ANCOVA: Reporting the Results
The covariate, partner’s libido, was significantly related to the participant’s libido, F(1,26) = 4.96, p<.05, r = There was also a significant effect of Viagra on levels of libido after controlling for the effect of partner’s libido, F(2,26) = 4.14, p<.05, partial 2 = .24.
Planned contrasts revealed that having a high dose of Viagra significantly increased libido compared to having a placebo, t(26) = -2.77, p<.05, r=.48, but not compared to having a low dose, t(26) = -0.54, p>.05, r = .11.
(see pp for specifics of results reporting).

42. Sample Problem #1: Hangover Cures
A marketing manager for a well-known beverage manufacturer (Alka- Seltzer) believes his product is more effective than other recommended hangover “cures.” To test his product against competitor cures he invites 15 people to join him at a bar and proceeds to get them drunk.
The next morning, he gives 5 of them strong coffee to drink (which is assumed to have no beneficial effect), 5 of them Alka Seltzer, and 5 of them Mimosas (champagne and orange juice—”hair of the dog that bit them”). This variable is called “drink”. He measures how well they feel two hours later on a Likert-type scale with the following anchors: 0 = “I feel terrible” to 10 “I feel terrific”. This variable is called “well”.
The marketing manager realizes that it is important to control for how drunk the person got the night before, and so he measures this on another Likert-type scale with the following anchors: 0 = “Completely sober” to 10 = “Completely intoxicated”. This variable is called “drunk”.

43. Hangover Cures Dataset
Strong Coffee
Alka Seltzer
Mimosas
Well
Drunk 5 6 2 3 4 1 8 7
Your task:
Conduct the appropriate analyses to see whether the drinks differ in their ability to treat hangovers when controlling for how much was drunk the night before. (Hint: Run the ANOVA first, then the ANCOVA).
Write up your results. Is Alka Seltzer better than the other cures? What will you conclude from the study?