Research methods and statistics Tests of Difference Research methods and statistics

Tests of Difference

Learning outcomes At the end of these session and with additional reading you will be able to: describe the assumptions of ANOVA, ANCOVA, MANOVA describe main effects and interactions interpret the F ratio decide when to use post hoc or planned comparisons

Analysis of Variance (ANOVA) ANOVA allows analysis on two or more independent variables e.g. ethnicity and gender ANOVA allows analysis on two or more conditions e.g. Black, White Asian MANOVA - Multivariate analysis of variance (ANOVA for several dependent variables)

ANCOVA Analysis of co-variance The analysis of continuous (ratio) variables that predict the dependent variable These variables are called covariates The analysis of covariates reduce the within-group error variance Elimination of confounding variables

Assumptions of ANOVA, MANOVA, ANCOVA The dependent variable comprises of interval or ratio data The dependent variable is normally distributed The variance in each condition are fairly similar (Homogeneity of variance) Observations should be independent (not applicable in repeated measures ANOVA’s)

How are ANOVA’s, MANOVA;s, ANCOVA’s described Independent variables are known as factors Conditions are know as levels e.g. gender is a factors with two levels male/female

Familywise error Why not just perform numerous t tests ? Every time you perform a test there is a 5% chance of the results occurring due to sampling error If you perform numerous test on the same data you will increase the likelihood of error: 1-(.95)n, where n is the no of tests required e.g. 3 tests on previous example of ethnicity 1-(.95)³ = 1-0.857=.143 or 14%

Interactions e.g. gender and ethnicity on performance By analysisng two or more factors we are able to investigate the interaction between the factors on the DV e.g. gender and ethnicity on performance using the first example this would be described as a 2 x 3 ANOVA (where 2 represents gender with 2 levels and 3 represents ethnicity with 3 levels)

Main effects and interaction A one way ANOVA where there is only one factor would produce: a main effect of A A two way ANOVA where there are two factors would produce: a main effect of B a two interaction between A and B

What are ANOVA’s looking for ANOVA’s are assessing whether the variance between a condition is larger than the variance within conditions. Only then can we suggest that a factor is having an effect on the DV.

The F ratio The variance brought about by other nuisance factors such as individual differences is often referred to as the error variance An ANOVA calculates the ratio of variance due to the manipulation of the IV and the error variance: this is know as the F ratio

The F ratio II If the error variance is small then the F ratio will be greater than 1 If the error variance is large then the F ratio will be less than 1

Post Hoc or planned comparisons The F ratio tells us only whether there is a difference between means in a factor, but it does not tell us exactly where it lies It is therefore necessary to conduct further analysis to find out which groups differ. However we must remember that we do not want to inflate the familywise error This can be achieved in two ways planned comparisons post hoc comparisons

Post Hoc or planned comparisons The difference between planned and post hoc comparisons are akin to one/two tailed hypothesis Planned comparison should be used when you have specific hypothesis that you want to test Post hoc comparisons should be performed when you have no specific hypothesis

Post hoc comparisons Bonferroni and Tukeys both control the type 1 error rate very well Bonferroni is best used when the number of comparisons are small Tukey is best used when there are a large number of comparisons

Planned comparisons Factors are broken down into smaller chunks of variance e.g. high/low/placebo effect contrast 1 will be between low and high against the placebo contrast 2 will be low against high

Planned comparisons Comparisons must then be weighted each chunk of variance must be given a weight each chunk of variance must be positive or negative the sum of weights should always add up to zero

Example Chunk 1 High/low Chunk 2 placebo contrast 1 1 2 weight positive negative sign high low placebo -2

Reporting results F (df) = F value, P< or P > than .05