Analysis of Variance: Some Final Issues Degrees of Freedom Familywise Error Rate (Bonferroni Adjustment) Magnitude of Effect: Eta Square, Omega Square Review: Main Effects, Interaction Effects and Simple Effects

Degrees of Freedom (df ) Number of “observations” free to vary. There will be one df associated with the effect you are reporting, and one associated with error. X Xdf total = N – 1 ( N observations) X Xdf groups = g – 1 (Number of groups) X Xdf error = g (n - 1) (An easier easy to compute: df total d – df groups)

Summary Table F(3,68) = 35.8; p<.05

Familywise Error Rate Suppose you are comparing 5 groups. Significant F only shows that not all groups are equalSuppose you are comparing 5 groups. Significant F only shows that not all groups are equal XWhat groups are different. Number of pairwise comparisons are 10 Error rate operates at the level of each comparison.Error rate operates at the level of each comparison. Error rate increases with number of comparisons.Error rate increases with number of comparisons.

In case of multiple comparisons: Bonferroni adjustment The more tests we run the more likely we are to make Type I error.The more tests we run the more likely we are to make Type I error. XGood reason to hold down number of tests Run t tests between pairs of groups, as usualRun t tests between pairs of groups, as usual XHold down number of t tests XReject if t exceeds critical value in Bonferroni table Works by using a more strict level of significance for each comparisonWorks by using a more strict level of significance for each comparison

Bonferroni t--cont. Critical value of a for each test set at.05/c, where c = number of tests runCritical value of a for each test set at.05/c, where c = number of tests run XAssuming familywise a =.05 Xe. g. with 3 tests, each t must be significant at.05/3 =.0167 level. With computer printout, just make sure calculated probability <.05/cWith computer printout, just make sure calculated probability <.05/c Necessary table is in the bookNecessary table is in the book

Magnitude of Effect Why you need to compute magnitude of effect indicesWhy you need to compute magnitude of effect indices XLevel of significance tells us nothing about size of effect XT and F values inflate with sample size XHard to compare with statistic with other kinds of analysis

Visual Explanation of Magnitude of Effect

Magnitude of Effect--cont. Eta squared (h 2 )Eta squared (h 2 ) XEasy to calculate XSomewhat biased on the high side XPercent of variation in the data that can be attributed to treatment differences Omega squared (w 2 )Omega squared (w 2 ) XMuch less biased than h 2 XNot as intuitive XWe adjust both numerator and denominator with MS error XFormula on next slide

h 2 and w 2 for example problem h 2 =.18: 18% of variability in symptoms can be accounted for by treatment h 2 =.18: 18% of variability in symptoms can be accounted for by treatment w 2 =.12: This is a less biased estimate, and note that it is 33% smaller. w 2 =.12: This is a less biased estimate, and note that it is 33% smaller.

R2 is also often used. It is based on the sum of squares. For experiments use Omega Squared. For correlations use R squared. Value of R square is greater than omega squared. Cohen classified effects as Small Effect:.01 Medium Effect:.06 Large Effect:.15

Factorial Analysis of Variance What is a factorial design?What is a factorial design? Main effectsMain effects InteractionsInteractions Simple effectsSimple effects Magnitude of effectMagnitude of effect

There are two factors in the analysis: Type of Rating and Why item was recommended If you examine effect of “Knowing Why” (ignoring Type of Rating for the time being), you are looking at the main effect of Knowing Why. If we look at the effect of Type of Rating, ignoring Knowing Why, then you are looking at the main effect of Knowing Why. Main effects

If you could restrict yourself to one level of one IV for the time being, and looking at the effect of the other IV within that level. Effect of Knowing Why at one level of Type of Rating at, then that is a simple effect of Knowing Why on Interest Ratings. Above is identical to t-test Simple effects Simple Effect of Knowing Why at one level of Type of Rating (I.e., Interest)

Interactions (Effect of one variable on the other) Does Knowing Why effect Interest and Confidence differentially?

Example Data with computation (cell means and standard deviations)

Plotting Results

Effects to be estimated Differences due to instructionsDifferences due to instructions XErrors more in condition without instructions Differences due to genderDifferences due to gender XMales appear higher than females Interaction of video and genderInteraction of video and gender XWhat is an interaction? XDo instructions effect males and females equally? Cont.

Estimated Effects--cont. ErrorError Xaverage within-cell variance Sum of squares and mean squaresSum of squares and mean squares XExtension of the same concepts in the one-way

Calculations Total sum of squaresTotal sum of squares Main effect sum of squaresMain effect sum of squares Cont.

Calculations--cont. Interaction sum of squaresInteraction sum of squares XCalculate SS cells and subtract SS V and SS G SS error = SS total - SS cellsSS error = SS total - SS cells Xor, MS error can be found as average of cell variances

Degrees of Freedom df for main effects = number of levels - 1df for main effects = number of levels - 1 df for interaction = product of df main effectsdf for interaction = product of df main effects df error = N - ab = N - # cellsdf error = N - ab = N - # cells df total = N - 1df total = N - 1

Calculations for Data SS total requires raw data.SS total requires raw data. XIt is actually = SS video SS video Cont.

Calculations--cont. SS genderSS gender Cont.

Calculations--cont. SS cellsSS cells SS VXG = SS cells - SS instruction - SS gender = = 0.125SS VXG = SS cells - SS instruction - SS gender = = Cont.

Calculations--cont. MS error = average of cell variances = ( )/4 =58.89/4 = MS error = average of cell variances = ( )/4 =58.89/4 = Note that this is MS error and not SS errorNote that this is MS error and not SS error

Summary Table

Elaborate on Interactions Diagrammed on next slide as line graphDiagrammed on next slide as line graph Note parallelism of linesNote parallelism of lines XInstruction differences did not depend on gender

Line Graph of Interaction