Running Scheffe’s Multiple Comparison Test in Excel For finding Inter-Groups Differences after getting significant results in overall ANOVA test

Scheffe’s Test Scheffe’s Test is a very popular and the most conservative Post Hoc Test (Post Hoc = unplanned before experiments). It is most useful in cases of UNequal sample size, when the number of groups increase or when planned comparison of contrast being required (Not a Post Hoc test then)!

Part One One-Way Anova

e.g. Completely Randomized Design Unequal Sample Size An experiment with Completely Randomized Design has been started with 10 equal weight chickens in each group of Treatment A (Control), Treatment B, Treatment C and Treatment D, with increasing dosage of a new drug that might increase growing rate. However, some chickens have died during the experiment, especially in groups with higher dosage. Please find any significant different increase of weight among the 4 groups.

In Excel with ‘Analysis ToolPak’ Add-In activated, click Data, Data Analysis :-

Choose ‘Anova: Single Factor’ = One-way Anova

Select Data Area including Labels:-

Overall Anova result:- The overall Anova result reject the null hypothesis that all group means are equal! For finding exactly where the differences exist, we proceed to run Scheffe’s Test!! Critical F value to be used later N Group Means to be used later Within Group Variance MSw to be used later

Step One – Creating a ‘Critical F value for Scheffe’ - F This ‘Critical F Value for Scheffe’ is calculated by: F = ‘F Crit’ in Anova X (Num. of Groups – 1) = 3.0088 X 3 = 9.0264

Step Two – Calculation of ‘F for Scheffe’ for all combinations Equation for calculation of Fs (F for Scheffe) :-

Any F > 9.0264 would indicate significant differences F for A vs B = (10.0456-10.3474)2/1.2408(1/10 +1/8) = 0.0911/0.2792 = 0.3262 < 9.0264 F for A vs C = (10.0456-12.0956)2/1.2408(1/10 +1/6) = 4.2025/0.3310 = 12.6964 > 9.0264 F for A vs D = (10.0456-11.8789)2/1.2408(1/10 +1/4) = 3.3610/0.4343 = 7.7389 > 9.0264 F for B vs C = (10.3474-12.0956)2/1.2408(1/8 +1/6) = 3.0562/0.3619 = 8.4449 > 9.0264 F for B vs D = (10.3474-11.8789)2/1.2408(1/8 +1/4) = 2.3455/0.4653 = 5.0408 > 9.0264 F for C vs D = (12.0956-11.8789)2/1.2408(1/6 +1/4) = 0.0470/0.5170 = 0.0910 > 9.0264 Significant Difference between A and C !!

Counter Checking with SPSS Using the same Data Set

Choose Post Hoc test e.g. Scheffe :-

Overall Anova result similiar to that in Excel:-

The Result in SPSS for the Scheffe Test well matched the result in Excel that only Group A and Group C are found to be significantly different in their Group Mean.

Proving that the Excel results are exactly equal to that in SPSS!! Although the Excel result for the Scheffe test well matched that in SPSS, this might not be enough to prove the figures they got are absolutely the same! . Unlike in Tukey’s Test that ‘Critical Differences’ are can be used for counter checking to 95% Conf. Int. of the SPSS output. But we can simply use another method by checking the ‘F-Table’ backward.

For Proving that the Excel result is consistent with SPSS!! For Example, for Group A vs Group C :- F for A vs C = (10.0456-12.0956)2/1.2408(1/10 +1/6) = 4.2025/0.3310 = 12.6964 > 9.0264 12.6964/(k-1) = 12.6964/3 = 4.2321 Num of Groups - 1 Let’s Check the F Table on df (3,24)

0.016 4.2321 SPSS output Excel Result

Conclusion After activating the ‘Analysis TookPak’ Add-in in Excel, we can have useful statistical tests to use including different Anova tests. We find that, if overall Anova result is significant, we can work further to run Post Hoc Test e.g. Scheffe’s Test to find where the mean differences exist, not too difficultly! For One-way Anova, the Excel result has been proved to be consistent with SPSS, even with Unequal Sample Size! Let’s go to Part 2 for Two-way Anova now!!

Part 2 Two-Way Anova

e.g. Scheffe’s HDS Test using Excel in aXb factorial Design With Replication - 6 cages each with 4 rats have been used for a Completely Randomized Two-Factors (a x b factorial) With Replication Design Experiment. The 24 rats had been assigned randomly to be subjects for the ‘combinations’ of factor one (Diet A, B, C, D) with factor two (Lighting 1, 2, 3-2 times each). The response is a ‘score’ after the 12 ‘treatments’ e.g. a growing rate in body weight within a certain period of time. Please find any Significant Differences caused by the two factors.

Running the Scheffe’s Test in Excel e.g.Two-way Anova aXb Factorial Design With Replication

In Excel with ‘Analysis ToolPak’ Add-In activated, click Data, Data Analysis :-

Choose ‘Two-Factor With Replication’:-

Select Data Area including all Labels :-

A closer look:- Range Including Labels Number of rows of Replication

Output :- Overall Anova Results

Pair of degree of freedom To be used for checking F!! ‘Critical F’ value for Scheffe’s Test Overall Anova Result Lighting ‘MSE’ for Scheffe’s Test (Significant) Diet (Significant) Interaction (Not Significant)

For the factor ‘Diet’ the Group Means are:-

Step One – Creating a ‘Critical F value for Scheffe’ - F This ‘Critical F Value for Scheffe’ is calculated by: F = ‘Critical F in Anova’ X (Num. of Groups – 1) = 3.4903 X 3 = 10.4709

Step Two – Calculation of ‘F for Scheffe’ for all combinations Equation for Group Differences in Scheffe’s Test:-

Any F > 10.479 would indicate significant differences for ‘Diet’ F for A vs B = (13.9167-22.3333)2/10.417(1/6+1/6) = 70.8392/3.4726 = 20.3977> 10.4709 F for A vs C = (13.9167-13.8333)2/10.417(1/6 +1/6) = 0.0070/3.4726 = 2.0158 < 10.479 F for A vs D = (13.9167-21.4167)2/10.4167(1/6+1/6) = 56.25/3.4726 = 16.198 > 10.479 F for B vs C = (22.3333-13.8333)2/10.4167(1/6 +1/6) = 72.25/3.4726 = 20.8057 > 10.479 F for B vs D = (10.3474-11.8789)2/10.4167(1/6 +1/6) = 0.8402/3.4726= 0.2420 < 10.479 F for C vs D = (13.8333-21.4167)2/10.4767(1/6 +1/6) = 57.5080/3.4726 = 16.5605 > 10.479 Significant Difference between A and B !! Significant Difference between A and D !! Significant Difference between B and C !! Significant Difference between C and D !!

Counter Checking with SPSS Using the same Data Set

The overall results are identical with that in Excel output previously:-

Proving that the Excel result are exactly equal to that in SPSS!! Although the Excel result for the Scheffe test well matched that in SPSS, this might not be enough to prove the figures they got are absolutely the same! . Unlike in Tukey’s Test that ‘Critical Differences’ are can be used for counter checking to 95% Conf. Int. of the SPSS output. But we can simply use another method by checking the ‘F-Table’ backward.

For Proving that the Excel result is consistent with SPSS!! F for A vs B = (13.9167-22.3333)2/10.4167(1/10 +1/10) SPSS sign. 20.3977/ 3 = 6.7992 0.006 (Group Num. -1) F for A vs D = (13.9167-21.4167)2/10.4167(1/10 +1/0) 16.1980/3 = 5.3993 0.014 F for B vs C = (22.3333-13.8333)2/10.4167(1/10 +1/10) 20.8057/ 3 = 6.3952 0.006 F for C vs D = (13.8333-21.4167)2/10.4767(1/10 +1/10) 16.5605/3 = 5.5202 0.013

The Significance checked from F value from Excel well matched that in the SPSS output!!

Conclusion After activating the ‘Analysis TookPak’ Add-in in Excel, we can have useful statistical tests to use including different Anova tests. We find that, if overall Anova result is significant, we can work further in Excel to run Post Hoc Test e.g. Scheffe’s Test to find where the mean differences exist, not too difficultly! We find that this is not only possible in One-way Anova, but even in Two-way Anova, such as aXb factorial tests!!

Thank You very much!