1. Post Hoc Tests

2. What is a Post Hoc Test? Review: What makes a test Post Hoc?
Adjusting Alpha Level
Multiple A Priori Comparisons
What makes a test Post Hoc?
Many tests could be Post Hoc… But, there are set Post Hoc tests

3. Studentized Range Statistic q

4. Studentized Range Statistic q
Independent Groups
Largest mean
Smallest mean
Example
8.2
11.8
Fail to Reject
Note: arrange means in ascending order!
= 3.77
critical value

5. Studentized Range Statistic q
q’s can tell us where differences are (more specific than F)
Solving q’s is just like solving t’s
But we solve a lot of q’s… can we speed things up?

6. Solving for the Smallest Significant Difference
Example
8.2
11.8
= 3.77
critical value

7. Solving for the Smallest Significant Difference
Solving for the smallest significant difference will help us make quicker comparisons
But we still need a way to organize things nicely…

8. Newman-Kewls smallest difference required was 6.61
smallest significant difference
If 2 steps
Example T1 T2 T3 r
3.6 3
6.61 2
5.41
8.2
11.8

9. A Better Newman-Kewls Example2 3 9 10

10. A Better Newman-Kewls Example* T1 T2 T3 T4 T5 r 1 7 8 5
4.04 6 4
3.79 3
3.44 2
2.86
Read Right to Left
UNTIL 1.
The row is completed 2.
A nonsignificant difference is found 3.
Reaching a column which was nonsignificant on the previous row

11. Newman-Kewls Summarized
Newman-Kewls tables help organize your q’s
When doing a set of post hoc comparisons it’s best to use a Newman-Kewls table

12. Unequal N’s
Tukey-Kramer
Replace
with

13. Unequal N’s * * Behrens-Fisher
Each particular pairing of means must be examined with a different critical value and their own
Thus, the smallest significant difference will vary even for a given

14. Using q’s will give use more Type I Errors
A Problem with q So Far…
Why are we doing q’s anyway? Why not do t’s instead?
But is q really controlling our alpha level?
NO!
Using q’s will give use more Type I Errors

15. What Happens to Alpha Level? Power?
Trying to fix q
Tukey's HSD
Tukey's WSD
N-K except
If there are 4 means, all differences are treated as 4 steps.
r = # of steps between the two
means to be compared.
What Happens to Alpha Level? Power?

16. Use Tukey’s WSD, not normal method for q
Tukey’s HSD and WSD T1 T2 T3 T4 T5 r 1 7 8 5
4.04 6 4
3.79 3
3.44 2
2.86
Use Tukey’s WSD, not normal method for q

17. Back to Post Hoc in General
What is a post hoc test again?
What are the real issues with Post Hoc tests? Alpha and Power…
q is just one type of post hoc (one way to balance alpha and power), what are others?

18. Dunnett’s Control vs. Treatment
run standard
and use
or, solve for critical difference (CV)
Example
Go to Table for *
Pros…? Cons…?

19. Sheffé’s Test Linear contrast MS(contrast) F = To evaluate 1)
consult F table and find critical value F.05 (k-1, dferror) (CV)
F =
MS(error) 2)
multiply CV by (k-1). (new CV)
It sets the family-wise Type-I Error rate ( in our case) for ALL possible linear contrasts, not merely the pair-wise comparisons.
Don’t use when only doing pair-wise, because it will be overly conservative.

20. Post Hoc Summary When to use what…
q in most situations… but use Tukey’s WSD for critical value
Put things in a Newman-Kewls table
when N’s are unequal, use Tukey’s correction
Dunnett’s when you have one control and multiple treatments
Sheffé’s ONLY when you are doing complex comparisons (i.e., contrasts)

21. Post Hoc Summary Be aware of the alpha level and power issues…
Why can’t we have a perfect test (i.e., low alpha level and high power)?
How does Tukey’s WSD and HSD relate to this?
How does Dunnett’s relate to this?
How does Sheffé’s relate to this?