Post Hoc Tests

What is a Post Hoc Test? Review: What makes a test Post Hoc? Adjusting Alpha Level Multiple A Priori Comparisons What makes a test Post Hoc? Many tests could be Post Hoc… But, there are set Post Hoc tests

Studentized Range Statistic q

Studentized Range Statistic q Independent Groups Largest mean Smallest mean Example 8.2 11.8 Fail to Reject Note: arrange means in ascending order! = 3.77 critical value

Studentized Range Statistic q q’s can tell us where differences are (more specific than F) Solving q’s is just like solving t’s But we solve a lot of q’s… can we speed things up?

Solving for the Smallest Significant Difference Example 8.2 11.8 = 3.77 critical value

Solving for the Smallest Significant Difference Solving for the smallest significant difference will help us make quicker comparisons But we still need a way to organize things nicely…

Newman-Kewls smallest difference required was 6.61 smallest significant difference If 2 steps Example T1 T2 T3 r 3.6 3 6.61 2 5.41 8.2 11.8

A Better Newman-Kewls Example 2 3 9 10

A Better Newman-Kewls Example * T1 T2 T3 T4 T5 r 1 7 8 5 4.04 6 4 3.79 3 3.44 2 2.86 Read Right to Left UNTIL 1. The row is completed 2. A nonsignificant difference is found 3. Reaching a column which was nonsignificant on the previous row

Newman-Kewls Summarized Newman-Kewls tables help organize your q’s When doing a set of post hoc comparisons it’s best to use a Newman-Kewls table

Unequal N’s Tukey-Kramer Replace with

Unequal N’s * * Behrens-Fisher Each particular pairing of means must be examined with a different critical value and their own Thus, the smallest significant difference will vary even for a given

Using q’s will give use more Type I Errors A Problem with q So Far… Why are we doing q’s anyway? Why not do t’s instead? But is q really controlling our alpha level? NO! Using q’s will give use more Type I Errors

What Happens to Alpha Level? Power? Trying to fix q Tukey's HSD Tukey's WSD N-K except If there are 4 means, all differences are treated as 4 steps. r = # of steps between the two means to be compared. What Happens to Alpha Level? Power?

Use Tukey’s WSD, not normal method for q Tukey’s HSD and WSD T1 T2 T3 T4 T5 r 1 7 8 5 4.04 6 4 3.79 3 3.44 2 2.86 Use Tukey’s WSD, not normal method for q

Back to Post Hoc in General What is a post hoc test again? What are the real issues with Post Hoc tests? Alpha and Power… q is just one type of post hoc (one way to balance alpha and power), what are others?

Dunnett’s Control vs. Treatment run standard and use or, solve for critical difference (CV) Example Go to Table for * Pros…? Cons…?

Sheffé’s Test Linear contrast MS(contrast) F = To evaluate 1) consult F table and find critical value F.05 (k-1, dferror) (CV) F = MS(error) 2) multiply CV by (k-1). (new CV) It sets the family-wise Type-I Error rate ( in our case) for ALL possible linear contrasts, not merely the pair-wise comparisons. Don’t use when only doing pair-wise, because it will be overly conservative.

Post Hoc Summary When to use what… q in most situations… but use Tukey’s WSD for critical value Put things in a Newman-Kewls table when N’s are unequal, use Tukey’s correction Dunnett’s when you have one control and multiple treatments Sheffé’s ONLY when you are doing complex comparisons (i.e., contrasts)

Post Hoc Summary Be aware of the alpha level and power issues… Why can’t we have a perfect test (i.e., low alpha level and high power)? How does Tukey’s WSD and HSD relate to this? How does Dunnett’s relate to this? How does Sheffé’s relate to this?