Post Hoc Tests

What is a Post Hoc Test? Review: – Adjusting Alpha Level – Multiple A Priori Comparisons What makes a test Post Hoc? – Many tests could be Post Hoc… But, there are set Post Hoc tests

Studentized Range Statistic q

Independent Groups Smallest mean Largest mean Example critical value = 3.77 Fail to Reject Note: arrange means in ascending order!

Studentized Range Statistic q q’s can tell us where differences are (more specific than F) Solving q’s is just like solving t’s But we solve a lot of q’s… can we speed things up?

Solving for the Smallest Significant Difference Example critical value = 3.77

Solving for the Smallest Significant Difference Solving for the smallest significant difference will help us make quicker comparisons But we still need a way to organize things nicely…

Newman-Kewls T1T1 T2T2 T3T3 r T1T T2T T3T3 smallest difference required was 6.61 If 2 steps smallest significant difference Example

A Better Newman-Kewls Example Example

A Better Newman-Kewls Example T1T1 T2T2 T3T3 T4T4 T5T5 r T1T T2T T3T T4T T5T5 T1T1 T2T2 T3T3 T4T4 T5T5 T1T1 ** T2T2 ** T3T3 ** T4T4 T5T5 Read Right to Left UNTIL 1.The row is completed 2.A nonsignificant difference is found 3.Reaching a column which was nonsignificant on the previous row

Newman-Kewls Summarized Newman-Kewls tables help organize your q’s When doing a set of post hoc comparisons it’s best to use a Newman-Kewls table

Unequal N’s Tukey-Kramer Replace with

Unequal N’s Behrens-Fisher * Each particular pairing of means must be examined with a different critical value and their own Thus, the smallest significant difference will vary even for a given *

A Problem with q So Far… Why are we doing q’s anyway? Why not do t’s instead? But is q really controlling our alpha level? NO! Using q’s will give use more Type I Errors

Trying to fix q Tukey's HSD N-K except If there are 4 means, all differences are treated as 4 steps. r = # of steps between the two means to be compared. Tukey's WSD What Happens to Alpha Level? Power?

Tukey’s HSD and WSD T1T1 T2T2 T3T3 T4T4 T5T5 r T1T T2T T3T T4T T5T5 Use Tukey’s WSD, not normal method for q

Back to Post Hoc in General What is a post hoc test again? – What are the real issues with Post Hoc tests? Alpha and Power… q is just one type of post hoc (one way to balance alpha and power), what are others?

Dunnett’s Control vs. Treatment run standardand useor, solve for critical difference (CV) Go to Table for * Example Pros…? Cons…?

Sheffé’s Test Linear contrast MS (contrast) MS (error) F = To evaluate 1)consult F table and find critical value F.05 (k- 1, df error ) (CV) 2)multiply CV by (k-1). (new CV) It sets the family-wise Type-I Error rate ( in our case) for ALL possible linear contrasts, not merely the pair-wise comparisons. Don’t use when only doing pair-wise, because it will be overly conservative.

Post Hoc Summary When to use what… – q in most situations… but use Tukey’s WSD for critical value Put things in a Newman-Kewls table when N’s are unequal, use Tukey’s correction – Dunnett’s when you have one control and multiple treatments – Sheffé’s ONLY when you are doing complex comparisons (i.e., contrasts)

Post Hoc Summary Be aware of the alpha level and power issues… – Why can’t we have a perfect test (i.e., low alpha level and high power)? – How does Tukey’s WSD and HSD relate to this? – How does Dunnett’s relate to this? – How does Sheffé’s relate to this?