1. Significance and Meaningfulness Effect Sizes

2. KNR 445 Statistics Effect sizes Slide 2 Significance vs. meaningfulness  Is your significant difference a real difference?

3. KNR 445 Statistics Effect sizes Slide 3 Significance vs. meaningfulness  Is your significant difference a real difference?

4. KNR 445 Statistics Effect sizes Slide 4 Significance vs. meaningfulness  Statistical Power

5. KNR 445 Statistics Effect sizes Slide 5 Significance vs. meaningfulness  Statistical Power  Smaller difference between means reduces power  Larger SE M reduces power

6. KNR 445 Statistics Effect sizes Slide 6 Significance vs. meaningfulness  Statistical Power  Smaller  reduces power

7. KNR 445 Statistics Effect sizes Slide 7 Significance vs. meaningfulness  As sample size increases, likelihood of significant difference increases The fact that this sample size is buried down here in the denominator of the test statistic means that as n  , p  0. So if your sample is big enough, it will generate significant results

8. KNR 445 Statistics Effect sizes Slide 8 Significance vs. meaningfulness  As sample size increases, likelihood of significant difference increases  So statistical difference does not always mean important difference  What to do about this?  Calculate a measure of the difference that is standardized to be expressed in terms of the variability in the 2 samples  = EFFECT SIZE

9. KNR 445 Statistics Effect sizes Slide 9 Significance vs. meaningfulness  EFFECT SIZE - FORMULA

10. KNR 445 Statistics Effect sizes Slide 10 Significance vs. meaningfulness  EFFECT SIZE – from SPSS  Using appendix B data set 2, and submitting DV salary to test of difference across gender, gives the following output (squashed here to fit): T-Test

11. KNR 445 Statistics Effect sizes Slide 11 Significance vs. meaningfulness  EFFECT SIZE – from SPSS T-Test SD’s to pool Mean difference to use

12. KNR 445 Statistics Effect sizes Slide 12 Significance vs. meaningfulness  EFFECT SIZE – from SPSS So…

13. KNR 445 Statistics Effect sizes Slide 13 Significance vs. meaningfulness  EFFECT SIZE – from SPSS Substituting…

14. KNR 445 Statistics Effect sizes Slide 14 Significance vs. meaningfulness  EFFECT SIZE – from SPSS Calculating…

15. KNR 445 Statistics Effect sizes Slide 15 Significance vs. meaningfulness  From Cohen, 1988:  d =.20 is small  d =.50 is moderate  d =.80 is large  So our effect size of.25 is small, and concurs on this occasion with the insignificant result  The finding is both insignificant and small  (a pathetic, measly, piddling little difference of no consequence whatsoever – trivial and beneath us)

16. Statistical Power Maximizing the likelihood of significance

17. KNR 445 Statistics Effect sizes Slide 17 Statistical Power  The likelihood of getting a significant relationship when you should (i.e. when there is a relationship in reality)  Recall from truth table, power = 1 -  (  = type II error)

18. KNR 445 Statistics Effect sizes Slide 18 Factors Affecting Statistical Power The big ones:  Effect size (bit obvious)  Select samples such that difference between them is maximized  Sample size  Most important: as n increases, SE M decreases, and test statistic then increases

19. KNR 445 Statistics Effect sizes Slide 19 Factors Affecting Statistical Power The others:  Level of significance  Smaller , less power  Larger , more power  1-tailed vs. 2-tailed tests  With good a priori info (i.e. research literature), selecting 1-tailed test increases power  Dependent samples  Correlation between samples reduces standard error, and thus increases test statistic

20. KNR 445 Statistics Effect sizes Slide 20 Calculating sample size a priori 1. Specify effect size 2. Set desired level of power 3. Enter values for effect size and power in appropriate table, and generate desired sample size:  Applet for calculating sample size based on above: http://www.stat.uiowa.edu/~rlenth/Power/  Applets for seeing power acting (and interacting) with sample size, effect size, etc… http://statman.stat.sc.edu/~west/applets/power.html http://acad.cgu.edu/wise/power/powerapplet1.html http://www.stat.sc.edu/%7Eogden/javahtml/power/power.html